30 October 2007

Query the obscure

Be the first to share

Query the obscure
Gaining a better understanding of queries is a top priority of the search industry.

When it comes to common searches that repeat millions of times like “Britney Spears” or “Hybrid Cars,” returning the most appropriate results, or advertisements, is not difficult. But what about queries that are exceptionally rare and may never repeat more than a single time? Clearly, these queries are infinitely harder for the search engine to understand.

Andrei Broder, Yahoo! Research Fellow and Vice President of Search Technology and Computational Advertising, and a team of Yahoo! researchers set out to tackle this problem. Their work is outlined in a paper called Robust Classification of Rare Queries Using Web Knowledge, that appeared in SIGIR 2007.

To address the problem, the Yahoo! team proposed a methodology for using search results, as well as information available on the Web, as a source of external knowledge. To this end, they sent rare queries to a search engine and assumed that a majority of the highest-ranking search results were relevant to the query. Categorizing these results allowed the team to classify the original query with high accuracy.

The results definitively confirmed that using the Web as a repository of world knowledge contributes valuable information about the query, and aids in its correct classification. “We discovered the best source of information to understand what these rare queries are about is to look at the search results,” Broder explains. “If you look at each returned page as a vote on what the query is about, you find that the majority tends to be correct even though many individual pages are wrong.”

Read full story
Download paper

Be the first to share
19 January 2017
Don Norman on what Apple, Google and Tesla get wrong
Don Norman is a technological optimist. The author of The Design of Everyday Things and head of UC San Diego's Design Lab believes that artificial intelligence might only take the worst parts of our jobs, …
23 December 2016
Innovative PhD dissertations at Harvard use ethnographic methods to understand organizational culture
Three innovative PhD dissertations have been published at the Harvard Business School's Doctoral Program on Organizational Behavior that use ethnography and qualitative research to understand and map organizational culture: Curtis K. Chan Chan’s research (The Double-Edged Sword …
18 December 2016
Why UX research is business research
Libby Maurer, UX leader at Widen Enterprise (a content technology company specializing in digital asset management software and content production services), makes a strong case for the value of UX research in a hardcore business …
14 December 2016
Experience design and Big Data at BBVA bank
Over the last 6 months, Fabien Girardin has been working in a rather unique position at BBVA Data & Analytics, a center of excellence in financial data analysis. My job is to make the design …
17 November 2016
[Book] Collaborative Ethnography in Business Environments
Collaborative Ethnography in Business Environments Edited by Maryann McCabe Routledge 2017, 138 pages In a global and rapidly changing commercial environment, businesses increasingly use collaborative ethnographic research to understand what motivates their employees and what their customers value. In …
15 November 2016
[Book] Sensemaking
Sensemaking: The Power of the Humanities in the Age of the Algorithm by Christian Madsbjerg Hachette Books March 2017, 256 pages Inspired by his work with companies like Ford and Coca-Cola, Madsbjerg's Sensemaking is a provocative stand against the …
14 November 2016
Billions spent on cyber security tech while human error ignored
Billions spent on cyber security tech while human error ignored, writes Misha Glenny in the Financial Times. Armies of zombie computers threaten us all. Despite the headlines, the cyber security industry has prioritised the development of …
8 November 2016
[Book] Woo, Wow, and Win: Service Design, Strategy, and the Art of Customer Delight
Woo, Wow, and Win: Service Design, Strategy, and the Art of Customer Delight by Thomas A. Stewart and Patricia O’Connell Harper Business November 2016, 336 pages In this pioneering guide, two business authorities introduce the new discipline of Service …

We are an international experience design consultancy helping companies and organisations to innovate their products, services and processes by putting people and their experiences first.

12 January 2017
Experientia’s CITYOPT project awarded prestigious French award for its sustainable development design

Following the 2016 Smart Innovation Award at “FIMBACTE Trophées du Cadre de vie”, the CITYOPT project has once again been recognized, this time in the prestigious French design competition: “Observeur du Design 2017”, in the Service Design category. In June 2016, CITYOPT won the first stage of the Observeur du Design. Now the project has […]

1 December 2016
More on upcoming conference on design & sustainable innovation for smart cities

Last month Putting People First announced the upcoming conference on design & sustainable innovation for smart cities in Nice France. Meanwhile we are pleased to announce the full event agenda (see below). This event will feature professionals from leading research institutes and industry gathering to present key initiatives which combine Energy Efficiency and Service Design […]

29 October 2016
Upcoming conference on design & sustainable innovation for smart cities

Invitation to the International Conference on Design & Sustainable Innovation for SmartCities Nice (France) 8 December 2016 On the 8th December 2016, the CITYOPT project will host an international conference on Design and sustainable innovation for SmartCities, at the Centre Universitaire Méditerranéen, France. An open invitation to attend is offered to people and organisations who […]

28 October 2016
Experientia’s President, Michele Visciòla, panel judge for MacArthur Foundation’s “100&Change” competition

The 100&Change is an international competition and a landmark opportunity for thinkers and designers to tackle critical challenges affecting the world. Michele Visciòla will be one of the panel of expert judges who will select which project is worthy of the $100 million grant. 100&Change is the MacArthur Foundation competition – launched this year for […]

5 September 2016
Great engine, but the fuel seems poor. Discussing insight development in corporate marketing

The September issue of the Harvard Business Review (HBR) contains a lengthy essay, entitled Building an Insights Engine, on how Unilever has created the organizational capabilities to “transform data into insights about consumers’ motivations and to turn those insights into strategy.” The article was written by Frank van den Driest and Keith Weed of a […]

29 August 2016
Experientia discussing ethnography and patient-centricity at EPIC 2016

This week Experientia joins our colleagues and peers in Minneapolis at EPIC 2016, the premier international gathering on ethnography and design in industry. The theme for the conference this year is Pathmaking, emphasizing the power of ethnography to create transformative innovation, growth and strategic success for companies, industries and communities. On the second day of […]

See all articles