Information Retrieval Evaluation
- ISBN-10
- 1598299727
- ISBN-13
- 9781598299724
- Category
- Technology & Engineering
- Pages
- 119
- Language
- English
- Published
- 2011-06-06
- Publisher
- Morgan & Claypool Publishers
- Author
- Donna Harman
Description
Evaluation has always played a major role in information retrieval, with the early pioneers such as Cyril Cleverdon and Gerard Salton laying the foundations for most of the evaluation methodologies in use today. The retrieval community has been extremely fortunate to have such a well-grounded evaluation paradigm during a period when most of the human language technologies were just developing. This lecture has the goal of explaining where these evaluation methodologies came from and how they have continued to adapt to the vastly changed environment in the search engine world today. The lecture starts with a discussion of the early evaluation of information retrieval systems, starting with the Cranfield testing in the early 1960s, continuing with the Lancaster "user" study for MEDLARS, and presenting the various test collection investigations by the SMART project and by groups in Britain. The emphasis in this chapter is on the how and the why of the various methodologies developed. The second chapter covers the more recent "batch" evaluations, examining the methodologies used in the various open evaluation campaigns such as TREC, NTCIR (emphasis on Asian languages), CLEF (emphasis on European languages), INEX (emphasis on semi-structured data), etc. Here again the focus is on the how and why, and in particular on the evolving of the older evaluation methodologies to handle new information access techniques. This includes how the test collection techniques were modified and how the metrics were changed to better reflect operational environments. The final chapters look at evaluation issues in user studies -- the interactive part of information retrieval, including a look at the search log studies mainly done by the commercial search engines. Here the goal is to show, via case studies, how the high-level issues of experimental design affect the final evaluations. Table of Contents: Introduction and Early History / "Batch" Evaluation Since 1992 / Interactive Evaluation / Conclusion
Get the book
Other editions
Similar books
-
Animal Sciences: The Biology, Care, and Production of Domestic Animals, Fourth Edition
By John R. Campbell, M. Douglas Kenealy, Karen L. CampbellTimberlake claimed in 1980 that a fundamental problem with Singer's work is the lack of an adequate definition of suffering ...
-
Resilient Agriculture: Cultivating Food Systems for a Changing Climate
By Laura Lengnick3. D. Layne. 2013. Tree Fruit: Protecting Your Investment. American/Western Fruit Grower, September/October. 4. R. Snyder and J. Melu-Abreu. 2005. Frost ...
-
Biodesign: The Process of Innovating Medical Technologies
By Stefanos Zenios, Josh Makower, Paul YockAt that time, these were in the low $10s of millions. ... be a good partner going forward, even though it takes longer to get the deal done," offered Chess.
-
Advanced Signal Processing: A Concise Guide
By Amir-Homayoon Najmi, Todd Moon[ 59 ] S. Kotz , T. J. Kozubowski , and K. Podgorski , The Laplace ... valued signal processing : The proper way to deal with impropriety , ” IEEE Trans .
-
Resources for Technical Communication
By Pearson Education, Pearson Education Staff, David P Pearson EducationSome documents are annotated; some are left without annotations to provide more flexibility for instructors. This booklet can be packaged at no additional cost with any Longman title in technical communication.
-
Chemistry: An Introduction to General, Organic, and Biological Chemistry; Chemistry Study Pack Version 2.0 CD-ROM; The Chemistry of Life CD-ROM;...
By Karen C. TimberlakeChemistry: An Introduction to General, Organic, and Biological Chemistry; Chemistry Study Pack Version 2.0 CD-ROM; The Chemistry of Life CD-ROM;...
-
Poultry Science: Fifth Edition
By Colin G. Scanes, Karen D. ChristensenThe emission rates for ammonia (Casey et al., 2006): • Layers: 116 g NH3 per AU (AU or animal unit or 500 kg). • Broilers: 135 g NH3 per AU (AU or animal unit or 500 kg). Emission rates in different reports vary from less than either 10 ...
-
Solid State Materials Chemistry
By Patrick M. Woodward, Pavel Karen, John S. O. Evans[45] B.F. Hoskins, R. Robson, “Design and construction of a new class of scaffolding-like materials comprising infinite polymeric frameworks of 3D-linked molecular rods. A reappraisal of the zinc cyanide and cadmium cyanide structures ...
-
Abbreviations Dictionary
By Dean A. Stahl, Karen Landen... Tallest Mountain Mount Robson—12,972 feet or 3,954 meters—in the Canadian Rockies Canada's Westernmost City Dawson, Yukon Canada's Westernmost Point in Yukon Territory just east of Alaska's Demarcation Point Canary Islands' Largest ...
-
Agriculture: A Very Short Introduction
By Paul Brassley, Richard SoffeACCOUNTING Christopher Nobes ADVERTISING Winston Fletcher AFRICAN AMERICAN RELIGION Eddie S. Glaude Jr AFRICAN HISTORY ... Hugh Bowden ALGEBRA Peter M. Higgins AMERICAN HISTORY Paul S. Boyer AMERICAN IMMIGRATION David A. Gerber AMERICAN ...