Information Retrieval Evaluation
- ISBN-10
- 3031022769
- ISBN-13
- 9783031022760
- Category
- Computers
- Pages
- 107
- Language
- English
- Published
- 2022-05-31
- Publisher
- Springer Nature
- Author
- Donna Harman
Description
Evaluation has always played a major role in information retrieval, with the early pioneers such as Cyril Cleverdon and Gerard Salton laying the foundations for most of the evaluation methodologies in use today. The retrieval community has been extremely fortunate to have such a well-grounded evaluation paradigm during a period when most of the human language technologies were just developing. This lecture has the goal of explaining where these evaluation methodologies came from and how they have continued to adapt to the vastly changed environment in the search engine world today. The lecture starts with a discussion of the early evaluation of information retrieval systems, starting with the Cranfield testing in the early 1960s, continuing with the Lancaster "user" study for MEDLARS, and presenting the various test collection investigations by the SMART project and by groups in Britain. The emphasis in this chapter is on the how and the why of the various methodologies developed. The second chapter covers the more recent "batch" evaluations, examining the methodologies used in the various open evaluation campaigns such as TREC, NTCIR (emphasis on Asian languages), CLEF (emphasis on European languages), INEX (emphasis on semi-structured data), etc. Here again the focus is on the how and why, and in particular on the evolving of the older evaluation methodologies to handle new information access techniques. This includes how the test collection techniques were modified and how the metrics were changed to better reflect operational environments. The final chapters look at evaluation issues in user studies -- the interactive part of information retrieval, including a look at the search log studies mainly done by the commercial search engines. Here the goal is to show, via case studies, how the high-level issues of experimental design affect the final evaluations. Table of Contents: Introduction and Early History / "Batch" Evaluation Since 1992 / Interactive Evaluation / Conclusion
Get the book
Other editions
Similar books
-
Family History Digital Libraries
By William Sims BainbridgeOne named Sara and Timberlake had 11 male workers, 1 female worker, and 4 children workers, so it might have employed the Minor family.
-
Foundation Dreamweaver MX
By Craig Grannell, Jerome Turner, Matt StephensSo here's what we need to do to arrive at our layout: s Create the main table to hold all the page elements. s Deal with the navigation area which is ...
-
Cisco CCNA Certification, 2 Volume Set: Exam 200-301
By Todd LammleThis inclusive, two-book set provides what you need to know to succeed on the new CCNA exam. The set includes Understanding Cisco Networking Technologies: Volume 1 and the CCNA Certification Study Guide: Volume 2.
-
CompTIA Network+ Study Guide: Exam N10-006
By Todd Lammle... you can use: –a –A –c –n –r –R –S –s All nbtstat switches are case sensitive. Generally speaking, lowercase switches deal with NetBIOS names of hosts, ...
-
CompTIA Network+ Study Guide with Online Labs: N10-007 Exam
By Todd Lammle, Jon Buhagiar... you can use: –a –A –c –n –r –R –S –s All nbtstat switches are case sensitive. Generally speaking, lowercase switches deal with NetBIOS names of hosts, ...
-
CCNA: Cisco Certified Network Associate FastPass
By Todd LammleS The S reference point defines the point between the customer router and an ... with the letter E deal with using ISDN on the existing telephone network.
-
Stranger in the Chat Room
By Todd Hafer, Jedd HaferA sequel to In the Chat Room With God finds a group of teens contacted by a mysterious and increasingly malevolent character who claims to know about their encounters with the Almighty and challenges their beliefs. Original.
-
Error Correction Coding: Mathematical Methods and Algorithms
By Todd K. MoonM M−1∑ k=0 −∞ ∞ k=0 The average energy per signal E s ∫ can be related to the ... we will deal primarily with additive white Gaussian noise (AWGN), ...
-
Security+ Training Guide
By Todd King... to deal with most , but unfortunately not all , of these potential threats . ... The S / MIME standard implements encryption for message content using ...
-
CCDA: Cisco Certified Design Associate Study Guide: Exam 640-861
By Todd Lammle, Andy BarklS reference point The S reference point defines the reference point between ... with the letter E deal with using ISDN on the existing telephone network.