Interactive IR User Study Design, Evaluation, and Reporting

Description

Since user study design has been widely applied in search interactions and information retrieval (IR) systems evaluation studies, a deep reflection and meta-evaluation of interactive IR (IIR) user studies is critical for sharpening the instruments of IIR research and improving the reliability and validity of the conclusions drawn from IIR user studies. To this end, we developed a faceted framework for supporting user study design, reporting, and evaluation based on a systematic review of the state-of-the-art IIR research papers recently published in several top IR venues (n=462). Within the framework, we identify three major types of research focuses, extract and summarize facet values from specific cases, and highlight the under-reported user study components which may significantly affect the results of research. Then, we employ the faceted framework in evaluating a series of IIR user studies against their respective research questions and explain the roles and impacts of the underlying connections and "collaborations" among different facet values. Through bridging diverse combinations of facet values with the study design decisions made for addressing research problems, the faceted framework can shed light on IIR user study design, reporting, and evaluation practices and help students and young researchers design and assess their own studies.

Get the book

Amazon.com

Search the book

eBay.com

Search the book

Other editions

• 

  Interactive IR User Study Design, Evaluation, and Reporting

  • 2019-06-04
  • 93 pages
  • Paperback
  • Synthesis Lectures on Informat

Similar books

• 

  Family History Digital Libraries

  By William Sims Bainbridge

  One named Sara and Timberlake had 11 male workers, 1 female worker, and 4 children workers, so it might have employed the Minor family.

• 

  Foundation Dreamweaver MX

  By Craig Grannell, Jerome Turner, Matt Stephens

  So here's what we need to do to arrive at our layout: s Create the main table to hold all the page elements. s Deal with the navigation area which is ...

• 

  Cisco CCNA Certification, 2 Volume Set: Exam 200-301

  By Todd Lammle

  This inclusive, two-book set provides what you need to know to succeed on the new CCNA exam. The set includes Understanding Cisco Networking Technologies: Volume 1 and the CCNA Certification Study Guide: Volume 2.

• 

  CompTIA Network+ Study Guide: Exam N10-006

  By Todd Lammle

  ... you can use: –a –A –c –n –r –R –S –s All nbtstat switches are case sensitive. Generally speaking, lowercase switches deal with NetBIOS names of hosts, ...

• 

  CompTIA Network+ Study Guide with Online Labs: N10-007 Exam

  By Todd Lammle, Jon Buhagiar

  ... you can use: –a –A –c –n –r –R –S –s All nbtstat switches are case sensitive. Generally speaking, lowercase switches deal with NetBIOS names of hosts, ...

• 

  CCNA: Cisco Certified Network Associate FastPass

  By Todd Lammle

  S The S reference point defines the point between the customer router and an ... with the letter E deal with using ISDN on the existing telephone network.

• 

  Stranger in the Chat Room

  By Todd Hafer, Jedd Hafer

  A sequel to In the Chat Room With God finds a group of teens contacted by a mysterious and increasingly malevolent character who claims to know about their encounters with the Almighty and challenges their beliefs. Original.

• 

  Error Correction Coding: Mathematical Methods and Algorithms

  By Todd K. Moon

  M M−1∑ k=0 −∞ ∞ k=0 The average energy per signal E s ∫ can be related to the ... we will deal primarily with additive white Gaussian noise (AWGN), ...

• 

  Security+ Training Guide

  By Todd King

  ... to deal with most , but unfortunately not all , of these potential threats . ... The S / MIME standard implements encryption for message content using ...

• 

  CCDA: Cisco Certified Design Associate Study Guide: Exam 640-861

  By Todd Lammle, Andy Barkl

  S reference point The S reference point defines the reference point between ... with the letter E deal with using ISDN on the existing telephone network.