ECIR 2016 in Padova will feature LiLa 2016, the first edition of our tutorial on Living Labs for Online Evaluation.

  • Anne Schuth, University of Amsterdam, The Netherlands
  • Krisztian Balog, University of Stavanger, Norway
  • Duration: Half-Day

Experimental evaluation was always central to Information Retrieval research. The field is increasingly moving towards online evaluation, which involves experimenting with real, unsuspecting users in their natural task environments, a so-called living lab. Specifically the recent introduction of the Living Labs for IR Evaluation initiative, which also runs as a lab at CLEF and TREC OpenSearch, made it possible for researchers to have direct access to such labs. With recent developments, we believe that online evaluation will be an exciting area to work on in the future. This half-day tutorial aims to provide a comprehensive overview of the underlying theory and complement it with practical guidance.