TREC OpenSearch

Summary

TREC OpenSearch presented at ILPS Soos addresses fundamental issues in information retrieval evaluation methodology, advocating for user-centered evaluation approaches over traditional relevance assessment methods. The work critiques the low agreement rates in current relevance judgments and proposes a shift toward “sustainable evaluation” that incorporates actual user interactions and clicks rather than relying solely on expert annotations. This research contributes to the ongoing discussion about improving test collection methodologies in information retrieval by emphasizing the need for more user-centric and practically sustainable evaluation frameworks.