Welcome to the Relevance Voting Demo
You’ll be shown one search term and one document at a time. Your task: decide whether the document is relevant to the search term.
Each vote helps improve the system by:
- Updating a confidence-weighted relevance label for the document
- Reordering the priority queue so the highest-impact document is always shown next
Our updated labels help:
- Build a labeled dataset used to calculate evaluation metrics (precision@k, recall@k, MAP, NDCG)
- Compare ranking models and retrieval algorithms to see which systems return more relevant results (Next Page)
Shortcuts: Y=Yes, N=No, S=Skip, ←/→=adjust confidence, Enter=Next