Rank-Biased Precision for Measurement of Retrieval Effectiveness
Author
Moffat, A; Zobel, JDate
2009-01-01Source Title
ACM Transactions on Information SystemsPublisher
ASSOC COMPUTING MACHINERYAffiliation
Computer Science and Software EngineeringMetadata
Show full item recordDocument Type
Journal ArticleCitations
Moffat, A. & Zobel, J. (2009). Rank-Biased Precision for Measurement of Retrieval Effectiveness. ACM TRANSACTIONS ON INFORMATION SYSTEMS, 27 (1), https://doi.org/10.1145/1416950.1416952.Access Status
This item is currently not available from this repositoryAbstract
<jats:p>
A range of methods for measuring the effectiveness of information retrieval systems has been proposed. These are typically intended to provide a quantitative single-value summary of a document ranking relative to a query. However, many of these measures have failings. For example, recall is not well founded as a measure of satisfaction, since the user of an actual system cannot judge recall. Average precision is derived from recall, and suffers from the same problem. In addition, average precision lacks key stability properties that are needed for robust experiments. In this article, we introduce a new effectiveness metric,
<jats:italic>rank-biased precision</jats:italic>
, that avoids these problems. Rank-biased pre-cision is derived from a simple model of user behavior, is robust if answer rankings are extended to greater depths, and allows accurate quantification of experimental uncertainty, even when only partial relevance judgments are available.
</jats:p>
Keywords
Information SystemsExport Reference in RIS Format
Endnote
- Click on "Export Reference in RIS Format" and choose "open with... Endnote".
Refworks
- Click on "Export Reference in RIS Format". Login to Refworks, go to References => Import References