Compact features for detection of near-duplicates in distributed retrieval
Loading...
Citations
Scopus:
Altmetric:
Altmetric::
Download
Associated Records
Authors
File Licence
Editors
Date
2006-01-01
Source Title
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Journal Title
Journal ISSN
Volume Title
Publisher
SPRINGER-VERLAG BERLIN
Melbourne Authors
Affiliation
Chancellery Research
Document Type
Conference Paper
Subject Code
Citation
Bernstein, Y., Shokouhi, M. & Zobel, J. (2006). Compact features for detection of near-duplicates in distributed retrieval. Crestani, F (Ed.) Ferragina, P (Ed.) Sanderson, M (Ed.) Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 4209 LNCS, pp.110-121. SPRINGER-VERLAG BERLIN. https://doi.org/10.1007/11880561_10.
Access Status
This item is currently not available from this repository
Access URL
Open Access URL
Open Access at PMC
Linked Resource URL
NHMRC Grant Code
ARC Grant Code
Description
Abstract
In distributed information retrieval, answers from separate collections are combined into a single result set. However, the collections may overlap. The fact that the collections are distributed means that it is not in general feasible to prune duplicate and near-duplicate documents at index time. In this paper we introduce and analyze the grainy hash vector, a compact document representation that can be used to efficiently prune duplicate and near-duplicate documents from result lists. We demonstrate that, for a modest bandwidth and computational cost, many near-duplicates can be accurately removed from result lists produced by a cooperative distributed information retrieval system.
Notes
Terms of Use
Keywords
Information Systems