Supporting Interoperability Between Open-Source Search Engines with the Common Index File Format

    Thumbnail
    Citations
    Altmetric
    Author
    Lin, J; Mackenzie, J; Kamphuis, C; Macdonald, C; Mallia, A; Siedlaczek, M; Trotman, A; de Vries, A
    Date
    2020-07-25
    Source Title
    Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval
    Publisher
    ACM
    University of Melbourne Author/s
    Mackenzie, Joel
    Affiliation
    Computing and Information Systems
    Metadata
    Show full item record
    Document Type
    Conference Paper
    Citations
    Lin, J., Mackenzie, J., Kamphuis, C., Macdonald, C., Mallia, A., Siedlaczek, M., Trotman, A. & de Vries, A. (2020). Supporting Interoperability Between Open-Source Search Engines with the Common Index File Format. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.2149-2152. ACM. https://doi.org/10.1145/3397271.3401404.
    Access Status
    Access this item via the Open Access location
    URI
    http://hdl.handle.net/11343/254112
    DOI
    10.1145/3397271.3401404
    Open Access URL
    Accepted version
    Abstract
    There exists a natural tension between encouraging a diverse ecosystem of open-source search engines and supporting fair, replicable comparisons across those systems. To balance these two goals, we examine two approaches to providing interoperability between the inverted indexes of several systems. The first takes advantage of internal abstractions around index structures and building wrappers that allow one system to directly read the indexes of another. The second involves sharing indexes across systems via a data exchange specification that we have developed, called the Common Index File Format (CIFF). We demonstrate the first approach with the Java systems Anserini and Terrier, and the second approach with Anserini, JASSv2, OldDog, PISA, and Terrier. Together, these systems provide a wide range of implementations and features, with different research goals. Overall, we recommend CIFF as a low-effort approach to support independent innovation while enabling the types of fair evaluations that are critical for driving the field forward.

    Export Reference in RIS Format     

    Collections