University Library
  • Login
A gateway to Melbourne's research publications
Minerva Access is the University's Institutional Repository. It aims to collect, preserve, and showcase the intellectual output of staff and students of the University of Melbourne for a global audience.
View Item 
  • Minerva Access
  • Medicine, Dentistry & Health Sciences
  • Melbourne School of Psychological Sciences
  • Melbourne School of Psychological Sciences - Theses
  • View Item
  • Minerva Access
  • Medicine, Dentistry & Health Sciences
  • Melbourne School of Psychological Sciences
  • Melbourne School of Psychological Sciences - Theses
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

    Contemporary methods for identifying and leveraging expertise in collective decision-making

    Thumbnail
    Citations
    Altmetric
    Author
    Martinie, Marcellin Ferdinand Ruxuan Carlos
    Date
    2020
    Affiliation
    Melbourne School of Psychological Sciences
    Metadata
    Show full item record
    Document Type
    PhD thesis
    Access Status
    This item is embargoed and will be available on 2022-09-21.
    URI
    http://hdl.handle.net/11343/243001
    Description

    © 2020 Marcellin Ferdinand Ruxuan Carlos Martinie

    Abstract
    From company board committees to grand juries to political parties, decision making often involves a group of individuals with differing views on what the best decision may be. Individuals have different views, in part, because some individuals have more knowledge than others. In theory, decision makers can make better decisions by leveraging the expertise of individuals in the crowd by identifying those who have more knowledge than others. Unfortunately, it is often difficult to identify and leverage expertise in practice. Decision makers often have no records of individuals' past performance from which they can estimate expertise, and subjective measures such as confidence and self-ratings of expertise are often considered as unreliable. In this thesis, I demonstrate how individuals' meta-predictions -- predictions about what other individuals will predict -- can be used to identify and leverage expertise in the crowd on 'single-question' problems, where records on individuals' past performance are unavailable. I first examine how a recent algorithm in the literature can be used to distinguish between subsets of high-performing and low-performing individuals in the crowd on binary decision problems. I show that this algorithm is in fact weighting individuals by the absolute difference between an individual's vote and their meta-prediction about the votes of others, and thus this weighting metric can be used to identify and leverage expertise in the crowd. I develop an improved weighting approach that uses individuals' probabilistic forecasts and their meta-predictions about the average probability forecast of others, and show that this outperforms the top alternative probabilistic aggregation approaches in the literature on a large range of decision problems. Furthermore, I demonstrate that this improved weighting approach provides a superior measure of expertise than existing single-question approaches to identifying expertise in the literature. As an additional test, I compare this single-question approach with cross-domain weighting -- weighting by individuals' performance on problems on unrelated domains -- and show that cross-domain weighting is favoured over single-question approaches in cases where it is possible to obtain individuals' past performance on problems -- even if those questions are from unrelated domains. In general, our results demonstrate the potential for using individuals' meta-predictions and performance on problems from unrelated domains to identify expertise in cases where other approaches might be ineffective or unavailable.
    Keywords
    wisdom of crowds; forecast aggregation; decision making; identifying expertise; meta-cognitive knowledge

    Export Reference in RIS Format     

    Endnote

    • Click on "Export Reference in RIS Format" and choose "open with... Endnote".

    Refworks

    • Click on "Export Reference in RIS Format". Login to Refworks, go to References => Import References


    Collections
    • Minerva Elements Records [45689]
    • Melbourne School of Psychological Sciences - Theses [294]
    Minerva AccessDepositing Your Work (for University of Melbourne Staff and Students)NewsFAQs

    BrowseCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects
    My AccountLoginRegister
    StatisticsMost Popular ItemsStatistics by CountryMost Popular Authors