University Library
  • Login
A gateway to Melbourne's research publications
Minerva Access is the University's Institutional Repository. It aims to collect, preserve, and showcase the intellectual output of staff and students of the University of Melbourne for a global audience.
View Item 
  • Minerva Access
  • Medicine, Dentistry & Health Sciences
  • Melbourne School of Population and Global Health
  • Melbourne School of Population and Global Health - Research Publications
  • View Item
  • Minerva Access
  • Medicine, Dentistry & Health Sciences
  • Melbourne School of Population and Global Health
  • Melbourne School of Population and Global Health - Research Publications
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

    Setting health research priorities using the CHNRI method: V. Quantitative properties of human collective knowledge

    Thumbnail
    Download
    Published version (1.501Mb)

    Citations
    Scopus
    Web of Science
    Altmetric
    7
    6
    Author
    Rudan, I; Yoshida, S; Wazny, K; Chan, KY; Cousens, S
    Date
    2016-06-01
    Source Title
    Journal of Global Health
    Publisher
    UNIV EDINBURGH, GLOBAL HEALTH SOC
    University of Melbourne Author/s
    Chan, Kit Yee
    Affiliation
    Melbourne School of Population and Global Health
    Metadata
    Show full item record
    Document Type
    Journal Article
    Citations
    Rudan, I., Yoshida, S., Wazny, K., Chan, K. Y. & Cousens, S. (2016). Setting health research priorities using the CHNRI method: V. Quantitative properties of human collective knowledge. JOURNAL OF GLOBAL HEALTH, 6 (1), https://doi.org/10.7189/jogh.06.010502.
    Access Status
    Open Access
    URI
    http://hdl.handle.net/11343/258095
    DOI
    10.7189/jogh.06.010502
    Abstract
    INTRODUCTION: The CHNRI method for setting health research priorities has crowdsourcing as the major component. It uses the collective opinion of a group of experts to generate, assess and prioritize between many competing health research ideas. It is difficult to compare the accuracy of human individual and collective opinions in predicting uncertain future outcomes before the outcomes are known. However, this limitation does not apply to existing knowledge, which is an important component underlying opinion. In this paper, we report several experiments to explore the quantitative properties of human collective knowledge and discuss their relevance to the CHNRI method. METHODS: We conducted a series of experiments in groups of about 160 (range: 122-175) undergraduate Year 2 medical students to compare their collective knowledge to their individual knowledge. We asked them to answer 10 questions on each of the following: (i) an area in which they have a degree of expertise (undergraduate Year 1 medical curriculum); (ii) an area in which they likely have some knowledge (general knowledge); and (iii) an area in which they are not expected to have any knowledge (astronomy). We also presented them with 20 pairs of well-known celebrities and asked them to identify the older person of the pair. In all these experiments our goal was to examine how the collective answer compares to the distribution of students' individual answers. RESULTS: When answering the questions in their own area of expertise, the collective answer (the median) was in the top 20.83% of the most accurate individual responses; in general knowledge, it was in the top 11.93%; and in an area with no expertise, the group answer was in the top 7.02%. However, the collective answer based on mean values fared much worse, ranging from top 75.60% to top 95.91%. Also, when confronted with guessing the older of the two celebrities, the collective response was correct in 18/20 cases (90%), while the 8 most successful individuals among the students had 19/20 correct answers (95%). However, when the system in which the students who were not sure of the correct answer were allowed to either choose an award of half of the point in all such instances, or withdraw from responding, in order to improve the score of the collective, the collective was correct in 19/20 cases (95%), while the 3 most successful individuals were correct in 17/20 cases (85%). CONCLUSIONS: Our experiments showed that the collective knowledge of a group with expertise in the subject should always be very close to the true value. In most cases and under most assumption, the collective knowledge will be more accurate than the knowledge of an "average" individual, but there always seems to be a small group of individuals who manage to out-perform the collective. The accuracy of collective prediction may be enhanced by allowing the individuals with low confidence in their answer to withdraw from answering.

    Export Reference in RIS Format     

    Endnote

    • Click on "Export Reference in RIS Format" and choose "open with... Endnote".

    Refworks

    • Click on "Export Reference in RIS Format". Login to Refworks, go to References => Import References


    Collections
    • Minerva Elements Records [53039]
    • Melbourne School of Population and Global Health - Research Publications [5329]
    Minerva AccessDepositing Your Work (for University of Melbourne Staff and Students)NewsFAQs

    BrowseCommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects
    My AccountLoginRegister
    StatisticsMost Popular ItemsStatistics by CountryMost Popular Authors