Weighting and aggregating expert ecological judgments
AuthorHemming, V; Hanea, AM; Walshe, T; Burgman, MA
Source TitleEcological Applications
PublisherEcological Society of America
AffiliationSchool of Historical and Philosophical Studies
School of BioSciences
School of Ecosystem and Forest Sciences
Document TypeJournal Article
CitationsHemming, V., Hanea, A. M., Walshe, T. & Burgman, M. A. (2020). Weighting and aggregating expert ecological judgments. Ecological Applications, 30 (4), https://doi.org/10.1002/eap.2075.
Access StatusOpen Access
Performance weighted aggregation of expert judgments, using calibration questions, has been advocated to improve pooled quantitative judgments for ecological questions. However, there is little discussion or practical advice in the ecological literature regarding the application, advantages or challenges of performance weighting. In this paper we (1) illustrate how the IDEA protocol with four‐step question format can be extended to include performance weighted aggregation from the Classical Model, and (2) explore the extent to which this extension improves pooled judgments for a range of performance measures. Our case study demonstrates that performance weights can improve judgments derived from the IDEA protocol with four‐step question format. However, there is no a‐priori guarantee of improvement. We conclude that the merits of the method lie in demonstrating that the final aggregation of judgments provides the best representation of uncertainty (i.e., validation), whether that be via equally weighted or performance weighted aggregation. Whether the time and effort entailed in performance weights can be justified is a matter for decision‐makers. Our case study outlines the rationale, challenges, and benefits of performance weighted aggregations. It will help to inform decisions about the deployment of performance weighting and avoid common pitfalls in its application.
- Click on "Export Reference in RIS Format" and choose "open with... Endnote".
- Click on "Export Reference in RIS Format". Login to Refworks, go to References => Import References