A comparison of the inferential, computational, and predictive performance of joint species distribution models

Abstract

The standard correlative species distribution model accounts for the effect of the environment on species distributions without explicitly accounting for the effect of species interactions. Community ecology, in contrast, studies species co-occurrence patterns without accounting for the co-occurrence that could be explained by the shared response of species to the environment. Joint species distribution models (JSDMs) are a relatively recent development in the ecological literature that extend the single species distribution model framework to model multiple species simultaneously while accounting for species co-occurrence, and are an exciting prospect for bridging the gap between these two disciplines. Research into JSDMs is still in its early stages as a field and we have identified six outstanding problems in the literature that are potentially hampering the wider uptake of JSDMs by the wider ecological community.

The confluence of statistical and computational advancements has led to the rapid development of JSDMs, and the proposal of several models and implementations. These models are statistically complex but, while well defined as stand-alone studies, are described using a diverse array of model notation, terminology, and symbols. This often conflicting notation makes comparing the different JSDM implementations a daunting task to most ecologists who are trying to identify which model is best suited to their purpose. In this thesis I develop and present a clearly-defined, singular notation for the description of JSDMs and use it to elucidate the similarities and differences between seven different JSDM implementations.

All of the newly proposed JSDM implementations have been presented on completely different datasets to each other and thus any measure of inferential performance is not directly comparable. Without a direct comparison potential practitioners are unable to make an informed decision about the performance of these models. In this thesis I conduct a direct comparison of seven JSDM implementations on six datasets to assess their inferential performance. I found that all JSDMs identified similar species-environment relationships to each other. All JSDMs identified species co-occurrence patterns in the same direction but with different strengths and uncertainties. The newly proposed JSDMs have been implemented using a variety of software with different default or suggested parameters for model fitting. In addition to being fit to datasets of a varying size it is difficult to directly compare the computational performance of different implementations. Potential practitioners are going to be influenced by factors such as how fast a model can run and the size of dataset it is capable of feasible fitting to. In this thesis I perform a direct comparison of seven JSDMs on six datasets to directly assess their computational performance. I found a greater difference in computational performance between JSDMs than for inferential performance. I found evidence that scaling issues due to the size of datasets, and thus the number of parameters to estimate, was more prevalent for some implementations than others. A comparison of effective sample size in their respective Markov chain Monte Carlo regimes also indicated that some implementations are far more computationally efficient than others.

The use of JSDMs for prediction has only recently begun to be addressed in the literature, but has almost exclusively focussed on environment-only prediction types akin to those of single species models. The multivariate nature of JSDMs, however, allows for new, community-level predictions to be used. To date none of these prediction types have a formal definition for JSDMs. In this thesis I present four types of prediction that can be applied with JSDMs. Marginal predictions are environment-only predictions that do not account for the residual associations between species. Joint predictions are a community-level prediction that can return either a probability of observing a particular assemblage or generate a series of plausible assemblages at a site. These two prediction types can then be conditioned upon the known occurrence states of one or more species at a site, and thus present conditional marginal and conditional prediction types respectively.

Single-species models have been subject to substantial research into the evaluation of their prediction, but this has seen minimal to date for JSDMs. In this thesis I present five classes of evaluation metrics that assess different aspects of species distributions and the community assemblage process. Threshold-independent and threshold-dependent metrics operate at the species-level, community dissimilarity and species richness metrics operate at the community-level, and likelihood-based metrics which assess model fit.

The literature on JSDM prediction has to date been limited in its comparisons of predictive performance. Studies have either compared a limited number of models, a subset of evaluation metrics, and/or generally only considered environment-only prediction types. In this thesis I present a comparison of predictive performance for six JSDMs and two stacked single-species models. I fit the models to 22 real and simulated datasets, predict with the four prediction types I have defined for JSDMs, and evaluate these predictions with up to 32 metrics from five classes. I found that likelihood-based metrics suggested the JSDMs were better fit to the data but that the other metric classes generally showed them to underperform compared to the standard stacked single species model for all prediction types. The stacked single species model with the SESAM constraint was found to consistently outperform all other models. This suggests an interesting avenue of future research by applying this constraint to JSDM predictions.