Model performance and extrapolation between populations.

BES Beyond Demography Symposium: The Challenges of Prediction

Last week I was at the BES Beyond Demography symposium. Looking at the speaker list it’s a who’s who of not just demography, but population and ecological modelling in general. One persistent topic across many of the talks and general hallway talk was how our population models often have very poor predictive power.

This is a problem because we face a lot of challenges where good predictions of population growth under different environmental conditions would be very useful. For example, predicting which species will be most impacted by climate change and which are likely to become damaging invasive species. Making good predictions would allow us to focus our limited resources on the species that will be the most affected or would have the biggest impact.

Predictions require models. The state of the art in population modelling right now is to fit some kind of statistical model between environmental variables (like rainfall, soil, fire, etc…) and population processes like mortality, seed production and growth. Those fitted responses to the environment are then used in a population model of some kind, generally an integral projection model.

Currently these models are quite poor at making predictions of things like population growth rate and extinction risk. Our models are especially poor when we try and apply them beyond the range of environmental conditions they are parameterised with. Unfortunately this is the exact situation we face when trying to predict the response of populations to climate change, or when a species is introduced into a new region.

Why are our models so poor, is there anything we can do about it and does it matter if we can’t? There are multiple reasons why our models are poor at making predictions, some of them we can fix and some of them we can’t. The first is that population dynamics are strongly influenced by things like rainfall, storms and fire. We cannot predict what these important environmental factors will do more than a few weeks into the future (at best), so it is not very surprising that we can’t predict the population dynamics these factors shape. There is not much that population modellers can do about this except hope that increasing computer power combined with improved weather and climate modelling allow predictions of these processes to become possible at the temporal and spatial resolution required. This is a massive area of research in its own right, but the problem is a hard one so progress will likely be slow.

The second reason is that we can’t measure every aspect of an environment and the things we often don’t measure (like bacteria and fungi in the soil) can have a big effect on the process of birth, death and growth. While we can never measure everything, we can do a better job than we currently do now and our ability to measure the environment is getting cheaper all the time. Advancements in genetic sequencing make it increasingly affordable to sequence whole communities of soil biota. Also remote sensing and small cheap sensors are allowing us to capture more and more environmental information at large scales and fine temporal resolutions.

Related to the problem of not being able to measure everything is that two individuals in the same small area can behave in completely different ways and we often don’t understand why. The reason is likely to be a combination of very small scale environmental variation, pathogens, genetic differences between individuals and plasticity in individual traits. We have a relatively poor understanding of how these different factors relate to each other to affect growth, survival and fecundity. One possibility is to also model these underlying processes and their effect on the vital rates of each individual, an active area of research known as physiological models. This has two big problems; firstly a model that captured all these processes for individuals in a population would be very complicated, making it hard to understand. Second, even if we built such a model it would have lots of different parameters that would need to be measured, taking us back to the measurement problem above.

Model performance and extrapolation between populations.
Model performance and extrapolation between populations.

An alternative to explicitly modelling every population we are interested in is to extrapolate from populations we already have information for. It is still unclear how far we can extrapolate and for what. In my current work I am using a global database of plant population matrix models to test how well closely related species and physically close populations, predict population behaviour. At this stage I am focusing on elasticity’s, which tell us how important different vital rates are for population growth rate. My preliminary results suggest that we can extrapolate demography between populations, but only at relatively small scales. Populations within 20km of each other and species that diverged less than 50mya tend to have similar elasticity structures. For more detail see the talk I gave at the symposium:

From a practical perspective how good do our predictions have to be? To date we are not very good at answering this question. The answer will depend on multiple factors. One important factor is the question being addressed. Take the example of a species responding to climate change. If a species has a narrow climatic range we will need to predict how well that species can track the changing climate with greater accuracy. On the other hand if the species has a wide climatic range we may have more room for error in our predictions. Another important factor is how risk averse we willing to be. If we fail to act because we predicted a species would not go extinct, and it does, then the consequences are large (global extinction) and permanent. In this situation we might be risk adverse and take action to save the species even if our model predicted a small (but non-zero) risk of extinction. Given that our current models often make predictions with a wide range of uncertainty, population extinction will often be included in that range. If we believe that this will be the outcome from that start, we might consider skipping the modelling step and go straight to management actions. A more formal treatment of this idea is called value of information analysis. A value of information analysis asks, what is the benefit of new information? This benefit is measured by the improvement in management resulting from that new information. This leads to two general insights. The first is, if our model predictions come with so much uncertainty that almost any action might look reasonable under one of the many possible predictions, then the model will be of little value. The second is that if we face a problem with no feasible management options then it does not matter how accurate our models are since we would not be able to do anything even with a very accurate model.