Tag Archives: testing predictions

The Meaningless of Random Sampling

Leave a reply

Statisticians tell us that random sampling is necessary for making general inferences from the particular to the general. If field ecologists accept this dictum, we can only conclude that it is very difficult to nearly impossible to reach generality. We can reach conclusions about specific local areas, and that is valuable, but much of our current ecological wisdom on populations and communities relies on the faulty model of non-random sampling. We rarely try to define the statistical ‘population’ which we are studying and attempting to make inferences about with our data. Some examples might be useful to illustrate this problem.

Marine ecologists ae mostly agreed that sea surface temperature rise is destroying coral reef ecosystems. This is certainly true, but it camouflages the fact that very few square kilometres of coral reefs like the Great Barrier Reef have been comprehensively studied with a proper sampling design (e.g. Green 1979, Lewis 2004). When we analyse the details of coral reef declines, we find that many species are affected by rising sea temperatures, but some are not, and it is possible that some species will adapt by natural selection to the higher temperatures. So we quite rightly raise the alarm about the future of coral reefs. But in doing so we neglect in many cases to specify the statistical ‘population’ to which our conclusions apply.

Most people would agree that such an approach to generalizing ecological findings is tantamount to saying the problem is “how many angels can dance on the head of a pin”, and in practice we can ignore the problem and generalize from the studied reefs to all reefs. And scientists would point out that physics and chemistry seek generality and ignore this problem because one can do chemistry in Zurich or in Toronto and use the same laws that do not change with time or place. But the ecosystems of today are not going to be the ecosystems of tomorrow, so generality in time cannot be guaranteed, as paleoecologists have long ago pointed out.

It is the spatial problem of field studies that collides most strongly with the statistical rule to random sample. Consider a hypothetical example of a large national park that has recently been burned by this year’s fires in the Northern Hemisphere. If we wish to measure the recovery process of the vegetation, we need to set out plots to resample. We have two choices: (1) lay out as many plots as possible, and sample these for several years to plot recovery. Or (2) lay out plots at random each year, never repeating the same exact areas to satisfy the specifications of statisticians to “random sample” the recovery in the park. We typically would do (1) for two reasons. Setting up new plots each year as per (2) would greatly increase the initial field work of defining the random plots and would probably mean that travel time between the plots would be greatly increased. Using approach (1) we would probably set out plots with relatively easy access from roads or trails to minimize costs of sampling. We ignore the advice of statisticians because of our real-world constraints of time and money. And we hope to answer the initial questions about recovery with this simpler design.

I could find few papers in the ecological literature that discuss this general problem of inference from the particular to the general (Ives 2018, Hauss 2018) and only one that deals with a real-world situation (Ducatez 2019). I would be glad to be sent more references on this problem by readers.

The bottom line is that if your supervisor or research coordinator criticizes your field work because your study areas are not randomly placed or your replicate sites were not chosen at random, tell him or her politely that virtually no ecological research in the field is done by truly random sampling. Does this make our research less useful for achieving ecological understanding – probably not. And we might note that medical science works in exactly the same way field ecologists work, do what you can with the money and time you have. The law that scientific knowledge requires random sampling is often a pseudo-problem in my opinion.  

Ducatez, S. (2019) Which sharks attract research? Analyses of the distribution of research effort in sharks reveal significant non-random knowledge biases. Reviews in Fish Biology and Fisheries, 29, 355-367. doi. 10.1007/s11160-019-09556-0

Green, R.H. (1979) Sampling Design and Statistical Methods for Environmental Biologists. Wiley, New York. 257 pp.

Hauss, K. (2018) Statistical Inference from Non-Random Samples. Problems in Application and Possible Solutions in Evaluation Research. Zeitschrift fur Evaluation, 17, 219-240. doi.

Ives, A.R. (2018) Informative Irreproducibility and the Use of Experiments in Ecology. BioScience, 68, 746-747. doi. 10.1093/biosci/biy090

Lewis, J. (2004) Has random sampling been neglected in coral reef faunal surveys? Coral Reefs, 23, 192-194. doi: 10.1007/s00338-004-0377-y.

The Two Questions: So what? What next?

Leave a reply

Assuming that these two questions are not copyright, I wanted to explore them as a convenient part of writing a scientific or popular paper in ecology, conservation, and wildlife and fisheries management. To protect the innocent, I will not identify which of many ecological colleagues has stimulated this blog.

The first question should be addressed in every scientific paper but clearly is not if you read a random sample of the articles in many ecological journals. So what? is the critical question of exactly what current problem this paper or book will contribute to. It is the microscopic and macroscopic focus of why we do science, and it does not matter at all if it addresses a minor problem or a major catastrophe like species loss in conservation. In writing one should assume that time is the critical limiting factor in our lives, and while it is fine to be entertained by watching a movie, scientists do not read scientific papers to be entertained. Some journals demand that the abstract of every paper ends with a statement of the importance of the research findings, captured by So what? Too often these statements are weak and editors as well as granting agencies should demand more incisive statements. Asking yourself So what? can be a useful guide as you progress in your research and evaluate others.

While most scientists should agree on the findings presented in a paper or lecture, not all of them will agree about the importance of the answer to So what? What is a major and important scientific finding for some may be of minor significance to others, but the key is to remember here that science is a broad church that should be progressing on a broad front, so that differences of opinion are to be expected, and we rely on evidence to evaluate these differences of opinion. Tests of ideas that turn out to be incorrect or only partly correct must not be considered as failures. If you doubt that, interview any senior scientist in your area and ask about progress and regress during their scientific career. If you find a scientist who insists that they were correct in all their ideas, you should probably request them to go into politics to improve decision making in the real world.

The second question is probably the most critical for all scientific research. Once research is completed, there are two paths. If the original question or problem is solved or answered, the question becomes what does this work suggest needs to be done to advance the general area of research. Most typically however a research project will end up with more questions than it solves. The growing end of science is the critical one, and by asking What next? we delve deeper into the area of research to fill in details that were not evident when it was started. Read Sutherland et al. (2013, 2022) for an excellent example of this approach in conservation science. A simple example of this approach comes from many conservation problems. A particular species of bird may be thought to be declining in numbers, so the first issue is whether this is correct, and so an investigation into the changes in abundance of the species becomes the first step. This could lead to an analysis of the demography of the species population, birth, death and movement rates could be determined to isolate more precisely why abundance is changing. Given these data, the next step might be (for example) why the death rate is increasing if indeed this is the case. The next step is what management methods can be applied to reduce the death rate, and does this situation apply to other closely related species. It is important that asking What next? does not imply a linear sequence in time, and a study could be designed to address more than one question at the same time. We finish the What next? approach with a web of information and conclusions that address a broader question than the original simple question. And What next? should not be answered with a broad set of statements like “climate change is the cause” but by suggestions of very specific experiments and studies to carry investigations forward.

The result in ecology is an increasing precision of thought into ecological interactions and the processes that link species, communities, and ecosystems to very large questions such as the environmental response to climate change. Not all questions need to be large-scale because there are important local questions about the adequacy of designated parks and protected areas to protect species, communities, and ecosystems. The key message is that ecological understanding is not static but grows incrementally by well-designed research programs that by themselves seem to address only small-scale issues.

Seemingly failed research programs are not to be scorned but rather to indicate what avenues of research have not led to good insights. In a sense ecological science is like an evolutionary tree in which some branches fade away with time and others blossom into a variety of forms that surprise us all. So, my advice is to carry on asking these two simple questions in science to help sharpen your research program.

Sutherland, W.J., Freckleton, R.P., Godfray, H.C.J., Beissinger, S.R., Benton, T., Cameron, D.D., Carmel, Y., Coomes, D.A., Coulson, T., Emmerson, M.C., Hails, R.S., Hays, G.C., Hodgson, D.J., Hutchings, M.J., Johnson, D., Jones, J.P.G., Keeling, M.J., Kokko, H., Kunin, W.E. & Lambin, X. (2013) Identification of 100 fundamental ecological questions. Journal of Ecology, 101, 58-67.doi: 10.1111/1365-2745.12025.

Sutherland, W.J. & Jake M. Robinson, D.C.A., Tim Alamenciak, Matthew Armes, Nina Baranduin, Andrew J. Bladon, Martin F. Breed, Nicki Dyas, Chris S. Elphick, Richard A. Griffiths, Jonny Hughes, Beccy Middleton, Nick A. Littlewood, Roger Mitchell, William H. Morgan, Roy Mosley, Silviu O. Petrovan, Kit Prendergast, Euan G. Ritchie,Hugh Raven, Rebecca K. Smith, Sarah H. Watts, Ann Thornton (2022) Creating testable questions in practical conservation: a process and 100 questions. Conservation Evidence Journal, 19, 1-7.doi: 10.52201/CEJ19XIFF2753.

Is Ecology Becoming a Correlation Science?

2 Replies

One of the first lessons in Logic 101 is classically called “Post hoc, ergo propter hoc” or in plain English, “After that, therefore because of that”. The simplest example of many you can see in the newspapers might be: “The ocean is warming up, salmon populations are going down, it must be another effect of climate change. There is a great deal of literature on the problems associated with these kinds of simple inferences, going back to classics like Romesburg (1981), Cox and Wermuth (2004), Sugihara et al. (2012), and Nichols et al. (2019). My purpose here is only to remind you to examine cause and effect when you make ecological conclusions.

My concern is partly related to news articles on ecological problems. A recent example is the collapse of the snow crab fishery in the Gulf of Alaska which in the last 5 years has gone from a very large and profitable fishery interacting with a very large crab population to, at present, a closed fishery with very few snow crabs. What has happened? Where did the snow crabs go? No one really knows but there are perhaps half a dozen ideas put forward to explain what has happened. Meanwhile the fishery and the local economy are in chaos. Without very many critical data on this oceanic ecosystem we can list several factors that might be involved – climate change warming of the Bering Sea, predators, overfishing, diseases, habitat disturbances because of bottom trawl fishing, natural cycles, and then recognizing that we have no simple way for deciding cause and effect and therefore making management choices.

The simplest solution is to say that many interacting factors are involved and many papers indicate the complexity of populations, communities and ecosystems (e,g, Lidicker 1991, Holmes 1995, Howarth et al. 2014). Everyone would agree with this general idea, “the world is complex”, but the arguments have always been “how do we proceed to investigate ecological processes and solve ecological problems given this complexity?” The search for generality has led mostly into replications in which ‘identical’ populations or communities behave very differently. How can we resolve this problem? A simple answer to all this is to go back to the correlation coefficient and avoid complexity.

Having some idea of what is driving changes in ecological systems is certainly better than having no idea, but it is a problem when only one explanation is pushed without a careful consideration of alternative possibilities. The media and particularly the social media are encumbered with oversimplified views of the causes of ecological problems which receive wide approbation with little detailed consideration of alternative views. Perhaps we will always be exposed to these oversimplified views of complex problems but as scientists we should not follow in these footsteps without hard data.

What kind of data do we need in science? We must embrace the rules of causal inference, and a good start might be the books of Popper (1963) and Pearl and Mackenzie (2018) and for ecologists in particular the review of the use of surrogate variables in ecology by Barton et al. (2015). Ecologists are not going to win public respect for their science until they can avoid weak inference, minimize hand waving, and follow the accepted rules of causal inference. We cannot build a science on the simple hypothesis that the world is complicated or by listing multiple possible causes for changes. Correlation coefficients can be a start to unravelling complexity but only a weak one. We need better methods for resolving complex issues in ecology.

Barton, P.S., Pierson, J.C., Westgate, M.J., Lane, P.W. & Lindenmayer, D.B. (2015) Learning from clinical medicine to improve the use of surrogates in ecology. Oikos, 124, 391-398.doi: 10.1111/oik.02007.

Cox, D.R. and Wermuth, N. (2004). Causality: a statistical view. International Statistical Reviews 72: 285-305.

Holmes, J.C. (1995) Population regulation: a dynamic complex of interactions. Wildlife Research, 22, 11-19.

Howarth, L.M., Roberts, C.M., Thurstan, R.H. & Stewart, B.D. (2014) The unintended consequences of simplifying the sea: making the case for complexity. Fish and Fisheries, 15, 690-711.doi: 10.1111/faf.12041

Lidicker, W.Z., Jr. (1991) In defense of a multifactor perspective in population ecology. Journal of Mammalogy, 72, 631-635.

Nichols, J.D., Kendall, W.L. & Boomer, G.S. (2019) Accumulating evidence in ecology: Once is not enough. Ecology and Evolution, 9, 13991-14004.doi: 10.1002/ece3.5836.

Pearl, J., and Mackenzie, D. 2018. The Book of Why. The New Science of Cause and Effect. Penguin, London, U.K. 432 pp. ISBN: 978-1541698963

Popper, K.R. 1963. Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge and Kegan Paul, London. 608 pp. ISBN: 978-1541698963

Romesburg, H.C. (1981) Wildlife science: gaining reliable knowledge. Journal of Wildlife Management, 45, 293-313.

Sugihara, G., et al. (2012) Detecting causality in complex ecosystems. Science, 338, 496-500.doi: 10.1126/science.1227079.

On the Meaning of ‘Food Limitation’ in Population Ecology

Leave a reply

There are many different ecological constraints that are collected in the literature under the umbrella of ‘food limitation’ when ecologists try to explain the causes of population changes or conservation problems. ‘Sockeye salmon in British Columbia are declining in abundance because of food limitation in the ocean’. ’Jackrabbits in some states in the western US are increasing because climate change has increased plant growth and thus removed the limitation of their plant food supplies.’ ‘Moose numbers in western Canada are declining because their food plants have shifted their chemistry to cope with the changing climate and now suffer food limitation”. My suggestion here is that ecologists should be careful in defining the meaning of ‘limitation’ in discussing these kinds of population changes in both rare and abundant species.

Perhaps the first principle is that it is the definition of life that food is always limiting. One does not need to do an experiment to demonstrate this truism. So to start we must agree that modern agriculture is built on the foundation that food can be improved and that this form of ‘food limitation’ is not what ecologists who are interested in population changes in the real world are trying to test. The key to explain population differences must come from resource differences in the broad sense, not food alone but a host of other ecological causal factors that may produce changes in birth and death rates in populations.

‘Limitation’ can be used in a spatial or a temporal context. Population density of deer mice can differ in average density in 2 different forest types, and this spatial problem would have to be investigated as a search for the several possible mechanisms that could be behind this observation. Often this is passed off too easily by saying that “resources” are limiting in the poorer habitat, but this statement takes us no closer to understanding what the exact food resources are. If food resources carefully defined are limiting density in the ‘poorer’ habitat, this would be a good example of food limitation in a spatial sense. By contrast if a single population is increasing in one year and declining in the next year, this could be an example of food limitation in a temporal sense.

The more difficult issue now becomes what evidence you have that food is limiting in either time or space. Growth in body size in vertebrates is one clear indirect indicator but we need to know exactly what food resources are limiting. The temptation is to use feeding experiments to test for food limitation (reviewed in Boutin 1990). Feeding experiments in the lab are simple, in the field not simple. Feeding an open population can lead to immigration and if your response variable is population density, you have an indirect effect of feeding. If animals in the experimentally fed area grow faster or have a higher reproductive output, you have evidence of the positive effect of the feeding treatment. You can then claim ‘food limitation’ for these specific variables. If population density increases on your feeding area relative to unfed controls, you can also claim ‘food limitation of density’. The problems then come when you consider the temporal dimension due to seasonal or annual effects. If the population density falls and you are still feeding in season 2 or year 2, then food limitation of density is absent, and the change must have been produced by higher mortality in season 2 or higher emigration.

Food resources could be limiting because of predator avoidance (Brown and Kotler 2007). The ecology of fear from predation has blossomed into a very large literature that explores the non-consumptive effects of predators on prey foraging that can lead to food limitation without food resources being in short supply (e.g., Peers et al. 2018, Allen et al. 2022).

All of this seems to be terribly obvious but the key point is that if you examine the literature about “food limitation” look at the evidence and the experimental design. Ecologists like medical doctors at times have a long list of explanations designed to sooth the soul without providing good evidence of what exact mechanism is operating. Economists are near the top with this distinguished approach, exceeded only by politicians, who have an even greater art in explaining changes after the fact with limited evidence.

As a footnote to defining this problem of food limitation, you should read Boutin (1990). I have also raved on about this topic in Chapter 8 of my 2013 book on rodent populations if you wish more details.

Allen, M.C., Clinchy, M. & Zanette, L.Y. (2022) Fear of predators in free-living wildlife reduces population growth over generations. Proceedings of the National Academy of Sciences (PNAS), 119, e2112404119. doi: 10.1073/pnas.2112404119.

Boutin, S. (1990). Food supplementation experiments with terrestrial vertebrates: patterns, problems, and the future. Canadian Journal of Zoology 68(2): 203-220. doi: 10.1139/z90-031.

Brown, J.S. & Kotler, B.P. (2007) Foraging and the ecology of fear. Foraging: Behaviour and Ecology (eds. D.W. Stephens, J.S. Brown & R.C. Ydenberg), pp. 437-448.University of Chicago Press, Chicago. ISBN: 9780226772646

Krebs, C.J. (2013) Chapter 8, The Food Hypothesis. In Population Fluctuations in Rodents. University of Chicago Press, Chicago. ISBN: 978-0-226-01035-9

On Global Science and Local Science

Leave a reply

I suggest that the field of ecology is fragmenting into two large visions of the science which for the sake of simplicity I will call Global Science and Local Science. This fragmentation is not entirely new, and some history might be in order.

Local Science deals with local problems, and while it aspires to develop conclusions that apply to a broader area than the small study area, it has always been tied to useful answers for practical questions. Are predators the major control of caribou declines in northern Canada? Can rats on islands reduce ground-nesting birds to extinction? Does phosphate limit primary production in temperate lakes? Historically Local Science has arisen from the practical problems of pest control and wildlife and fisheries management with a strong focus on understanding how populations and communities work and how humans might solve the ecological problems they have largely produced (Kingsland 2005). The focus of Local Science was always on a set of few species that were key to the problem being studied. As more and more wisdom accumulated on local problems, ecologists turned to broadening the scope of enquiry, asking for example if solutions discovered in Minnesota might also be useful in England or vice versa. Consequently, Local Science began to be amalgamated into a broader program of Global Science.

Global Science can be defined in several ways. One is purely financial and big dollars; this not what I will discuss here. I want to discuss Global Science in terms of ecological syntheses, and Global Science papers can often be recognized by having dozens to hundreds of authors, all with data to share, and with meta-analysis as the major tool of analysis. Global Science is now in my opinion moving away from the experimental approach that was a triumph of Local Science. The prelude to Global Science was the International Biological Program (IBP) of the 1970s that attempted to produce large-scale systems analyses of communities and ecosystems but had little effect in convincing many ecologists that this was the way to the future. At the time the problem was largely the development of a theory of stability, a property barely visible in most ecological systems.

Global Science depends on describing patterns that occur across large spatial scales. These patterns can be discovered only by having an extensive, reliable set of local studies and this leads to two problems. The first is that there may be too few reliable local studies. This may occur because different ecologists use different methods of measurement, do not use a statistically reliable sampling design, or may be constrained by a lack of funding or time. The second problem is that different areas may show different patterns of the variables under measurement or have confounding causes that are not recognized. The approach through meta-analysis is fraught with the decisions that must be made to include or exclude specific studies. For example, a recent meta-analysis of the global insect decline surveyed 5100 papers and used 166 of them for analysis (van Klink et al. 2020). It is not that the strengths and limitations of meta-analysis have been missed (Gurevitch et al. 2018) but rather the question of whether they are increasing our understanding of the Earth’s ecology. Meta-analyses can be useful in suggesting patterns that require more detailed analyses. In effect they violate many of the rules of conventional science in not having an experimental design, so that they suggest patterns but can be validated only by a repeat of the observations. So, in the best situations meta-analyses lead us back to Local Science. In some situations, meta-analyses lead to no clear understanding at all, as illustrated in the conclusions of Geary et al. (2020) who investigated the response of terrestrial vertebrate predators to fire:

“There were no clear, general responses of predators to fire, nor relationships with geographic area, biome or life-history traits (e.g. body mass, hunting strategy and diet). Responses varied considerably between species.” (page 955)

Note that this study is informative in that it indicates that ecologists have not yet identified the variables that determine the response of predators to fire. In other cases, meta-analysis has been useful in redirecting ecological questions because the current global model does not fit the facts very well (Szuwalski et al. 2015).

The result of this movement within both ecological and conservation science toward Global Science has been a shift in the amount of field work being done. Rios-Saldana et al. (2018) surveyed the conservation literature over the last 35 years and found that fieldwork-based publications decreased by 20% in comparison to a rise of 600% and 800% in modelling and data analysis studies. This conclusion could be interpreted that ecologists now realize that less fieldwork is needed at this time, or perhaps the opposite. 

In an overview of ecological science David Currie (2019) described an approach to understanding how progress in ecology has differed from that in the physical sciences. He suggests that the physical sciences focused on a set of properties of nature whose variation they analyzed. They developed ‘laws’ Like Newton’s laws or motion that could be tested in simple or complex systems. By contrast ecology has developed largely by asking how processes like competition or predation work, and not by asking questions about the properties of natural systems, which is what interests the general public trying to solve problems in conservation or pest or fisheries management. Currie (2019) summarized his approach as follows:

“Successful disciplines identify specific goals and measure progress toward those goals. Predictive accuracy of properties of nature is a measure of that progress in ecology. Predictive accuracy is the objective evidence of understanding. It is the most useful tool that science can offer society.” (page 18)

Many of these same questions underlay the critical appraisal of ecology by Peters (1991).

There is no one approach to ecological science, but we need to continue to ask what progress is being made with every approach. These are key questions for the future of ecological research, and they are worthy of much more discussion because they determine what students will be taught and what kinds of research will be favoured for funding in the future.

Currie, D.J. (2019). Where Newton might have taken ecology. Global Ecology and Biogeography 28, 18-27. doi: 10.1111/geb.12842.

Geary, W.L., Doherty, T.S., Nimmo, D.G., Tulloch, A.I.T., and Ritchie, E.G. (2020). Predator responses to fire: A global systematic review and meta-analysis. Journal of Animal Ecology 89, 955-971. doi: 10.1111/1365-2656.13153.

Gurevitch, J., Koricheva, J., Nakagawa, S., and Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature 555, 175-182. doi: 10.1038/nature25753.

Kingsland, Sharon .E. (2005) ‘The Evolution of American Ecology, 1890-2000  ‘ (Johns Hopkins University Press: Baltimore.) ISBN: 0801881714

Peters, R.H. (1991) ‘A Critique for Ecology.’ (Cambridge University Press: Cambridge, England.) ISBN: 0521400171

Ríos-Saldaña, C. Antonio, Delibes-Mateos, Miguel, and Ferreira, Catarina C. (2018). Are fieldwork studies being relegated to second place in conservation science? Global Ecology and Conservation 14: e00389. doi: 10.1016/j.gecco.2018.e00389.

Szuwalski, C.S., Vert-Pre, K.A., Punt, A.E., Branch, T.A., and Hilborn, R. (2015). Examining common assumptions about recruitment: a meta-analysis of recruitment dynamics for worldwide marine fisheries. Fish and Fisheries 16, 633-648. doi: 10.1111/faf.12083.

van Klink, R., Bowler, D.E., Gongalsky, K.B., Swengel, A.B., Gentile, A. and Chase, J.M. (2020). Meta-analysis reveals declines in terrestrial but increases in freshwater insect abundances. Science 368, 417-420. doi: 10.1126/science.aax9931.

The Central Predicament of Ecological Science

Leave a reply

Ecology like all the hard sciences aims to find generalizations that are eternally true. Just as physicists assume that the universal law of gravitation will still be valid 10,000 years from now, so do ecologists assume that we can find laws or generalizations for populations and ecosystems that will be valid into the future. But the reality for ecological science is quite different. If the laws of ecology depend on the climate being stable, soil development being ongoing, evolution being optimized, and extinction being slow in human-generation time, we are in serious trouble.

Paleoecology is an important subdiscipline of ecology because, like human history, we need to understand the past. But the generalizations of paleoecology may be of little use to understand the future changes the Earth faces for one major reason – human disturbance of both climate and landscapes. Climates are changing due to rising greenhouse gases that have a long half-life. Land and water are being appropriated by a rising human population that is very slow to stabilize, so natural habitats are continually lost. There is little hope in the absence of an Apocalypse that these forces will alleviate during the next 200 years. Given these changes in the Anthropocene where does ecology sit and what can we do about it?

If climate is a major driver of ecological systems, as Andrewartha and Birch (1954) argued (to the scorn of the Northern Hemisphere ecologists of the time), the rules of the past will not necessarily apply to a future in which climate is changing. Plant succession, that slow and orderly process we now use to predict future communities, will change in speed and direction under the influence of climatic shifts and the introduction of new plant species, plant pests, and diseases that we have little control over. Technological optimists in agriculture and forestry assume that by genetic manipulations and proper artificial selection we can outwit climate change and solve pest problems, and we can only hope that they are successful. Understanding all these changes in slow-moving ecosystems depends on climate models that are accurate in projecting future climate changes. Success to date has been limited because of both questionable biology and poor statistical procedures in climate models (Frank 2019; Kumarathunge et al. 2019; Yates et al. 2018).

If prediction is the key to ecological understanding, as Houlahan et al. (2017) have cogently argued, we are in a quandary if the models that provide predictions wander with time to become less predictive. Yates et al. (2018) have provided an excellent review of the challenges of making good models for ecological prediction. As such their review is either encouraging – ‘here are the challenges in bold type’ – or terribly depressing – ‘where are the long-term, precise data for predictive model evaluation?’ My colleagues and I have spent 47 years trying to provide reliable data on one small part of the boreal forest ecosystem, and the models we have developed to predict changes in this ecosystem are probably still too imprecise to use for management. Additional years of observations produce some ecosystem states that have been predictable but other changes that we have never seen before over this time frame of nearly 50 years.

In contrast to the optimism of Yates et al. (2018), Houlahan et al. (2017) state that:

“Ecology, with a few exceptions, has abandoned prediction and therefore the ability to demonstrate understanding. Here we address how this has inhibited progress in ecology and explore how a renewed focus on prediction would benefit ecologists. The lack of emphasis on prediction has resulted in a discipline that tests qualitative, imprecise hypotheses with little concern for whether the results are generalizable beyond where and when the data were collected.  (page 1)

I see this difference in views as a dilemma because despite much talk, there is little money or interest in the field work that would deliver reliable data for models in order to test their accuracy in predictions at small and large scales. An example this year is the failure of the expected large salmon runs to the British Columbia fishery, with model failure partly due to the lack of monitoring in the North Pacific (https://globalnews.ca/news/5802595/bc-salmon-stocks-plunge/; https://www.citynews1130.com/2019/09/09/worst-year-for-salmon/ , and in contrast with Alaska runs: https://www.adn.com/business-economy/2019/07/25/bristol-bay-sockeye-harvest-blowing-away-forecast-once-again/ ). Whatever the cause of the failure of B.C. salmon runs in 2019, the lack of precision in models of a large commercial fishery that has been studied for at least 65 yeas is not a vote of confidence in our current ecological modelling.

Andrewartha, H.G. and Birch, L.C. (1954) ‘The Distribution and Abundance of Animals.’ University of Chicago Press: Chicago. 782 pp.

Frank, P. (2019). Propagation of error and the reliability of global air temperature projections. Frontiers in Earth Science 7, 223. doi: 10.3389/feart.2019.00223.

Houlahan, J.E., McKinney, S.T., Anderson, T.M., and McGill, B.J. (2017). The priority of prediction in ecological understanding. Oikos 126, 1-7. doi: 10.1111/oik.03726.

Kumarathunge, D.P., Medlyn, B.E., Drake, J.E., Tjoelker, M.G., Aspinwall, M.J., et al. (2019). Acclimation and adaptation components of the temperature dependence of plant photosynthesis at the global scale. New Phytologist 222, 768-784. doi: 10.1111/nph.15668.

Yates, K.L., Bouchet, P.J., Caley, M.J., Mengersen, K., Randin, C.F., Parnell, S., Fielding, A.H., Bamford, A.J., et al. (2018). Outstanding challenges in the transferability of ecological models. Trends in Ecology & Evolution 33, 790-802. doi: 10.1016/j.tree.2018.08.001.

Models need testable predictions to be useful

2 Replies

It has happened again.  I have just been to a seminar on genetic models – something about adaptation of species on the edges of their ranges.  Yes this is an interesting topic of relevance to interpreting species’ responses to changing environments.  It ended by the speaker saying something like, “It would be a lot of work to test this in the field”. How much more useful my hour would have been spent if the talk had ended with “Although it would be difficult to do, this model makes the following predictions that could be tested in the field,” or “The following results would reject the hypothesis upon which this model is based.”

Now it is likely that some found these theoretical machinations interesting and satisfying in some mathematical way, but I feel that it is irresponsible to not even consider how a model could be tested and the possibility (a likely possibility at that) that it doesn’t apply to nature and tells us nothing helpful about understanding what is going to happen to willow or birch shrubs at the edge of their ranges in the warming arctic (for example).

Recommendation – no paper on models should be published or talked about unless it makes specific, testable predictions of how the model can be tested.