Category Archives: Evaluating Research Quality

The Correlation Coefficient as an Enemy of Science

1 Reply

Science is under attack in many parts of the world today, but I would like to call attention to an internal enemy of scientific advancement that is equally worrisome. I will address my comments to ecological papers that employ correlations. I am sure that the errors I discuss will not ever be found in other sciences (see Ioannidis 2019).

Perhaps the most common statistic to be found in ecological papers is the correlation coefficient r. The correlation coefficient measures the strength of the association between two variables. Typically, in ecological use the two variables ought to have some mechanistic or causal relationship between them. This is the key question you must ask before calculating an r value. Many examples of nonsense correlations are given in Huff (1954) and Best (2001). A 20-year data set of the number of people attending a particular church in a specified area may be very highly correlated with the increase in automobile accidents in this area. We laugh at these sorts of examples but the central problem in ecology is the measure of cause and effect and how to decipher the effects of multiple causes on our variable of interest. Tree growth may be correlated with the amount of summer rainfall, but it may also be correlated with summer temperatures. How we proceed from here is determined by the purpose of the investigation. If you wish trees to grow better in a drought, be sure to water them. It may not matter to you what the temperature is in this study, particularly if you have no control of temperature but can control water. But if the purpose of the investigation is to determine all the factors that control tree growth, you will have a much more complex problem to resolve, concerning the chemistry of the soil and the genetic makeup of each tree, among other variables. In all these cases the key is to specify what is the question you wish to answer?

Much of ecological research is exploratory, step 1 of trying to understand species, populations or communities. Trap 1 which you must try to avoid is measure everything possible.  Falling into trap 1 leads to trap 2 which is you have much data with little insight. It is certainly possible to be lucky and gain a critical insight by measuring many variables. My impression which may be incorrect is that frequently you have missed some critical interaction. Everyone I know has made this mistake, but you can recover from it by becoming more specific in your aims.

At this point we can, if possible, move into experimental ecology in which we vary treatments to some populations or communities and have controls that are not manipulated. One outline of experimental approaches in wildlife management is given in Hone and Krebs (2023). But the warning is clear even with supposed experimental treatments of modern medicine where the consequences of mistakes come very close to home (Ioannidis 2021).

The take-home message from all these warnings is that when you discuss correlations be concerned with the variables that are omitted from your analysis, and because of that be cautious about your conclusions. The best way to indicate this in your writing is to present in your conclusions a simple statement of what needs to be done next on this ecological problem.

Best, J. 2001. Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists. University of California Press. 204 pp.

Hone, J. and Krebs, C.J. 2023. Causality and wildlife management. Journal of Wildlife Management 2023: e22412. doi:10.1002/jwmg.22412.

Huff, D. 1954. How to Lie With Statistics. R.S. Means Company pp. 142.

Ioannidis, J.P.A. 2019. What have we (not) learnt from millions of scientific papers with P values? American Statistician 73(sup1): 20-25. doi:10.1080/00031305.2018.1447512.

Ioannidis, J.P.A. 2021. Hundreds of thousands of zombie randomised trials circulate among us. Anaesthesia 76(4): 444-447. doi:10.1111/anae.15297.

How Can We Best Advance Ecological Research

Leave a reply

Everyone who supports scientific research wishes to see progress in understanding of the problems being studied. Ecologists face four critical difficulties in achieving these goals of progress.

  •  There are far too many ecological problems to be sorted out given the number of ecologists in the scientific world. There is a partial solution to this issue by dividing the ecological research into two broad categories: large scale critical problems and small-scale local problems. Large-scale problems are those that apply to many communities and ecosystems, from the local to the global. Large scale problems require first an agreement of what these problems are and second how they should be approached – both difficult to achieve in the current research environment. So there is too little agreement even on this simple issue. Large-scale problems require a coordinated team effort and to date in ecology large-scale problems have been poorly addressed. Small scale problems can be studied with less person power and are much more common. In an ideal world, governments would fund large-scale research programs that require effort that stretches over many years and require considerable reliable funding, and smaller grants could support the many specific postgraduate studies with a time limit of 3-4 years. It would be optimal for these two groups to meld together but this is difficult to achieve. The whole system we have currently fights against this cooperation, partly because in the world of science most of the recognition and rewards go to individual scientists and not to a research team.
  •  A second problem is that there are two paradigms of ecological science that are only partly overlapping. The first paradigm in its strong form states that ecological science will advance most rapidly by means of descriptive studies of changes in communities and ecosystems. Gaiser et al. (2020) provide good examples of this approach. Long-Term-Ecological-Research (LTER) approaches are typically large-scale and rely too much on the assumptions of correlation = causation science, but this paradigm suggests that long-term data will lead us to the long-term understanding of communities and ecosystems. The second approach can be described as the mechanistic paradigm because it attempts to explain long- and short-term population and community changes from whatever cause using experimental methods (Krebs 2002). Hone and Krebs (2023) detail the evidence levels needed for strong inference in both short-term and long-term issues. Lindenmayer (2018) provides an excellent synopsis of the difficulties of long-term research and a synopsis of how we might act to answer ecological questions about future issues (Lindenmayer et al. 2015, Lindenmayer 2018).
  • The third difficulty is that the individual scientist as the unit of research makes it unlikely in the present organization of science funding that these two paradigms can be easily brought together. Scientists by and large wish to be independent, a desirable trait, but we can and should work in a team effort, an interlocking independent research program that has specified objectives that all are organized into a partnership. If you want examples of this approach, you have only to look at the successes and failures of many long-term ecological research (LTER) programs around the world. One example of a successful LTER research program in Austria is reviewed in Gingrich et al. (2016). Additional evaluations of current LTER projects can be found in Vanderbilt and Gaiser (2017) and Rastetter et al. (2021). There has been a movement to integrate social and ecological frameworks to LTER research. A good example of this social-ecological approach is disturbance ecology described for the USA in Gaiser et al. (2020). But ecological approaches of the type described in Vanderbilt and Gaiser (2017) and Gaiser et al. (2020) appear to reduce ecological research to an endless study of descriptive changes in ecosystems with little theory. The hope is to advance ecological science by observations of changes in ecosystems as affected by human activity and climate change. The objective of these research programs is to provide solutions for future environmental problems from long term data sets. However, describing the past does not inherently predict the future, as evidenced by the issues surrounding climate change.
  • A fourth problem is that the long-term funding necessary to understand new and continuing ecological problems too often falls to a new Director or Chief who wishes to change the direction of the research or stop it altogether, so long term objectives are not supported. Another aspect of the funding problem for LTER research is the lack of substantial funding on a spatial and monetary scale that would permit comprehensive research with suitable replication. Cusser at al. (2021) have analysed LTER studies in the USA with respect to how long studies must be to achieve good results. They concluded that half of the LTER studies required 10 years or more to produce consistent results, and some required more than 20 years. Many of these LTER studies are focused on the descriptive paradigm and would not qualify as experimental ecology with specific hypotheses about community and ecosystem dynamics. LTER descriptive studies are useful for advancing knowledge of trends, but they may not be sufficient to identify and test the underlying drivers of community and ecosystem change.

Cusser, S., Helms IV, J., Bahlai, C.A., and Haddad, N.M. 2021. How long do population level field experiments need to be? Utilising data from the 40-year-old LTER network. Ecology Letters 24(5): 1103-1111. doi:10.1111/ele.13710.

Gaiser, E.E., Bell, D.M., Castorani, M.C.N., Childers, D.L., and Groffman, P.M. 2020. Long-term ecological research and evolving frameworks of disturbance ecology. Bioscience 70(2): 141-156. doi:10.1093/biosci/biz162.

Gingrich, S., Schmid, M., Dirnbock, T., Dullinger, I. et al. 2016. Long-Term Socio-Ecological Research in Practice: Lessons from Inter- and Transdisciplinary Research in the Austrian Eisenwurzen. Sustainability 8(8): 743. doi:10.3390/su8080743.

Hone, J., and Krebs, C.J. 2023. Causality and wildlife management. Journal of Wildlife Management 2023: e22412. doi:10.1002/jwmg.22412.

Krebs, C.J. 2002. Two complementary paradigms for analyzing population dynamics. Philosophical Transactions of the Royal Society of London, Series B 357: 1211-1219. doi:10.1098/rstb.2002.1122.

Lindenmayer, D.B., Burns, E.L., Tennant, P., Dickman, C.R., Green, P.T., Keith, D.A., et al. 2015. Contemplating the future: Acting now on long-term monitoring to answer 2050’s questions. Austral Ecology 40(3): 213-224. doi:10.1111/aec.12207.

Lindenmayer, D. 2018. Why is long-term ecological research and monitoring so hard to do? (And what can be done about it). Australian Zoologist 39: 576-580. doi:10.7882/az.2017.018.

Rastetter, E.B., Ohman, M.D., Elliott, K.J., Rehage, J.S., and al., e. 2021. Time lags: insights from the U.S. Long Term Ecological Research Network. Ecosphere 12(5): e03431. doi:10.1002/ecs2.3431.

Vanderbilt, K., and Gaiser, E. 2017. The International Long Term Ecological Research Network: a platform for collaboration. Ecosphere 8(2): e01697. doi:10.1002/ecs2.1697.

Should Ecology Abandon Popper?

2 Replies

The first question I must ask is whether you the reader have ever heard of Karl Popper. If the answer is no, then you could profit from reading Popper (1963) before you read this. An abbreviated version of the Popperian approach to science is presented in a short paper by Platt (1963) The simplest version of Popper and Platt is that we should have a hypothesis with specific predictions and one or more alternative hypotheses with other predictions, and science advances by finding out which hypotheses could be rejected with empirical evidence. The focus of this blog is on a recent paper by Raerinne (2024) claiming that Popperian ecology is a delusion. This is a claim well worth discussing particularly since most of the sciences progress using a Popperian approach to testing hypotheses.

To begin perhaps we should recognize two kinds of papers that appear in ecological journals. A very large set of ecological papers appear to be largely or entirely descriptive natural history typically of past or present events with no hypotheses in mind. Many of these papers end with a conclusion that could be designated as a hypothesis but with little discussion of alternatives. These papers can be very valuable in giving us the state of populations, communities, or ecosystems with recommendations for changes that should be made to alleviate developing problems. A good example are papers describing forest and grassland fires of recent years which can end with some management recommendations, and perhaps with alternative recommendations. These recommendations usually arise from experience and judgements, and they may or not be valid. The Popperian approach would be to set up hypotheses and test them empirically, but if we are people of action, we press onward with a preferred management action. The non-Popperian approach would be very efficient if we were correct in our diagnosis, and in many cases this approach works well. The basis of the issue here is what is evidence in ecology and how should it be sharpened into recommendations for conservation and management.  

The Popperian approach to ecological science is to recognize problems that require a solution to increase our knowledge base, and to suggest a series of alternative set of mechanisms that could solve or alleviate the problem. Ecological papers supporting this approach can often be recognized by searching for the word “hypothesis” in the text. A simple example of this Popperian approach could be finding the causes of the continuing decline of a commercial fishery. The decline might be due to predation on the target fish or invertebrate, a disease, added pollution to the water body, climate change increasing the water temperature and thus metabolic functions, introduced species of competitors for food or space. One or more of these causal factors could be involved and the job of the ecologist is to find out which one or several are diagnostic. Given the complexity of ecological problems, it is typically not possible to test these alternative hypotheses in one grand experiment, and the typical approach will involve adaptive management or evidence-based conservation (Gillson et al. 2019, Serrouya et al., Westgate et al. 2013). Complexity however should not be used as an excuse to do poor science.

What is the alternative if we abandon Popper? We could adopt the inductive approach and gather data that we put together with our judgement to declare that we have a correct answer to our questions, “seat of the pants” ecology. But this approach is heavily dependent on the idea that “the future will be like the past”. This approach to ecological problems will be most useful for the very short term. The simplest example comes from weather forecasting in which the prognosis for today’s weather is what it was like yesterday with minor adjustments. We could observe trends with this approach but then we must have a statistical model that predicts, for example, that the trend is linear or exponential. But the history of science is that we can do much better by understanding the mechanisms underlying the changes we see. A good overview of the dilemmas of this inductive approach for conservation biology is provided by Caughley (1994). The operative question here is whether the inductive approach achieves problem resolutions more efficiently than the Popperian approach through conjecture and refutation.

Raerinne (2023, 2024) does biology in general and ecology in particular a disservice in criticizing Popper’s approach to ecology by arguing that ecology should not be criticized nor evaluated from the Popperian perspective. I think this judgement is wrong, and Raerinne’s conclusion arises from a philosophical viewpoint which could well have little applicability to how ecologists solve empirical problems in the real world. But you can judge.  

Carducci, A., Federigi, I. & Verani, M. (2020) Airborne transmission and its prevention: Waiting for evidence or applying the Precautionary Principle? Atmosphere, 11 (7), 710.doi: 10.3390/atmos11070710.

Caughley, G. (1994) Directions in conservation biology. Journal of Animal Ecology, 63, 215-244. doi: 10.2307/5542.

Gillson, L., Biggs, H. & Rogers, K. (2019) Finding common ground between adaptive management and evidence-based approaches to biodiversity conservation. Trends in Ecology & Evolution, 34, 31-44.doi: 10.1016/j.tree.2018.10.003.

Platt, J.R. (1964) Strong inference. Science, 146, 347-353.doi: 10.1126/science.146.3642.347.

Popper, K.R. (1963) Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge and Kegan Paul, London.

Raerinne, J. (2023) Myths of past biases and progress in biology. Theory in Biosciences, 142, 383-399.doi: 10.1007/s12064-023-00403-2.

Raerinne, J. (2024) Popperian ecology is a delusion. Ecology and Evolution, 14, e11106.doi: 10.1002/ece3.11106.

Serrouya, R., Seip, D.R., Hervieux, D., McLellan, B.N., McNay, R.S., Steenweg, R., Heard, D.C., Hebblewhite, M., Gillingham, M. & Boutin, S. (2019) Saving endangered species using adaptive management. Proceedings of the National Academy of Sciences, 116, 6181-6186.doi: 10.1073/pnas.1816923116 .

Westgate, M.J., Likens, G.E. & Lindenmayer, D.B. (2013) Adaptive management of biological systems: A review. Biological Conservation, 158, 128-139.doi: 10.1016/j.biocon.2012.08.016 .

Do We Need to Replicate Ecological Experiments?

Leave a reply

If you read papers on the philosophy of science you will very quickly come across the concept of replication, the requirement to test the same hypothesis twice or more before you become too attached to your conclusions. As a new student or a research scientist you face this problem when you wish to replicate some previous study. If you do replicate, you risk being classed as an inferior scientist with no ideas of your own. If you refuse to replicate and try something new, you will be criticized as reckless and not building a solid foundation in your science.  

There is an excellent literature discussing the problem of replication in ecology in particular and science in general. Nichols et al. (2019) argue persuasively that a single experiment is not enough. Amrheim et al. (2019) approach the problem from a statistical point of view and caution that single statistical tests are a shaky platform for drawing solid conclusions. They point out that statistical tests not only test hypotheses, but also countless assumptions and particularly for ecological studies the exact plant and animal community in which the study takes place. In contrast to ecological science, medicine probably has more replication problems at the other extreme – too many replications – leading to a waste of research money and talent. (Siontis and Ioannidis 2018).

A graduate seminar could profitably focus on a list of the most critical experiments or generalizations of our time in any subdiscipline of ecology. Given such a list we could ask if the conclusions still stand as time has passed, or perhaps if climate change has upset the older predictions, or whether the observations or experiments have been replicated to test the strength of conclusions. We can develop a stronger science of ecology only if we recognize both the strengths and the limitations of our current ideas.

Baker (2016) approached this issue by asking the simple question “Is there a reproducibility crisis?” Her results are well worth visiting. She had to cast a wide net in the sciences so unfortunately there are no details specific to ecological science in this paper. A similar question in ecology would have to distinguish observational studies and experimental manipulations to narrow down a current view of this issue. An interesting example is explored in Parker (2013) who analyzed a particular hypothesis in evolutionary biology about plumage colour in a single bird species, and the array of problems of an extensive literature on sexual selection in this field is astonishing.

A critic might argue that ecology is largely a descriptive science that should not expect to develop observational or experimental conclusions that will extend very much beyond the present. If that is the case, one might argue that replication over time is important for deciding when an established principle is no longer valid. Ecological predictions based on current knowledge may have much less reliability than we would hope, but the only way to find out is to replicate. Scientific progress depends on identifying goals and determining how far we have progressed to achieving these goals (Currie 2019). To advance we need to discuss replication in ecology.

Amrhein, V., Trafinnow, D. & Greenland, S. (2019) Inferential statistics as descriptive statistics: There is no replication crisis if we don’t expect replication. American Statistician, 73, 262-270. doi: 10.1080/00031305.2018.1543137.

Baker, M. (2016) Is there a reproducibility crisis in science? Nature, 533, 452-454.

Currie, D.J. (2019) Where Newton might have taken ecology. Global Ecology and Biogeography, 28, 18-27. doi: 10.1111/geb.12842.

Nichols, J.D., Kendall, W.L. & Boomer, G.S. (2019) Accumulating evidence in ecology: Once is not enough. Ecology and Evolution, 9, 13991-14004. doi: 10.1002/ece3.5836.

Parker, T.H. (2013) What do we really know about the signalling role of plumage colour in blue tits? A case study of impediments to progress in evolutionary biology. Biological Reviews, 88, 511-536. doi: 10.1111/brv.12013.

Siontis, K.C. & Ioannidis, J.P.A. (2018) Replication, duplication, and waste in a quarter million systematic reviews and meta-analyses. Circulation: Cardiovascular quality and outcomes, 11, e005212. doi: 10.1161/CIRCOUTCOMES.118.005212.

On Critical Evaluation in Ecology

Leave a reply

Science proceeds by “conjecture-and-refutation” if we agree with Karl Popper (1963). There is a rich literature on science in general and ecological science in particular that is well worth a series of graduate discussions even if it is pre-2000 ancient history (Peters 1991, Weiner 1995, Woodward and Goodstein 1996). But I wish to focus on a current problem that I think is hindering ecological progress. I propose that ecological journals at this time are focusing their publications on papers that present apparent progress and are shedding papers that are critical of apparent progress. Or in Popper’s words, they focus on publishing ‘conjecture’ and avoid ‘refutation’. The most important aspect of this issue involves wildlife management and conservation issues. The human side of this issue may involve personal criticism and on occasion the loss of a job or promotion. The issue arises in part because of a confusion between the critique of ideas or data and the interpretation that all critiques are personal. So, the first principle of this discussion is that I discuss here only critiques of ideas or data.

There are many simple reasons for critiques of experimental design and data gathering. Are the treatments replicated, are the estimates of data variables reliable and sufficient, are proxy variables good or poor? Have the studies been carried out long enough? All these critiques can be summarized under the umbrella of measurement reliability. There are many examples we can use to illustrate these ideas. Are bird populations declining across the globe or locally? Are fisheries overharvesting particular species? Can we use climate change as a universal explanation of all changes in wildlife populations? Are survey methods for population changes across very large areas reliable? The problem is tied into the need for good or bad news that must be filtered to the news media or social media with high impact but little reliability. 

The problem at the level of science is the temptation to extrapolate beyond the limits of the available data. Now we come to the critical issue – how do our scientific journals respond to critical reviews of papers already published? My concern is that in the present time journals do not wish to receive or accept manuscripts that are critical of previously published papers. These decisions are no doubt confidential for journal publishers. There is perhaps some justification for this rejection policy, given that in the few cases where critiques are published on existing papers, the citation score of the original paper may greatly exceed that of the critique. So, conjecture pays, refutation does not.

Journals are flooded with papers and for the better journals I would expect at least a 60-80% rejection rate. For Science the rejection rate is 94%, for Nature 92%, and for the Journal of Animal Ecology 85% of submitted manuscripts are rejected. Consequently, the suggestion that they reserve space for ‘refutation’ is too negative to their publication model. There is little I can suggest if one in caught in this dilemma except to try another less premium journal, and remember that web searches find papers easily no matter where published. If you need inspiration, you can follow Peters (1991) and write a book critique and suffer the brickbats from the establishment (e.g. Nature 354: 444, 12 December 1991).

But if you are upset about a particular paper or series of papers, remember critiques are valuable but follow these rules for a critique:

  1. Keep it short, 5 typed pages should be near maximal length.
  2. Raise a set of major points. Do not try to cover everything.
  3. Summarize briefly the key points you are in agreement with, so they are not confounded in the discussion.
  4. Discuss what studies might distinguish hypothesis A vs B, or A+B vs C.
  5. Discuss what better methods of measurement might be used if funding is available.
  6. Never attack individuals or research groups. The discussion is about ideas, results, and inferences.

Decisions to accept some management actions may have to be taken immediately and journal editors must take that into consideration. Prognostication over accepting critiques may be damaging. But all actions must be continually evaluated and changed once the understanding of the problem changes.

There are too many examples to recommend reading about past and present controversies in ecology, so here are only two examples. Dowding et al. (2009) report a comment on suggested methods of controlling introduced pests on Macquarie Island in the Southern Ocean. I was involved in that discussion. A much bigger controversy in Canada involves Southern Mountain caribou populations which are in rapid decline. The proximate explanation for the decline is postulated to be predation by wolves and thus the suggested management action is shooting the wolves. Johnson et al. (2022), Lamb et al. (2022) and Superbie et al. (2022) provide an entre into this literature and the decisions of what to do now and in the future to prevent extinction of these ungulates. The caribou problem is complicated by the interaction of human alteration of landscapes and the natural processes of predation and food availability. Alas nothing is simple.

All these ecological dilemmas are controversial and the important role of criticism involving evaluations of alternative hypotheses are the only way forward for ecologists involved in controversies. In my opinion most ecological journals are not doing their part is publishing critiques of the conventional wisdom.

Dowding, J.E., Murphy, E.C., Springer, K., Peacock, A.J. & Krebs, C.J. (2009) Cats, rabbits, Myxoma virus, and vegetation on Macquarie Island: a comment on Bergstrom et al. (2009). Journal of Applied Ecology, 46, 1129-1132. doi: 10.1111/j.1365-2664.2009.01690.x.

Johnson, C.J., Ray, J.C. & St-Laurent, M.-H. (2022) Efficacy and ethics of intensive predator management to save endangered caribou. Conservation Science and Practice, 4: e12729. doi: 10.1111/csp2.12729.

Lamb, C.T., Willson, R., Richter, C., Owens-Beek, N., Napoleon, J., Muir, B., McNay, R.S., Lavis, E., Hebblewhite, M., Giguere, L., Dokkie, T., Boutin, S. & Ford, A.T. (2022) Indigenous-led conservation: Pathways to recovery for the nearly extirpated Klinse-Za mountain caribou. Ecological Applications 32 (5): e2581. doi: 10.1002/eap.2581.

Peters, R.H. (1991) A Critique for Ecology. Cambridge University Press, Cambridge, England. 366 pp. ISBN:0521400171.

Popper, K.R. (1963) Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge and Kegan Paul, London. 412 pp. ISBN-13: 978-0415285940.

Superbie, C., Stewart, K.M., Regan, C.E., Johnstone, J.F. & McLoughlin, P.D. (2022) Northern boreal caribou conservation should focus on anthropogenic disturbance, not disturbance-mediated apparent competition. Biological Conservation, 265, 109426. doi: 10.1016/j.biocon.2021.109426.

Weiner, J. (1995) On the practice of ecology. Journal of Ecology, 83, 153-158.

Woodward, J. & Goodstein, D. (1996) Conduct, misconduct and the structure of science. American Scientist, 84, 479-490.

The Problem of Evidence in Ecology

Leave a reply

The good news is that the general public are becoming more concerned about the problems of wildlife management and conservation in general. The bad news arising from this interest is the lack of understanding exhibited by many of the comments in the media about ecological problems. This leads to a suggestion that we need an ecological “fact checking” team that looks at what is said about broad scale environmental issues and points out how much evidence there is for what is stated in the media. My interest in this issue is driven by so many news stories that are stated as fact with very little scientific understanding. Too many well-meaning reports fly around the media that border on complete error or complete nonsense. One consequence of this problem is a failure of evidence-based decision making for ecological problems (Christie et al. 2022).

This is not of course a problem confined to ecological science as you can see by reading nonsense claims about medical issues like Covid. It will not go away and with the climate crisis the number of ‘experts’ has multiplied. The problem comes down to the issue of evidence and how we evaluate evidence. A partial solution to this is better education about what is evidence in ecology as well as all of science. We need to teach workshops or courses on concrete examples of what is suggested to be evidence in ecological papers. The first step might be to analyse one or a few papers with the following procedure:

  1. What is the major conclusion of the paper?
  2. What data are presented to reach this conclusion?
  3. What background assumptions are being made to move from data to conclusions?

These questions lead us back to basic questions illustrated well by statistical inference. What is the ‘population’ to which the major conclusions apply? There is very little discussion of this in most ecological papers and the consequence can be overgeneralizations. Suppose for example we are examining the hypothesis that the geographic range of a species set is moving toward the poles because of a warming climate. We must for practical purposes restrict our study to a small set of species, so this is a major assumption that the species selected are a random sample of the biota under discussion. Another limitation is that it may be difficult to isolate climate change without considering for example human disturbances to the landscape from forestry and agriculture. A consequence of these complications is that our major conclusion for all this research rests on minimal data. So, a conclusion might be that we need to design further extensive studies. But perhaps of the 6 species under study, 4 are moving as the climate hypothesis predicts, but one is not moving at all, and one is moving in the opposite direction to what is predicted. Do we now turn our attention to these anomalous species that do not follow our major hypothesis? Or should we be happy that most of our candidate species follow the rule specified in our major conclusion?

       By doing manipulative experiments ecologists attempt to insert more rigor into their conclusions, but many of the generic questions mentioned above apply equally to these experimental designs. If we do a set of experiments in Iowa and in Germany, should we get the same results? We are back to the question of generality in all our studies. We hope for global rules, but experiments are all limited in time and space.

Can we escape all these bottlenecks with models that capture the generality and behave according to our assumptions? But models suffer from the same problems that make empirical studies difficult – what are the hidden assumptions? Taper et al. (2021) discuss the problem of errors arising from model misspecification in evaluating empirical data. Perhaps every ecological publication should end with an additional short section listing the assumptions made in reaching the major conclusions of the research.

These points come to the fore when we attempt to predict future environmental changes. A simple example is the hypothesis that, by humans increasing CO2 in the atmosphere, plants will increase photosynthesis and thus negate part or all the effects of climate change on our current ecosystems. This has caused much discussion ranging from planting more trees to alleviate climate change to relying on engineering solutions to climate change.

The bottom line that we should all recognize is that our predictions in ecology and our understanding of ecosystem changes are more limited than we admit. We know that we cannot rely on the old adage of the equilibrium hypothesis that “Mother Nature will take care of the earth” so all will be well. Wisdom always relies on critical evaluations which are too often lost in the media of our current world.

An important alternative approach is illustrated by the Conservation Evidence Journal and the approaches recommended by Sutherland et al. (2022) to specify local actions that can improve the conservation status of particular species or groups of species, for example by reintroducing birds to islands or areas from which they have been extirpated. The dichotomy here is a divide between the particular and the general, from short-term local questions to long-term general questions (Saunders et al. 2020). The hope is that progress on local questions will gradually inform the dominant global theories of ecology to bring them together and support the “devil in the details’ approach that can define ecological progress in our time (Sutherland et al. 2021).

Christie, A.P., et al. (2022) Principles for the production of evidence-based guidance for conservation actions. Conservation Science and Practice, 4, e579.doi: 10.1111/csp2.12663 .

Saunders, M.E., Janes, J.K. & O’Hanlon, J.C. (2020) Moving on from the Insect Apocalypse Narrative: Engaging with Evidence-Based Insect Conservation. BioScience, 70, 80-89.doi: 10.1093/biosci/biz143.

Sutherland, W.J., Downey, H., Frick, W.F., Tinsley-Marshall, P. & McPherson, T. (2021) Planning practical evidence-based decision making in conservation within time constraints: the Strategic Evidence Assessment Framework. Journal for Nature Conservation, 60, 125975.doi: 10.1016/j.jnc.2021.125975.

Sutherland, W.J. et al. (2022) Creating testable questions in practical conservation: a process and 100 questions. Conservation Evidence Journal, 19, 1-7.doi: 10.52201/CEJ19XIFF2753.

Taper, M., Lele, S., Ponciano, J., Dennis, B. & Jerde, C. (2021) Assessing the global and local uncertainty of scientific evidence in the presence of model misspecification Frontiers in Ecology and Evolution, 9, 679155.doi: 10.3389/fevo.2021.679155.

The Problem of Time in Ecology

Leave a reply

There is a problem in doing ecological studies that is too little discussed – what is the time frame of a good study? The normal response would be that the time frame varies with each study so that no guidelines can be provided. There is increasing recognition that more long-term studies are needed in ecology (e.g. Hughes et al. 2017) but the guidelines remain unclear.

The first issue is usually to specify a time frame, e.g. 5 years, 10 years. But this puts the cart before the horse, as the first step ought to be to define the hypothesis being investigated. In practice hypotheses in many ecological papers are poorly presented and there should not be one hypothesis but a series of alternative hypotheses. Given that, the question of time can be given with more insight. How many replicated time periods do you need to measure the ecological variables in the study? If your time scale unit is one year, 2 or 3 years is not enough to come to any except very tentative conclusions. We have instantly fallen into a central dilemma of ecology – studies are typically planned and financed on a 3–5-year time scale, the scale of university degrees.

Now we come up against the fact of climate change and the dilemma of trying to understand a changing system when almost all field work assumes an unchanging environment. Taken to some extreme we might argue that what happens in this decade tells us little about what will happen in the next decade. The way around this problem is to design experiments to test the variables that are changing ahead of time, e.g., what a 5⁰C temperature increase will do to the survival of your corals. To follow this approach, which is the classic experimental approach of science, we must assume we know the major variables affecting our population or community changes. At present we do not know the answer to this question, and we rely on correlations of a few variables as predictors of how large a change to expect.

There is no way out of this empirical box, which defines clearly how physics and chemistry differ from ecology and medicine. There are already many large-scale illustrations of this problem. Forest companies cut down old-growth timber on the assumption that they can get the forest back by replanting seedlings in the harvested area. But what species of tree seedlings should we replant if we are concerned that reforestation often operates on a 100–500-year time scale? And in most cases, there is no consideration of the total disruption of the ecosystem, and we ignore all the non-harvestable biodiversity. Much research is now available on reforestation and the ecological problems it produces. Hole-nesting birds can be threatened if old trees with holes are removed for forestry or agricultural clearing (Saunders et al. 2023). Replanting trees after fire in British Columbia did not increase carbon storage over 55 years of recovery when compared with unplanted sites (Clason et al. 2022). Consequently, in some forest ecosystems tree planting may not be useful if carbon storage is the desired goal.

At the least we should have more long-term monitoring of the survival of replanted forest tree seedlings so that the economics of planting could be evaluated. Short-term Australian studies in replanted agricultural fields showed over 4 years differences in survival of different plant species (Jellinek et al. 2020). For an on-the-ground point of view story about tree planting in British Columbia see:
https://thetyee.ca/Opinion/2023/11/02/Dont-Thank-Me-Being-Tree-Planter/. But we need longer-term studies on control and replanted sites to be more certain of effective restoration management. Gibson et al. (2022) highlighted the fact that citizen science over a 20-year study could make a major contribution to measuring the effectiveness of replanting. Money is always in short supply in field ecology and citizen science is one way of achieving goals without too much cost. 

Forest restoration is only one example of applied ecology in which long-term studies are too infrequent. The scale of restoration of temperate and boreal ecosystems is around 100 years, and this points to one of the main failures of long-term studies, that they are difficult to carry on after the retirement of the principal investigators who designed the studies.

The Park Grass Experiment begun in 1856 on 2.8 ha of grassland in England is the oldest ecological experiment in existence (Silvertown et al. 2006). As such it is worth a careful evaluation for the questions it asked and did not ask, for the scale of the experiment, and for the experimental design. It raises the question of generality for all long-term studies and cautions us about the utility and viability of many of the large-scale, long-term studies now in progress or planned for the future.

The message of this discussion is that we should plan for long-term studies for most of our critical ecological problems with clear hypotheses of how to conserve biodiversity and manage our agricultural landscapes and forests. We should move away from 2–3-year thesis projects on isolated issues and concentrate on team efforts that address critical long-term issues with specific hypotheses. Which says in a nutshell that we must develop a vision that goes beyond our past practices in scatter-shot, short-term ecology and at the same time avoid poorly designed long-term studies of the future.

Clason, A.J., Farnell, I. & Lilles, E.B. (2022) Carbon 5–60 Years After Fire: Planting Trees Does Not Compensate for Losses in Dead Wood Stores. Frontiers in Forests and Global Change, 5, 868024. doi: 10.3389/ffgc.2022.868024.

Gibson, M., Maron, M., Taws, N., Simmonds, J.S. & Walsh, J.C. (2022) Use of citizen science datasets to test effects of grazing exclusion and replanting on Australian woodland birds. Restoration Ecology, 30, e13610. doi: 10.1111/rec.13610.

Hughes, B.B.,et al. (2017) Long-term studies contribute disproportionately to ecology and policy. BioScience, 67, 271-281. doi.: 10.1093/biosci/biw185.

Jellinek, S., Harrison, P.A., Tuck, J. & Te, T. (2020) Replanting agricultural landscapes: how well do plants survive after habitat restoration? Restoration Ecology, 28, 1454-1463. doi: 10.1111/rec.13242.

Saunders, D.A., Dawson, R. & Mawson, P.R. (2023) Artificial nesting hollows for the conservation of Carnaby’s cockatoo Calyptorhynchus latirostris: definitely not a case of erect and forget. Pacific Conservation Biology, 29, 119-129. doi: 10.1071/PC21061.

Silvertown, J., Silvertown, J., Poulton, P. & Biss, P.M. (2006) The Park Grass Experiment 1856–2006: its contribution to ecology. Journal of Ecology, 94, 801-814. doi: 10.1111/j.1365-2745.2006.01145.x.

Back to Nature vs. Nurture

2 Replies

The ancient argument of ‘nature’ versus ‘nurture’ continues to arise in biology. The question has arisen very forcefully in a new book by James Tabery (Tabery 2023). The broad question he examines in this book is the conflict between ‘nature’ and ‘nurture’ in western medicine. In a broad sense ‘nature’ is discussed as the modern push in medicine to find the genetic basis of some of the common human degenerative diseases – Parkinson’s, dementia, asthma, diabetes, cancer, hypertension – to mention only a few medical problems of our day. The ‘nature’ approach to medicine in this book is represented by molecular genetics and the Human Genome Project. The ‘nurture’ approach to treating these medical conditions is via studying health outcomes in people subject to environmental contamination, atmospheric pollution, water quality, chemicals in food preparations, asbestos in buildings, and other environmental issues including how children are raised and educated. The competition over these two approaches was won very early by the Human Genome Project, and many of the resources for medicine over the last 30 years were put into molecular biology which made spectacular progress in diving into the genome of affected people and then making great promises of personalized medicine. The environmental approach to these medical conditions received much less money and was not viewed as sufficiently scientific. The irony of all this in retrospect is that the ‘nature’ or DNA school had no hypotheses about the problems being investigated but relied on the assumption that if we got enough molecular genetic data on thousands of people that something would jump out at us, and we would locate for example the gene(s) causing Parkinson’s, and then we could alter these genes with gene therapy or specific pharmaceuticals. By contrast the ‘nurture’ school had many specific hypotheses to test about air pollution and children’s health, about lead in municipal water supply and brain damage, and a host of very specific insights about how some of these health problems could be alleviated by legislation and changes in diet for example.

So, the question then becomes where are we today? The answer Tabery (2023) gives is that the ‘nature’ or molecular genetic “personalized medicine” approach has largely failed in achieving its goals despite the large amount of money invested because there is no single or small set of genes that cause specific diseases, but many genes that have complex interactions. In contrast, the ‘nurture’ school has made progress in identifying conditions that help decrease the occurrence of some of our common diseases, realizing that the problems are often difficult because they require changes in human behaviour like stopping smoking or improving diets.

All this discussion would possibly produce the simple conclusion that both “nature” and “nurture” are both involved in these complex human conditions. So, what could this medical discussion tell us about the condition of modern ecological science? I think two things perhaps. First, it is a general error to use science without hypotheses. Yet this is too often what ecologists do – gather a large amount of data that can be measured without too much prolonged effort and then try to make sense of it by applying hypotheses after the fact. And second, technology in ecology can be a benefit or a curse. Take, for example, the advances in vertebrate ecology that have come from the ability to describe the movements of individual animals in space. To have a map of hundreds of locations of an individual animal provides good natural history but does not address any specific hypothesis. Contrast this approach with that of Studd et al. (2021) and Shiratsuru et al. (2023) who use movement data to test important questions about kill rates of predators on different species of prey.

Many large-scale ecological approaches suffer from the same problem as the ‘nature’ paradigm – use ‘big science’ to measure many variables and then try to answer some important question for example about how climate change is affecting communities of plants and animals. Nagy et al. (2021) and Li et al. (2022) provide excellent examples of this approach. Schimel and Keller (2015) discuss what is needed to bring hypothesis testing to ‘big science’. Lindenmayer et al. (2018) discuss how conventional, question-driven long-term monitoring and hypothesis testing need to be combined with ‘big science’ to better ecological understanding. Pau et al. (2022) give a warning of how ‘big science’ data from airborne imaging can fail to agree with ground-based field studies in one core NEON grassland site in central USA.

The conclusion to date is that there is little integration in ecology of the equivalent of “nature” and “nurture” in medicine if in ecology we match ‘big science’ with ‘nature’ and field studies on the ground with ‘nurture’. Without that integration we risk in future another negative review in ecology like that provided now by Tabery (2023) for medical approaches to human diseases.

Lindenmayer, D.B., Likens, G.E. & Franklin, J.F. (2018) Earth Observation Networks (EONs): Finding the Right Balance. Trends in Ecology & Evolution, 33, 1-3.doi: 10.1016/j.tree.2017.10.008.

Li, D., et al. (2022) Standardized NEON organismal data for biodiversity research. Ecosphere, 13, e4141.doi:10.1002/ecs2.4141.

Nagy, R.C., et al. (2021) Harnessing the NEON data revolution to advance open environmental science with a diverse and data-capable community. Ecosphere, 12, e03833.doi: 10.1002/ecs2.3833.

Pau, S., et al. (2022) Poor relationships between NEON Airborne Observation Platform data and field-based vegetation traits at a mesic grassland. Ecology, 103, e03590.doi: 10.1002/ecy.3590.

Schimel, D. & Keller, M. (2015) Big questions, big science: Meeting the challenges of global ecology. Oecologia, 177, 925-934.doi: 10.1007/s00442-015-3236-3.

Shiratsuru, S., Studd, E.K., Majchrzak, Y.N., Peers, M.J.L., Menzies, A.K., Derbyshire, R., Jung, T.S., Krebs, C.J., Murray, D.L., Boonstra, R. & Boutin, S. (2023) When death comes: Prey activity is not always predictive of diel mortality patterns and the risk of predation. Proceedings of the Royal Society B, 290, 20230661.doi.

Studd, E.K., Derbyshire, R.E., Menzies, A.K., Simms, J.F., Humphries, M.M., Murray, D.L. & Boutin, S. (2021) The Purr-fect Catch: Using accelerometers and audio recorders to document kill rates and hunting behaviour of a small prey specialist. Methods in Ecology and Evolution, 12, 1277-1287.doi. 10.1111/2041-210X.13605

Tabery, J. (2023) Tyranny of the Gene: Personalized Medicine and the Threat to Public Health. Knopf Doubleday Publishing Group, New York. 336 pp. ISBN: 9780525658207.

The Meaningless of Random Sampling

Leave a reply

Statisticians tell us that random sampling is necessary for making general inferences from the particular to the general. If field ecologists accept this dictum, we can only conclude that it is very difficult to nearly impossible to reach generality. We can reach conclusions about specific local areas, and that is valuable, but much of our current ecological wisdom on populations and communities relies on the faulty model of non-random sampling. We rarely try to define the statistical ‘population’ which we are studying and attempting to make inferences about with our data. Some examples might be useful to illustrate this problem.

Marine ecologists ae mostly agreed that sea surface temperature rise is destroying coral reef ecosystems. This is certainly true, but it camouflages the fact that very few square kilometres of coral reefs like the Great Barrier Reef have been comprehensively studied with a proper sampling design (e.g. Green 1979, Lewis 2004). When we analyse the details of coral reef declines, we find that many species are affected by rising sea temperatures, but some are not, and it is possible that some species will adapt by natural selection to the higher temperatures. So we quite rightly raise the alarm about the future of coral reefs. But in doing so we neglect in many cases to specify the statistical ‘population’ to which our conclusions apply.

Most people would agree that such an approach to generalizing ecological findings is tantamount to saying the problem is “how many angels can dance on the head of a pin”, and in practice we can ignore the problem and generalize from the studied reefs to all reefs. And scientists would point out that physics and chemistry seek generality and ignore this problem because one can do chemistry in Zurich or in Toronto and use the same laws that do not change with time or place. But the ecosystems of today are not going to be the ecosystems of tomorrow, so generality in time cannot be guaranteed, as paleoecologists have long ago pointed out.

It is the spatial problem of field studies that collides most strongly with the statistical rule to random sample. Consider a hypothetical example of a large national park that has recently been burned by this year’s fires in the Northern Hemisphere. If we wish to measure the recovery process of the vegetation, we need to set out plots to resample. We have two choices: (1) lay out as many plots as possible, and sample these for several years to plot recovery. Or (2) lay out plots at random each year, never repeating the same exact areas to satisfy the specifications of statisticians to “random sample” the recovery in the park. We typically would do (1) for two reasons. Setting up new plots each year as per (2) would greatly increase the initial field work of defining the random plots and would probably mean that travel time between the plots would be greatly increased. Using approach (1) we would probably set out plots with relatively easy access from roads or trails to minimize costs of sampling. We ignore the advice of statisticians because of our real-world constraints of time and money. And we hope to answer the initial questions about recovery with this simpler design.

I could find few papers in the ecological literature that discuss this general problem of inference from the particular to the general (Ives 2018, Hauss 2018) and only one that deals with a real-world situation (Ducatez 2019). I would be glad to be sent more references on this problem by readers.

The bottom line is that if your supervisor or research coordinator criticizes your field work because your study areas are not randomly placed or your replicate sites were not chosen at random, tell him or her politely that virtually no ecological research in the field is done by truly random sampling. Does this make our research less useful for achieving ecological understanding – probably not. And we might note that medical science works in exactly the same way field ecologists work, do what you can with the money and time you have. The law that scientific knowledge requires random sampling is often a pseudo-problem in my opinion.  

Ducatez, S. (2019) Which sharks attract research? Analyses of the distribution of research effort in sharks reveal significant non-random knowledge biases. Reviews in Fish Biology and Fisheries, 29, 355-367. doi. 10.1007/s11160-019-09556-0

Green, R.H. (1979) Sampling Design and Statistical Methods for Environmental Biologists. Wiley, New York. 257 pp.

Hauss, K. (2018) Statistical Inference from Non-Random Samples. Problems in Application and Possible Solutions in Evaluation Research. Zeitschrift fur Evaluation, 17, 219-240. doi.

Ives, A.R. (2018) Informative Irreproducibility and the Use of Experiments in Ecology. BioScience, 68, 746-747. doi. 10.1093/biosci/biy090

Lewis, J. (2004) Has random sampling been neglected in coral reef faunal surveys? Coral Reefs, 23, 192-194. doi: 10.1007/s00338-004-0377-y.

The Time Frame of Ecological Science

Leave a reply

Ecological research differs from many branches of science in having a more convoluted time frame. Most of the sciences proceed along paths that are more often than not linear – results A → results B → results C and so on. Of course, these are never straight linear sequences and were described eloquently by Platt (1964) as strong inference:

“Strong inference consists of applying the following steps to every problem in science, formally and explicitly and regularly: 1) Devising alternative hypotheses; 2) Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly as possible, exclude one or more of the hypotheses; 3) Carrying out the experiment so as to get a clean result; “Recycling the procedure, making sequential hypotheses to refine the possibilities that remain; and so on. It is like climbing a tree.” (page 347 in Platt).

If there is one paper that I would recommend all ecologists read it is this paper which is old but really is timeless and critical in our scientific research. It should be a required discussion topic for every graduate student in ecology.

Some ecological science progresses as Platt (1964) suggests and makes good progress, but much of ecology is lost in a failure to specify alternative hypotheses, in changing questions, in abandoning topics because they are too difficult, and in a shortage of time. It is the time component of ecological research that I wish to discuss in this blog.

The idea of long-term studies has always been present in ecology but was perhaps brought to our focus by the compilation by Gene Likens in 1989 in a book of 14 chapters that are as vital now as they were 34 years ago. Many discussions of long-term studies are now available to examine this issue. Buma et al. (2019) for example discuss plant primary succession at Glacier Bay, Alaska which has 100 years of data, and which illustrates in a very slow ecosystem a test of conventional rules of community development. Cusser et al. (2021) follow this by asking a critical question of how long field experiments need to be. They restrict long-term to be > 10 years of study and used data from the USA LTER sites. This question depends very much on the community or ecosystem of study. Studies in areas with a stable climate produced results more quickly than those in highly seasonal environments, and plant studies needed to be longer term than animal studies to reach stable conclusions. Ten years may not be enough.

Reinke et al. (2019) reviewed 3 long term field studies and suggest that long-term studies can be useful to allow us to predict how ecosystems will change with time. All these studies lead to three unanswered questions that are critical for progress in ecology. The first question is how we decide as a community exactly which ecological system we should be studying long-term. No one knows how to answer this question, and a useful graduate seminar could debate the utility of what are now considered model long-term studies, such as the three highlighted in Reinke et al. (2019) or the Park Grass Experiment (Addy et al. 2022). At the moment these decisions are opportunistic, and we should debate how best to proceed. Clearly, we cannot do everything for every population and community of interest, so how do we choose? We need model systems that can be applied to a wide variety of environments across the globe and that ask questions of global significance. Many groups of ecologists are trying to do this, but a host of decisions about who to fund and support in what institution are vital to avoid long-term studies driven more by convenience than by ecological importance.

A second question involves the implied disagreement whether many important questions in ecology today could be answered by short-term studies, so we reach a position where there is competition between short- and long-term funding. These decisions about where to do what for how long are largely uncontrolled. One would prefer to see an articulated set of hypotheses and predictions to proceed with decision making, whether for short-term studies suitable for graduate students or particularly for long-term studies that exceed the life of individual researchers.

A third question is the most difficult one of the objectives of long-term research. Given climate change as it is moving today, the hope that long-term studies will give us reliable predictions of changes in communities and ecosystems is at risk, the same problem of extrapolating a regression line beyond the range of the data. Depending on the answer to this climate dilemma, we could drop back to the suggestion that because we have only a poor ability to predict ecological change, we should concentrate more on widespread monitoring programs and less on highly localized studies of a few sites that are of unknown generality. Testing models with long-term data is enriching the ecological literature (e.g. Addy et al 2022). But the challenge is whether our current understanding is sufficient to make predictions for future populations or communities. Should ecology adopt the paradigm of global weather stations?

Addy, J.W.G., Ellis, R.H., MacLaren, C., Macdonald, A.J., Semenov, M.A. & Mead, A. (2022) A heteroskedastic model of Park Grass spring hay yields in response to weather suggests continuing yield decline with climate change in future decades. Journal of the Royal Society Interface, 19, 20220361. doi: 10.1098/rsif.2022.0361.

Buma, B., Bisbing, S.M., Wiles, G. & Bidlack, A.L. (2019) 100 yr of primary succession highlights stochasticity and competition driving community establishment and stability. Ecology, 100, e02885. doi: 10.1002/ecy.2885.

Cusser, S., Helms IV, J., Bahlai, C.A. & Haddad, N.M. (2021) How long do population level field experiments need to be? Utilising data from the 40-year-old LTER network. Ecology Letters, 24, 1103-1111. doi: 10.1111/ele.13710.

Hughes, B.B., Beas-Luna, R., Barner, A., et al. (2017) Long-term studies contribute disproportionately to ecology and policy. BioScience, 67, 271-281. doi: 10.1093/biosci/biw185.

Likens, G.E. (Editor, 1989) Long-term Studies in Ecology: Approaches and Alternatives. Springer Verlag, New York. 214 pp. ISBN: 0387967435.

Platt, J.R. (1964) Strong inference. Science, 146, 347-353. doi: 10.1126/science.146.3642.347.

Reinke, B.A., Miller, D.A.W. & Janzen, F.J. (2019) What have long-term field studies taught as about population dynamics? Annual Review of Ecology, Evolution, and Systematics, 50, 261-278. doi: 10.1146/annurev-ecolsys-110218-024717.