Category Archives: Evaluating Research Quality

The Problem of Evidence in Ecology

Leave a reply

The good news is that the general public are becoming more concerned about the problems of wildlife management and conservation in general. The bad news arising from this interest is the lack of understanding exhibited by many of the comments in the media about ecological problems. This leads to a suggestion that we need an ecological “fact checking” team that looks at what is said about broad scale environmental issues and points out how much evidence there is for what is stated in the media. My interest in this issue is driven by so many news stories that are stated as fact with very little scientific understanding. Too many well-meaning reports fly around the media that border on complete error or complete nonsense. One consequence of this problem is a failure of evidence-based decision making for ecological problems (Christie et al. 2022).

This is not of course a problem confined to ecological science as you can see by reading nonsense claims about medical issues like Covid. It will not go away and with the climate crisis the number of ‘experts’ has multiplied. The problem comes down to the issue of evidence and how we evaluate evidence. A partial solution to this is better education about what is evidence in ecology as well as all of science. We need to teach workshops or courses on concrete examples of what is suggested to be evidence in ecological papers. The first step might be to analyse one or a few papers with the following procedure:

  1. What is the major conclusion of the paper?
  2. What data are presented to reach this conclusion?
  3. What background assumptions are being made to move from data to conclusions?

These questions lead us back to basic questions illustrated well by statistical inference. What is the ‘population’ to which the major conclusions apply? There is very little discussion of this in most ecological papers and the consequence can be overgeneralizations. Suppose for example we are examining the hypothesis that the geographic range of a species set is moving toward the poles because of a warming climate. We must for practical purposes restrict our study to a small set of species, so this is a major assumption that the species selected are a random sample of the biota under discussion. Another limitation is that it may be difficult to isolate climate change without considering for example human disturbances to the landscape from forestry and agriculture. A consequence of these complications is that our major conclusion for all this research rests on minimal data. So, a conclusion might be that we need to design further extensive studies. But perhaps of the 6 species under study, 4 are moving as the climate hypothesis predicts, but one is not moving at all, and one is moving in the opposite direction to what is predicted. Do we now turn our attention to these anomalous species that do not follow our major hypothesis? Or should we be happy that most of our candidate species follow the rule specified in our major conclusion?

       By doing manipulative experiments ecologists attempt to insert more rigor into their conclusions, but many of the generic questions mentioned above apply equally to these experimental designs. If we do a set of experiments in Iowa and in Germany, should we get the same results? We are back to the question of generality in all our studies. We hope for global rules, but experiments are all limited in time and space.

Can we escape all these bottlenecks with models that capture the generality and behave according to our assumptions? But models suffer from the same problems that make empirical studies difficult – what are the hidden assumptions? Taper et al. (2021) discuss the problem of errors arising from model misspecification in evaluating empirical data. Perhaps every ecological publication should end with an additional short section listing the assumptions made in reaching the major conclusions of the research.

These points come to the fore when we attempt to predict future environmental changes. A simple example is the hypothesis that, by humans increasing CO2 in the atmosphere, plants will increase photosynthesis and thus negate part or all the effects of climate change on our current ecosystems. This has caused much discussion ranging from planting more trees to alleviate climate change to relying on engineering solutions to climate change.

The bottom line that we should all recognize is that our predictions in ecology and our understanding of ecosystem changes are more limited than we admit. We know that we cannot rely on the old adage of the equilibrium hypothesis that “Mother Nature will take care of the earth” so all will be well. Wisdom always relies on critical evaluations which are too often lost in the media of our current world.

An important alternative approach is illustrated by the Conservation Evidence Journal and the approaches recommended by Sutherland et al. (2022) to specify local actions that can improve the conservation status of particular species or groups of species, for example by reintroducing birds to islands or areas from which they have been extirpated. The dichotomy here is a divide between the particular and the general, from short-term local questions to long-term general questions (Saunders et al. 2020). The hope is that progress on local questions will gradually inform the dominant global theories of ecology to bring them together and support the “devil in the details’ approach that can define ecological progress in our time (Sutherland et al. 2021).

Christie, A.P., et al. (2022) Principles for the production of evidence-based guidance for conservation actions. Conservation Science and Practice, 4, e579.doi: 10.1111/csp2.12663 .

Saunders, M.E., Janes, J.K. & O’Hanlon, J.C. (2020) Moving on from the Insect Apocalypse Narrative: Engaging with Evidence-Based Insect Conservation. BioScience, 70, 80-89.doi: 10.1093/biosci/biz143.

Sutherland, W.J., Downey, H., Frick, W.F., Tinsley-Marshall, P. & McPherson, T. (2021) Planning practical evidence-based decision making in conservation within time constraints: the Strategic Evidence Assessment Framework. Journal for Nature Conservation, 60, 125975.doi: 10.1016/j.jnc.2021.125975.

Sutherland, W.J. et al. (2022) Creating testable questions in practical conservation: a process and 100 questions. Conservation Evidence Journal, 19, 1-7.doi: 10.52201/CEJ19XIFF2753.

Taper, M., Lele, S., Ponciano, J., Dennis, B. & Jerde, C. (2021) Assessing the global and local uncertainty of scientific evidence in the presence of model misspecification Frontiers in Ecology and Evolution, 9, 679155.doi: 10.3389/fevo.2021.679155.

The Problem of Time in Ecology

Leave a reply

There is a problem in doing ecological studies that is too little discussed – what is the time frame of a good study? The normal response would be that the time frame varies with each study so that no guidelines can be provided. There is increasing recognition that more long-term studies are needed in ecology (e.g. Hughes et al. 2017) but the guidelines remain unclear.

The first issue is usually to specify a time frame, e.g. 5 years, 10 years. But this puts the cart before the horse, as the first step ought to be to define the hypothesis being investigated. In practice hypotheses in many ecological papers are poorly presented and there should not be one hypothesis but a series of alternative hypotheses. Given that, the question of time can be given with more insight. How many replicated time periods do you need to measure the ecological variables in the study? If your time scale unit is one year, 2 or 3 years is not enough to come to any except very tentative conclusions. We have instantly fallen into a central dilemma of ecology – studies are typically planned and financed on a 3–5-year time scale, the scale of university degrees.

Now we come up against the fact of climate change and the dilemma of trying to understand a changing system when almost all field work assumes an unchanging environment. Taken to some extreme we might argue that what happens in this decade tells us little about what will happen in the next decade. The way around this problem is to design experiments to test the variables that are changing ahead of time, e.g., what a 5⁰C temperature increase will do to the survival of your corals. To follow this approach, which is the classic experimental approach of science, we must assume we know the major variables affecting our population or community changes. At present we do not know the answer to this question, and we rely on correlations of a few variables as predictors of how large a change to expect.

There is no way out of this empirical box, which defines clearly how physics and chemistry differ from ecology and medicine. There are already many large-scale illustrations of this problem. Forest companies cut down old-growth timber on the assumption that they can get the forest back by replanting seedlings in the harvested area. But what species of tree seedlings should we replant if we are concerned that reforestation often operates on a 100–500-year time scale? And in most cases, there is no consideration of the total disruption of the ecosystem, and we ignore all the non-harvestable biodiversity. Much research is now available on reforestation and the ecological problems it produces. Hole-nesting birds can be threatened if old trees with holes are removed for forestry or agricultural clearing (Saunders et al. 2023). Replanting trees after fire in British Columbia did not increase carbon storage over 55 years of recovery when compared with unplanted sites (Clason et al. 2022). Consequently, in some forest ecosystems tree planting may not be useful if carbon storage is the desired goal.

At the least we should have more long-term monitoring of the survival of replanted forest tree seedlings so that the economics of planting could be evaluated. Short-term Australian studies in replanted agricultural fields showed over 4 years differences in survival of different plant species (Jellinek et al. 2020). For an on-the-ground point of view story about tree planting in British Columbia see:
https://thetyee.ca/Opinion/2023/11/02/Dont-Thank-Me-Being-Tree-Planter/. But we need longer-term studies on control and replanted sites to be more certain of effective restoration management. Gibson et al. (2022) highlighted the fact that citizen science over a 20-year study could make a major contribution to measuring the effectiveness of replanting. Money is always in short supply in field ecology and citizen science is one way of achieving goals without too much cost. 

Forest restoration is only one example of applied ecology in which long-term studies are too infrequent. The scale of restoration of temperate and boreal ecosystems is around 100 years, and this points to one of the main failures of long-term studies, that they are difficult to carry on after the retirement of the principal investigators who designed the studies.

The Park Grass Experiment begun in 1856 on 2.8 ha of grassland in England is the oldest ecological experiment in existence (Silvertown et al. 2006). As such it is worth a careful evaluation for the questions it asked and did not ask, for the scale of the experiment, and for the experimental design. It raises the question of generality for all long-term studies and cautions us about the utility and viability of many of the large-scale, long-term studies now in progress or planned for the future.

The message of this discussion is that we should plan for long-term studies for most of our critical ecological problems with clear hypotheses of how to conserve biodiversity and manage our agricultural landscapes and forests. We should move away from 2–3-year thesis projects on isolated issues and concentrate on team efforts that address critical long-term issues with specific hypotheses. Which says in a nutshell that we must develop a vision that goes beyond our past practices in scatter-shot, short-term ecology and at the same time avoid poorly designed long-term studies of the future.

Clason, A.J., Farnell, I. & Lilles, E.B. (2022) Carbon 5–60 Years After Fire: Planting Trees Does Not Compensate for Losses in Dead Wood Stores. Frontiers in Forests and Global Change, 5, 868024. doi: 10.3389/ffgc.2022.868024.

Gibson, M., Maron, M., Taws, N., Simmonds, J.S. & Walsh, J.C. (2022) Use of citizen science datasets to test effects of grazing exclusion and replanting on Australian woodland birds. Restoration Ecology, 30, e13610. doi: 10.1111/rec.13610.

Hughes, B.B.,et al. (2017) Long-term studies contribute disproportionately to ecology and policy. BioScience, 67, 271-281. doi.: 10.1093/biosci/biw185.

Jellinek, S., Harrison, P.A., Tuck, J. & Te, T. (2020) Replanting agricultural landscapes: how well do plants survive after habitat restoration? Restoration Ecology, 28, 1454-1463. doi: 10.1111/rec.13242.

Saunders, D.A., Dawson, R. & Mawson, P.R. (2023) Artificial nesting hollows for the conservation of Carnaby’s cockatoo Calyptorhynchus latirostris: definitely not a case of erect and forget. Pacific Conservation Biology, 29, 119-129. doi: 10.1071/PC21061.

Silvertown, J., Silvertown, J., Poulton, P. & Biss, P.M. (2006) The Park Grass Experiment 1856–2006: its contribution to ecology. Journal of Ecology, 94, 801-814. doi: 10.1111/j.1365-2745.2006.01145.x.

Back to Nature vs. Nurture

2 Replies

The ancient argument of ‘nature’ versus ‘nurture’ continues to arise in biology. The question has arisen very forcefully in a new book by James Tabery (Tabery 2023). The broad question he examines in this book is the conflict between ‘nature’ and ‘nurture’ in western medicine. In a broad sense ‘nature’ is discussed as the modern push in medicine to find the genetic basis of some of the common human degenerative diseases – Parkinson’s, dementia, asthma, diabetes, cancer, hypertension – to mention only a few medical problems of our day. The ‘nature’ approach to medicine in this book is represented by molecular genetics and the Human Genome Project. The ‘nurture’ approach to treating these medical conditions is via studying health outcomes in people subject to environmental contamination, atmospheric pollution, water quality, chemicals in food preparations, asbestos in buildings, and other environmental issues including how children are raised and educated. The competition over these two approaches was won very early by the Human Genome Project, and many of the resources for medicine over the last 30 years were put into molecular biology which made spectacular progress in diving into the genome of affected people and then making great promises of personalized medicine. The environmental approach to these medical conditions received much less money and was not viewed as sufficiently scientific. The irony of all this in retrospect is that the ‘nature’ or DNA school had no hypotheses about the problems being investigated but relied on the assumption that if we got enough molecular genetic data on thousands of people that something would jump out at us, and we would locate for example the gene(s) causing Parkinson’s, and then we could alter these genes with gene therapy or specific pharmaceuticals. By contrast the ‘nurture’ school had many specific hypotheses to test about air pollution and children’s health, about lead in municipal water supply and brain damage, and a host of very specific insights about how some of these health problems could be alleviated by legislation and changes in diet for example.

So, the question then becomes where are we today? The answer Tabery (2023) gives is that the ‘nature’ or molecular genetic “personalized medicine” approach has largely failed in achieving its goals despite the large amount of money invested because there is no single or small set of genes that cause specific diseases, but many genes that have complex interactions. In contrast, the ‘nurture’ school has made progress in identifying conditions that help decrease the occurrence of some of our common diseases, realizing that the problems are often difficult because they require changes in human behaviour like stopping smoking or improving diets.

All this discussion would possibly produce the simple conclusion that both “nature” and “nurture” are both involved in these complex human conditions. So, what could this medical discussion tell us about the condition of modern ecological science? I think two things perhaps. First, it is a general error to use science without hypotheses. Yet this is too often what ecologists do – gather a large amount of data that can be measured without too much prolonged effort and then try to make sense of it by applying hypotheses after the fact. And second, technology in ecology can be a benefit or a curse. Take, for example, the advances in vertebrate ecology that have come from the ability to describe the movements of individual animals in space. To have a map of hundreds of locations of an individual animal provides good natural history but does not address any specific hypothesis. Contrast this approach with that of Studd et al. (2021) and Shiratsuru et al. (2023) who use movement data to test important questions about kill rates of predators on different species of prey.

Many large-scale ecological approaches suffer from the same problem as the ‘nature’ paradigm – use ‘big science’ to measure many variables and then try to answer some important question for example about how climate change is affecting communities of plants and animals. Nagy et al. (2021) and Li et al. (2022) provide excellent examples of this approach. Schimel and Keller (2015) discuss what is needed to bring hypothesis testing to ‘big science’. Lindenmayer et al. (2018) discuss how conventional, question-driven long-term monitoring and hypothesis testing need to be combined with ‘big science’ to better ecological understanding. Pau et al. (2022) give a warning of how ‘big science’ data from airborne imaging can fail to agree with ground-based field studies in one core NEON grassland site in central USA.

The conclusion to date is that there is little integration in ecology of the equivalent of “nature” and “nurture” in medicine if in ecology we match ‘big science’ with ‘nature’ and field studies on the ground with ‘nurture’. Without that integration we risk in future another negative review in ecology like that provided now by Tabery (2023) for medical approaches to human diseases.

Lindenmayer, D.B., Likens, G.E. & Franklin, J.F. (2018) Earth Observation Networks (EONs): Finding the Right Balance. Trends in Ecology & Evolution, 33, 1-3.doi: 10.1016/j.tree.2017.10.008.

Li, D., et al. (2022) Standardized NEON organismal data for biodiversity research. Ecosphere, 13, e4141.doi:10.1002/ecs2.4141.

Nagy, R.C., et al. (2021) Harnessing the NEON data revolution to advance open environmental science with a diverse and data-capable community. Ecosphere, 12, e03833.doi: 10.1002/ecs2.3833.

Pau, S., et al. (2022) Poor relationships between NEON Airborne Observation Platform data and field-based vegetation traits at a mesic grassland. Ecology, 103, e03590.doi: 10.1002/ecy.3590.

Schimel, D. & Keller, M. (2015) Big questions, big science: Meeting the challenges of global ecology. Oecologia, 177, 925-934.doi: 10.1007/s00442-015-3236-3.

Shiratsuru, S., Studd, E.K., Majchrzak, Y.N., Peers, M.J.L., Menzies, A.K., Derbyshire, R., Jung, T.S., Krebs, C.J., Murray, D.L., Boonstra, R. & Boutin, S. (2023) When death comes: Prey activity is not always predictive of diel mortality patterns and the risk of predation. Proceedings of the Royal Society B, 290, 20230661.doi.

Studd, E.K., Derbyshire, R.E., Menzies, A.K., Simms, J.F., Humphries, M.M., Murray, D.L. & Boutin, S. (2021) The Purr-fect Catch: Using accelerometers and audio recorders to document kill rates and hunting behaviour of a small prey specialist. Methods in Ecology and Evolution, 12, 1277-1287.doi. 10.1111/2041-210X.13605

Tabery, J. (2023) Tyranny of the Gene: Personalized Medicine and the Threat to Public Health. Knopf Doubleday Publishing Group, New York. 336 pp. ISBN: 9780525658207.

The Meaningless of Random Sampling

Leave a reply

Statisticians tell us that random sampling is necessary for making general inferences from the particular to the general. If field ecologists accept this dictum, we can only conclude that it is very difficult to nearly impossible to reach generality. We can reach conclusions about specific local areas, and that is valuable, but much of our current ecological wisdom on populations and communities relies on the faulty model of non-random sampling. We rarely try to define the statistical ‘population’ which we are studying and attempting to make inferences about with our data. Some examples might be useful to illustrate this problem.

Marine ecologists ae mostly agreed that sea surface temperature rise is destroying coral reef ecosystems. This is certainly true, but it camouflages the fact that very few square kilometres of coral reefs like the Great Barrier Reef have been comprehensively studied with a proper sampling design (e.g. Green 1979, Lewis 2004). When we analyse the details of coral reef declines, we find that many species are affected by rising sea temperatures, but some are not, and it is possible that some species will adapt by natural selection to the higher temperatures. So we quite rightly raise the alarm about the future of coral reefs. But in doing so we neglect in many cases to specify the statistical ‘population’ to which our conclusions apply.

Most people would agree that such an approach to generalizing ecological findings is tantamount to saying the problem is “how many angels can dance on the head of a pin”, and in practice we can ignore the problem and generalize from the studied reefs to all reefs. And scientists would point out that physics and chemistry seek generality and ignore this problem because one can do chemistry in Zurich or in Toronto and use the same laws that do not change with time or place. But the ecosystems of today are not going to be the ecosystems of tomorrow, so generality in time cannot be guaranteed, as paleoecologists have long ago pointed out.

It is the spatial problem of field studies that collides most strongly with the statistical rule to random sample. Consider a hypothetical example of a large national park that has recently been burned by this year’s fires in the Northern Hemisphere. If we wish to measure the recovery process of the vegetation, we need to set out plots to resample. We have two choices: (1) lay out as many plots as possible, and sample these for several years to plot recovery. Or (2) lay out plots at random each year, never repeating the same exact areas to satisfy the specifications of statisticians to “random sample” the recovery in the park. We typically would do (1) for two reasons. Setting up new plots each year as per (2) would greatly increase the initial field work of defining the random plots and would probably mean that travel time between the plots would be greatly increased. Using approach (1) we would probably set out plots with relatively easy access from roads or trails to minimize costs of sampling. We ignore the advice of statisticians because of our real-world constraints of time and money. And we hope to answer the initial questions about recovery with this simpler design.

I could find few papers in the ecological literature that discuss this general problem of inference from the particular to the general (Ives 2018, Hauss 2018) and only one that deals with a real-world situation (Ducatez 2019). I would be glad to be sent more references on this problem by readers.

The bottom line is that if your supervisor or research coordinator criticizes your field work because your study areas are not randomly placed or your replicate sites were not chosen at random, tell him or her politely that virtually no ecological research in the field is done by truly random sampling. Does this make our research less useful for achieving ecological understanding – probably not. And we might note that medical science works in exactly the same way field ecologists work, do what you can with the money and time you have. The law that scientific knowledge requires random sampling is often a pseudo-problem in my opinion.  

Ducatez, S. (2019) Which sharks attract research? Analyses of the distribution of research effort in sharks reveal significant non-random knowledge biases. Reviews in Fish Biology and Fisheries, 29, 355-367. doi. 10.1007/s11160-019-09556-0

Green, R.H. (1979) Sampling Design and Statistical Methods for Environmental Biologists. Wiley, New York. 257 pp.

Hauss, K. (2018) Statistical Inference from Non-Random Samples. Problems in Application and Possible Solutions in Evaluation Research. Zeitschrift fur Evaluation, 17, 219-240. doi.

Ives, A.R. (2018) Informative Irreproducibility and the Use of Experiments in Ecology. BioScience, 68, 746-747. doi. 10.1093/biosci/biy090

Lewis, J. (2004) Has random sampling been neglected in coral reef faunal surveys? Coral Reefs, 23, 192-194. doi: 10.1007/s00338-004-0377-y.

The Time Frame of Ecological Science

Leave a reply

Ecological research differs from many branches of science in having a more convoluted time frame. Most of the sciences proceed along paths that are more often than not linear – results A → results B → results C and so on. Of course, these are never straight linear sequences and were described eloquently by Platt (1964) as strong inference:

“Strong inference consists of applying the following steps to every problem in science, formally and explicitly and regularly: 1) Devising alternative hypotheses; 2) Devising a crucial experiment (or several of them), with alternative possible outcomes, each of which will, as nearly as possible, exclude one or more of the hypotheses; 3) Carrying out the experiment so as to get a clean result; “Recycling the procedure, making sequential hypotheses to refine the possibilities that remain; and so on. It is like climbing a tree.” (page 347 in Platt).

If there is one paper that I would recommend all ecologists read it is this paper which is old but really is timeless and critical in our scientific research. It should be a required discussion topic for every graduate student in ecology.

Some ecological science progresses as Platt (1964) suggests and makes good progress, but much of ecology is lost in a failure to specify alternative hypotheses, in changing questions, in abandoning topics because they are too difficult, and in a shortage of time. It is the time component of ecological research that I wish to discuss in this blog.

The idea of long-term studies has always been present in ecology but was perhaps brought to our focus by the compilation by Gene Likens in 1989 in a book of 14 chapters that are as vital now as they were 34 years ago. Many discussions of long-term studies are now available to examine this issue. Buma et al. (2019) for example discuss plant primary succession at Glacier Bay, Alaska which has 100 years of data, and which illustrates in a very slow ecosystem a test of conventional rules of community development. Cusser et al. (2021) follow this by asking a critical question of how long field experiments need to be. They restrict long-term to be > 10 years of study and used data from the USA LTER sites. This question depends very much on the community or ecosystem of study. Studies in areas with a stable climate produced results more quickly than those in highly seasonal environments, and plant studies needed to be longer term than animal studies to reach stable conclusions. Ten years may not be enough.

Reinke et al. (2019) reviewed 3 long term field studies and suggest that long-term studies can be useful to allow us to predict how ecosystems will change with time. All these studies lead to three unanswered questions that are critical for progress in ecology. The first question is how we decide as a community exactly which ecological system we should be studying long-term. No one knows how to answer this question, and a useful graduate seminar could debate the utility of what are now considered model long-term studies, such as the three highlighted in Reinke et al. (2019) or the Park Grass Experiment (Addy et al. 2022). At the moment these decisions are opportunistic, and we should debate how best to proceed. Clearly, we cannot do everything for every population and community of interest, so how do we choose? We need model systems that can be applied to a wide variety of environments across the globe and that ask questions of global significance. Many groups of ecologists are trying to do this, but a host of decisions about who to fund and support in what institution are vital to avoid long-term studies driven more by convenience than by ecological importance.

A second question involves the implied disagreement whether many important questions in ecology today could be answered by short-term studies, so we reach a position where there is competition between short- and long-term funding. These decisions about where to do what for how long are largely uncontrolled. One would prefer to see an articulated set of hypotheses and predictions to proceed with decision making, whether for short-term studies suitable for graduate students or particularly for long-term studies that exceed the life of individual researchers.

A third question is the most difficult one of the objectives of long-term research. Given climate change as it is moving today, the hope that long-term studies will give us reliable predictions of changes in communities and ecosystems is at risk, the same problem of extrapolating a regression line beyond the range of the data. Depending on the answer to this climate dilemma, we could drop back to the suggestion that because we have only a poor ability to predict ecological change, we should concentrate more on widespread monitoring programs and less on highly localized studies of a few sites that are of unknown generality. Testing models with long-term data is enriching the ecological literature (e.g. Addy et al 2022). But the challenge is whether our current understanding is sufficient to make predictions for future populations or communities. Should ecology adopt the paradigm of global weather stations?

Addy, J.W.G., Ellis, R.H., MacLaren, C., Macdonald, A.J., Semenov, M.A. & Mead, A. (2022) A heteroskedastic model of Park Grass spring hay yields in response to weather suggests continuing yield decline with climate change in future decades. Journal of the Royal Society Interface, 19, 20220361. doi: 10.1098/rsif.2022.0361.

Buma, B., Bisbing, S.M., Wiles, G. & Bidlack, A.L. (2019) 100 yr of primary succession highlights stochasticity and competition driving community establishment and stability. Ecology, 100, e02885. doi: 10.1002/ecy.2885.

Cusser, S., Helms IV, J., Bahlai, C.A. & Haddad, N.M. (2021) How long do population level field experiments need to be? Utilising data from the 40-year-old LTER network. Ecology Letters, 24, 1103-1111. doi: 10.1111/ele.13710.

Hughes, B.B., Beas-Luna, R., Barner, A., et al. (2017) Long-term studies contribute disproportionately to ecology and policy. BioScience, 67, 271-281. doi: 10.1093/biosci/biw185.

Likens, G.E. (Editor, 1989) Long-term Studies in Ecology: Approaches and Alternatives. Springer Verlag, New York. 214 pp. ISBN: 0387967435.

Platt, J.R. (1964) Strong inference. Science, 146, 347-353. doi: 10.1126/science.146.3642.347.

Reinke, B.A., Miller, D.A.W. & Janzen, F.J. (2019) What have long-term field studies taught as about population dynamics? Annual Review of Ecology, Evolution, and Systematics, 50, 261-278. doi: 10.1146/annurev-ecolsys-110218-024717.

The Two Questions: So what? What next?

Leave a reply

Assuming that these two questions are not copyright, I wanted to explore them as a convenient part of writing a scientific or popular paper in ecology, conservation, and wildlife and fisheries management. To protect the innocent, I will not identify which of many ecological colleagues has stimulated this blog.

The first question should be addressed in every scientific paper but clearly is not if you read a random sample of the articles in many ecological journals. So what? is the critical question of exactly what current problem this paper or book will contribute to. It is the microscopic and macroscopic focus of why we do science, and it does not matter at all if it addresses a minor problem or a major catastrophe like species loss in conservation. In writing one should assume that time is the critical limiting factor in our lives, and while it is fine to be entertained by watching a movie, scientists do not read scientific papers to be entertained. Some journals demand that the abstract of every paper ends with a statement of the importance of the research findings, captured by So what? Too often these statements are weak and editors as well as granting agencies should demand more incisive statements. Asking yourself So what? can be a useful guide as you progress in your research and evaluate others.

While most scientists should agree on the findings presented in a paper or lecture, not all of them will agree about the importance of the answer to So what? What is a major and important scientific finding for some may be of minor significance to others, but the key is to remember here that science is a broad church that should be progressing on a broad front, so that differences of opinion are to be expected, and we rely on evidence to evaluate these differences of opinion. Tests of ideas that turn out to be incorrect or only partly correct must not be considered as failures. If you doubt that, interview any senior scientist in your area and ask about progress and regress during their scientific career. If you find a scientist who insists that they were correct in all their ideas, you should probably request them to go into politics to improve decision making in the real world.

The second question is probably the most critical for all scientific research. Once research is completed, there are two paths. If the original question or problem is solved or answered, the question becomes what does this work suggest needs to be done to advance the general area of research. Most typically however a research project will end up with more questions than it solves. The growing end of science is the critical one, and by asking What next? we delve deeper into the area of research to fill in details that were not evident when it was started. Read Sutherland et al. (2013, 2022) for an excellent example of this approach in conservation science. A simple example of this approach comes from many conservation problems. A particular species of bird may be thought to be declining in numbers, so the first issue is whether this is correct, and so an investigation into the changes in abundance of the species becomes the first step. This could lead to an analysis of the demography of the species population, birth, death and movement rates could be determined to isolate more precisely why abundance is changing. Given these data, the next step might be (for example) why the death rate is increasing if indeed this is the case. The next step is what management methods can be applied to reduce the death rate, and does this situation apply to other closely related species. It is important that asking What next? does not imply a linear sequence in time, and a study could be designed to address more than one question at the same time. We finish the What next? approach with a web of information and conclusions that address a broader question than the original simple question. And What next? should not be answered with a broad set of statements like “climate change is the cause” but by suggestions of very specific experiments and studies to carry investigations forward.

The result in ecology is an increasing precision of thought into ecological interactions and the processes that link species, communities, and ecosystems to very large questions such as the environmental response to climate change. Not all questions need to be large-scale because there are important local questions about the adequacy of designated parks and protected areas to protect species, communities, and ecosystems. The key message is that ecological understanding is not static but grows incrementally by well-designed research programs that by themselves seem to address only small-scale issues.

Seemingly failed research programs are not to be scorned but rather to indicate what avenues of research have not led to good insights. In a sense ecological science is like an evolutionary tree in which some branches fade away with time and others blossom into a variety of forms that surprise us all. So, my advice is to carry on asking these two simple questions in science to help sharpen your research program.

Sutherland, W.J., Freckleton, R.P., Godfray, H.C.J., Beissinger, S.R., Benton, T., Cameron, D.D., Carmel, Y., Coomes, D.A., Coulson, T., Emmerson, M.C., Hails, R.S., Hays, G.C., Hodgson, D.J., Hutchings, M.J., Johnson, D., Jones, J.P.G., Keeling, M.J., Kokko, H., Kunin, W.E. & Lambin, X. (2013) Identification of 100 fundamental ecological questions. Journal of Ecology, 101, 58-67.doi: 10.1111/1365-2745.12025.

Sutherland, W.J. & Jake M. Robinson, D.C.A., Tim Alamenciak, Matthew Armes, Nina Baranduin, Andrew J. Bladon, Martin F. Breed, Nicki Dyas, Chris S. Elphick, Richard A. Griffiths, Jonny Hughes, Beccy Middleton, Nick A. Littlewood, Roger Mitchell, William H. Morgan, Roy Mosley, Silviu O. Petrovan, Kit Prendergast, Euan G. Ritchie,Hugh Raven, Rebecca K. Smith, Sarah H. Watts, Ann Thornton (2022) Creating testable questions in practical conservation: a process and 100 questions. Conservation Evidence Journal, 19, 1-7.doi: 10.52201/CEJ19XIFF2753.

Belief vs. Evidence

2 Replies

There is an interesting game you could enter into if you classified the statements you hear or read in the media or in ecological papers. The initial dichotomy is whether or not a statement is a BELIEF or EVIDENCE BASED. There is a continuum between these polar opposites so there can easily be disagreements based on a person’s background. If I say “I believe that the earth is round” you will recognize that this is not a simple belief but a physical fact that is evidence-based. Consequently we use the word ‘belief’ in many different ways. If I say that “Aliens from outer space are firing ray guns to cause flooding in California and Australia”, it is unlikely that you will be convinced because there is no evidence of how this process could work.

If we listen to the media or read the news, you will hear many statements that I or we ‘believe’ that speed limits on streets should be reduced, or that certain types of firearms should be prohibited. The natural response of a scientist to such statements is to ask for what evidence is available that such actions will solve problems, and if there is no evidence, we deal only with opinions or beliefs. If  you lived several hundred years ago, you would be told that “malaria” was a disease caused by “bad air” coming from swamps and rivers, since there was no evidence at the time about microorganisms causing disease. So in a broad sense historical progress was made by people looking for ‘evidence’ to temper and test ‘beliefs’.

How does all this relate to ecological science? I would add the requirement to papers that state some conclusions in ecology journals to also state the beliefs the paper rely on to reach its conclusions, in addition to stating clear hypotheses and alternative hypotheses. Consider the simple case of random sampling, a basic requirement in all statistical methods. But almost no paper states what statistical population is being sampled, and if it does often the study plots are not placed randomly. The standard excuse to this is that our results apply to a large biome, and it is not physically possible to sample randomly, or that we get the same results whether we sample randomly or not. Whatever the excuse, we need to recognize this as a belief or an assumption, a less damning scientific term. And if this assumption is not accepted it is possible to sample other areas or with other methods to test if the evidence validates the assumption. Evidence can always be improved with enough funding, and this replication is exactly what many scientists are doing daily.

Until recently most scientists believed that CO2 was good for plants, and so the more CO2 the better. But the evidence provided was based on simple theory and short term lab experiments. Reich et al. (2018) and Zhu et al. (2018) pointed out that this was not correct when long-term studies were done on C3 plants like rice. So this is a good illustration of the progress of science from belief to evidence. And over the past 50 years it has become very clear that increased CO2 increases atmospheric temperature with drastic climatic and biodiversity consequences (Ripple et al. 2021). The result of these scientific advances is that now there is an extensive amount of scientific research giving the empirical evidence of climate change and CO2 effects on plants and animals. Most people agree with these broad conclusions, but there are people in large corporations and governments around the world who deny these scientific conclusions because they believe that climate change is not happening and is of little consequence to biodiversity or to daily life.

It is quite possible to ignore all the scientific literature about the consequences of climate change, CO2 increase, and biodiversity loss but the end result of passing over these problems now will fall heavily onto your children and grandchildren. The biosphere is screaming the message that ignorance will not necessarily lead to bliss.

Reich, P.B., Hobbie, S.E., Lee, T.D. & Pastore, M.A. (2018) Unexpected reversal of C3 versus C4 grass response to elevated CO2 during a 20-year field experiment. Science, 360, 317-320.doi: 10.1126/science.aas9313.

Ripple, W.J., Wolf, C., Newsome, T.M., Gregg, J.W., Lenton, T.M., Palomo, I., Eikelboom, J.A.J., Law, B.E., Huq, S., Duffy, P.B. & Rockström, J. (2021) World Scientists’ Warning of a Climate Emergency 2021. BioScience, 71, 894-898.doi: 10.1093/biosci/biab079.

Shivanna, K.R. (2022) Climate change and its impact on biodiversity and human welfare. Proceedings of the Indian National Science Academy, 88, 160-171.doi: 10.1007/s43538-022-00073-6.

Watson, R., Kundzewicz, Z.W. & Borrell-Damián, L. (2022) Covid-19, and the climate change and biodiversity emergencies. Science of The Total Environment, 844, 157188.doi: 10.1016/j.scitotenv.2022.157188.

Williams, S.E., Williams, S.E. & de la Fuente, A. (2021) Long-term changes in populations of rainforest birds in the Australia Wet Tropics bioregion: A climate-driven biodiversity emergency. PLoS ONE, 16.doi: 10.1371/journal.pone.0254307.

Zhu, C., Kobayashi, K., Loladze, I., Zhu, J. & Jiang, Q. (2018) Carbon dioxide (CO2) levels this century will alter the protein, micronutrients, and vitamin content of rice grains with potential health consequences for the poorest rice-dependent countries. Science Advances, 4, eaaq1012 doi: 10.1126/sciadv.aaq1012.

Is Ecology Becoming a Correlation Science?

2 Replies

One of the first lessons in Logic 101 is classically called “Post hoc, ergo propter hoc” or in plain English, “After that, therefore because of that”. The simplest example of many you can see in the newspapers might be: “The ocean is warming up, salmon populations are going down, it must be another effect of climate change. There is a great deal of literature on the problems associated with these kinds of simple inferences, going back to classics like Romesburg (1981), Cox and Wermuth (2004), Sugihara et al. (2012), and Nichols et al. (2019). My purpose here is only to remind you to examine cause and effect when you make ecological conclusions.

My concern is partly related to news articles on ecological problems. A recent example is the collapse of the snow crab fishery in the Gulf of Alaska which in the last 5 years has gone from a very large and profitable fishery interacting with a very large crab population to, at present, a closed fishery with very few snow crabs. What has happened? Where did the snow crabs go? No one really knows but there are perhaps half a dozen ideas put forward to explain what has happened. Meanwhile the fishery and the local economy are in chaos. Without very many critical data on this oceanic ecosystem we can list several factors that might be involved – climate change warming of the Bering Sea, predators, overfishing, diseases, habitat disturbances because of bottom trawl fishing, natural cycles, and then recognizing that we have no simple way for deciding cause and effect and therefore making management choices.

The simplest solution is to say that many interacting factors are involved and many papers indicate the complexity of populations, communities and ecosystems (e,g, Lidicker 1991, Holmes 1995, Howarth et al. 2014). Everyone would agree with this general idea, “the world is complex”, but the arguments have always been “how do we proceed to investigate ecological processes and solve ecological problems given this complexity?” The search for generality has led mostly into replications in which ‘identical’ populations or communities behave very differently. How can we resolve this problem? A simple answer to all this is to go back to the correlation coefficient and avoid complexity.

Having some idea of what is driving changes in ecological systems is certainly better than having no idea, but it is a problem when only one explanation is pushed without a careful consideration of alternative possibilities. The media and particularly the social media are encumbered with oversimplified views of the causes of ecological problems which receive wide approbation with little detailed consideration of alternative views. Perhaps we will always be exposed to these oversimplified views of complex problems but as scientists we should not follow in these footsteps without hard data.

What kind of data do we need in science? We must embrace the rules of causal inference, and a good start might be the books of Popper (1963) and Pearl and Mackenzie (2018) and for ecologists in particular the review of the use of surrogate variables in ecology by Barton et al. (2015). Ecologists are not going to win public respect for their science until they can avoid weak inference, minimize hand waving, and follow the accepted rules of causal inference. We cannot build a science on the simple hypothesis that the world is complicated or by listing multiple possible causes for changes. Correlation coefficients can be a start to unravelling complexity but only a weak one. We need better methods for resolving complex issues in ecology.

Barton, P.S., Pierson, J.C., Westgate, M.J., Lane, P.W. & Lindenmayer, D.B. (2015) Learning from clinical medicine to improve the use of surrogates in ecology. Oikos, 124, 391-398.doi: 10.1111/oik.02007.

Cox, D.R. and Wermuth, N. (2004). Causality: a statistical view. International Statistical Reviews 72: 285-305.

Holmes, J.C. (1995) Population regulation: a dynamic complex of interactions. Wildlife Research, 22, 11-19.

Howarth, L.M., Roberts, C.M., Thurstan, R.H. & Stewart, B.D. (2014) The unintended consequences of simplifying the sea: making the case for complexity. Fish and Fisheries, 15, 690-711.doi: 10.1111/faf.12041

Lidicker, W.Z., Jr. (1991) In defense of a multifactor perspective in population ecology. Journal of Mammalogy, 72, 631-635.

Nichols, J.D., Kendall, W.L. & Boomer, G.S. (2019) Accumulating evidence in ecology: Once is not enough. Ecology and Evolution, 9, 13991-14004.doi: 10.1002/ece3.5836.

Pearl, J., and Mackenzie, D. 2018. The Book of Why. The New Science of Cause and Effect. Penguin, London, U.K. 432 pp. ISBN: 978-1541698963

Popper, K.R. 1963. Conjectures and Refutations: The Growth of Scientific Knowledge. Routledge and Kegan Paul, London. 608 pp. ISBN: 978-1541698963

Romesburg, H.C. (1981) Wildlife science: gaining reliable knowledge. Journal of Wildlife Management, 45, 293-313.

Sugihara, G., et al. (2012) Detecting causality in complex ecosystems. Science, 338, 496-500.doi: 10.1126/science.1227079.

How to Destroy a Research Station

4 Replies

I have had the ‘privilege’ over the last 60 years of watching three ecological field stations be destroyed. Admittedly this is a small sample, against which every ecologist can complain, but I wanted to present to you my list of how to achieve this kind of destruction should you ever be commanded to do so. I will not name names or specific places, since the aim is to develop a general theory rather than to name and pillory specific historical actions and people. I suggest that nine rules are needed to proceed smoothly in this matter if you are given this job.

  1.  Have a clear vision why you wish to destroy an existing station. Do not vacillate. The background may be money, or philosophy of science, or orders from those higher in the echelon, or a personal peeve. Remember you are an administrator, and no one can challenge your wisdom in making major changes or closing the station.  
  2. Speak to none of the current users of the research station. If the research station has a Users Committee, avoid talking to them until after all the decisions are made. A users committee is just an honorary appointment, and it helps if very few of the users are actually people who do research at the station. It is very important that your vision should not be clouded by personnel or research programs currently running at the station. And it is best if the scientists using the station have no information except gossip about the changes that are coming.
  3. Avoid loose talk around your office. If you or your group are paying a visit in the field to the research station before closing it or repositioning its purpose, give out no information to anyone on future courses of action.
  4. Communicate upwards in the hierarchy, never downwards. You must keep all the members of the higher echelons fully informed. Do not dwell on the details of your progress in destruction but emphasize the gains that will flow from this dismantling. Tell fibs as much as you like because no one will question your version of events.
  5. Never read anything about the history of the research station or read any of the papers and reports that have originated there. The key is that you as an administrator know what should be done, and the last consideration is history. Administrators must keep a clear mind, unconcerned with historical trivia.
  6. Let none of the destruction news reach the media lest the public in general might begin to see what is happening. Newspaper and media coverage are rarely flattering to bureaucrats. If possible, line up a sympathetic media person who can talk about the brilliant future of the research station and the wisdom of the decisions you have made.
  7. Take a strong business approach. Do not worry if you must fire people currently running the research station or eject scientists currently working there. Everyone must retire at some point and all business leaders have solid recipes for hiring contractors to take care of any problems with the buildings. No matter what the extra cost.
  8. Sell the research station if you possibly can in order to gain revenue for your yet to be revealed vision. You may talk complete nonsense to explain why you are making major changes or closing the research station because few of your possible critics will be in a position to distinguish nonsense statements from truth. ‘Alternative facts’ are very useful if your decisions are questioned.
  9. Realize that if you have made a mistake in destroying a research station, your employer will not know that for several years. By that time, you will have ascended in the hierarchy of your employment unit for having carried out such a definitive action. And if your co-workers know the poor job you are doing, they will write sterling letters of reference for you to move you to another position in a different department or agency so that the worse the job you have done, the stronger will be the reference letters to recommend you for another job.

There is almost no literature I can find on this topic of administering a field station. If you think field stations are eternal, it may be a sign that you are very young, or you are very fortunate in working for an agency where moving forward is correctly labeled as progress. I have always thought that long-term field research stations were considered sacred but clearly not everyone agrees. Administrators must have something to do to leave their mark on the world for better or worse. All we can do is watch and be alert for emerging symptoms of collapse.

Swanson, F.J. (2015). Confluence of arts, humanities, and science at sites of long-term ecological inquiry. Ecosphere 6 (8), Article 132. doi: 10.1890/ES15-00139.1.

On Assumptions in Ecology Papers

1 Reply

What can we do as ecologists to improve the publishing standards of ecology papers? I suggest one simple but bold request. We should require at the end of every published paper a annotated list of the assumptions made in providing the analysis reported in the paper. A tabular format could be devised with columns for the assumption, the perceived support of and tests for the assumption, and references for this support or lack thereof. I can hear the screaming already, so this table could be put in the Supplementary Material which most people do not read. We could add to each paper in the final material where there are statements of who did the writing, who provided the money, and add a reference to this assumptions table in the Supplementary Material or a statement that no assumptions about anything were made to reach these conclusions.

The first response I can detect to this recommendation is that many ecologists will differ in what they state are assumptions to their analysis and conclusions. As an example, in wildlife studies, we commonly make the assumption that an individual animal having a radio collar will behave and survive just like another animal with no collar. In analyses of avian population dynamics, we might commonly assume that our visiting nests does not affect their survival probability. We make many such assumptions about random or non-random sampling. My question then is whether or not there is any value in listing these kinds of assumptions. My response is that this approach of listing what the authors think they are assuming should alert the reviewers to the elephants in the room that have not been listed.

My attention was called to this general issue by the recent paper of Ginzburg and Damuth (2022) in which they contrasted the assumptions of two general theories of functional responses of predators to prey – “prey dependence” versus “ratio dependence”. We have in ecology many such either-or discussions that never seem to end. Consider the long-standing discussion of whether populations can be regulated by factors that are “density dependent” or “density independent”, a much-debated issue that is still with us even though it was incisively analyzed many years ago.  

Experimental ecology is not exempt from assumptions, as outlined in Kimmel et al. (2021) who provide an incisive review of cause and effect in ecological experiments. Pringle and Hutchinson (2020) discuss the failure of assumptions in food web analysis and how these might be resolved with new techniques of analysis. Drake et al. (2021) consider the role of connectivity in arriving at conservation evaluations of patch dynamics, and the importance of demographic contributions to connectivity via dispersal. The key point is that, as ecology progresses, the role of assumptions must be continually questioned in relation to our conclusions about population and community dynamics in relation to conservation and landscape management.

Long ago Peters (1991) wrote an extended critique of how ecology should operate to avoid some of these issues, but his 1991 book is not easily available to students (currently available on Amazon for about $90). To encourage more discussion of these questions from the older to the more current literature, I have copied Peters Chapter 4 to the bottom of my web page at https://www.zoology.ubc.ca/~krebs/books.html for students to download if they wish to discuss these issues in more detail.

Perhaps a possible message in all this has been that ecology has always wished to be “physics-in-miniature” with grand generalizations like the laws we teach in the physical sciences. Over the last 60 years the battle in the ecology literature has been between this model of physics and the view that every population and community differ, and everything is continuing to change under the climate emergency so that we can have little general theory in ecology. There are certainly many current generalizations, but they are relatively useless for a transition from the general to the particular for the development of a predictive science. The consequence is that we now bounce from individual study to individual study, typically starting from different assumptions, with very limited predictability that is empirically testable. And the central issue for ecological science is how can we move from the present fragmentation in our knowledge to a more unified science. Perhaps starting to examine the assumptions of our current publications would be a start in this direction.  

Drake, J., Lambin, X., and Sutherland, C. (2021). The value of considering demographic contributions to connectivity: a review. Ecography 44, 1-18. doi: 10.1111/ecog.05552.

Ginzburg, L.R. and Damuth, J. (2022). The Issue Isn’t Which Model of Consumer Interference Is Right, but Which One Is Least Wrong. Frontiers in Ecology and Evolution 10, 860542. doi: 10.3389/fevo.2022.860542.

Kimmel, K., Dee, L.E., Avolio, M.L., and Ferraro, P.J. (2021). Causal assumptions and causal inference in ecological experiments. Trends in Ecology & Evolution 36, 1141-1152. doi: 10.1016/j.tree.2021.08.008.

Peters, R.H. (1991) ‘A Critique for Ecology.’ (Cambridge University Press: Cambridge, England.) ISBN:0521400171 (Chapter 4 pdf available at https://www.zoology.ubc.ca/~krebs/books.html)

Pringle, R.M. and Hutchinson, M.C. (2020). Resolving Food-Web Structure. Annual Review of Ecology, Evolution, and Systematics 51, 55-80. doi: 10.1146/annurev-ecolsys-110218-024908.