How Much Evidence is Enough?

The scientific community in general considers a conclusion about a problem resolved if there is enough evidence. There are many excellent books and papers that discuss what “enough evidence” means in terms of sampling design, experimental design, and statistical methods (Platt 1964, Shadish et al. 2002, Johnson 2002, and many others) so I will skip over these technical issues and discuss the nature of evidence we typically see in ecology and management.

An overall judgement one can make is that there is a great diversity among the different sciences about how much evidence is enough. If replication is expensive, typically fewer experiments are deemed sufficient. If human health is involved, as we see with Covid-19, many controlled experiments with massive replication is usually required. For fisheries and wildlife management much less evidence is typically quoted as sufficient. For much of conservation biology the problem arises that no experimental design can be considered if the species or taxa are threatened or endangered. In these cases we have to rely on a general background of accepted principles to guide our management actions. It is these cases that I want to focus on here.

Two guiding lights in the absence of convincing experiments are the Precautionary Principle and the Hippocratic Oath. The simple prescription of the Hippocratic Oath for medical doctors has always been “Do no harm”. The Precautionary Principle has been spread more widely and has various interpretations, most simply “Look before you leap” (Akins et al. 2019). But if applied too strictly some would argue, this principle might stop “green” projects that are in themselves directed toward sustainability. Wind turbine tower effects on birds are one example (Coppes et al. 2020). The conservation of wild bees may impact current agricultural production positively (Drossart and Gerard 2020) or negatively depending on the details of the conservation practices. Trade offs are a killer for many conservation solutions, jobs vs. the environment.

Many decisions about conservation action and wildlife management rest on less than solid empirical evidence. This observation could be tested in any graduate seminar by dissecting a series of papers on explicit conservation problems. Typically, those cases involving declining large bodied species like caribou or northern spotted owls or tigers are affected by a host of interconnected problems involving human usurpation of habitats for forestry, agriculture, or cities, backed up by poaching or direct climate change due to air pollution, or diseases introduced by domestic animals or introduced species. In some fraction of cases the primary cause of decline is well documented but cannot be changed by conservation biologists (e.g. CO2 and coral bleaching). 

Nichols et al. (2019) recommend a model-based approach to answering conservation and management questions as a way to increase the rate of learning about which set of hypotheses best predict ecological changes. The only problem with their approach is the time scale of learning, which for immediate conservation issues may be limiting. But for problems that have a longer time scale for hypothesis testing and decision making they have laid out an important pathway to problem solutions.

In many ecological and conservation publications we are allowed to suggest weak hypotheses for the explanation of pest outbreaks or population declines, and in the worst cases rely on “correlation = causation” arguments. This will not be a problem if we explicitly recognize weak hypotheses and specify a clear path to more rigorous hypotheses and experimental tests. Climate change is the current panchrestron or universal explanation because it shows weak associations with many ecological changes. There is no problem with invoking climate change as an explanatory variable if there are clear biological mechanisms linking this cause to population or community changes.

All of this has been said many times in the conservation and wildlife management literature, but I think needs continual reinforcement. Ask yourself: Is this evidence strong enough to support this conclusion? Weak conclusions are perhaps useful at the start of an investigation but are not a good basis for conservation or wildlife management decision making. Ensuring that our scientific conclusions “Do no harm” is a good principle for ecology as well as medicine.

Akins, A., et al. (2019). The Precautionary Principle in the international arena. Sustainability 11 (8), 2357. doi: 10.3390/su11082357.

Coppes, J., et al. (2020). The impact of wind energy facilities on grouse: a systematic review. Journal of Ornithology 161, 1-15. doi: 10.1007/s10336-019-01696-1.

Drossart, M. and Gerard, M. (2020). Beyond the decline of wild bees: Optimizing conservation measures and bringing together the actors. Insects (Basel, Switzerland) 11, 649. doi: 10.3390/insects11090649.

Johnson, D.H. (2002). The importance of replication in wildlife research. Journal of Wildlife Management 66, 919-932.

Nichols, J.D., Kendall, W.L., and Boomer, G.S. (2019). Accumulating evidence in ecology: Once is not enough. Ecology and Evolution 9, 13991-14004. doi: 10.1002/ece3.5836.

Platt, J. R. (1964). Strong inference. Science 146, 347-353. doi: 10.1126/science.146.3642.347.

Shadish, W.R, Cook, T.D., and Campbell, D.T. (2002) ‘Experimental and Quasi-Experimental Designs for Generalized Causal Inference.‘ (Houghton Mifflin Company: New York.)