Category Archives: Experimental Design in Ecology

On the Meaning of ‘Food Limitation’ in Population Ecology

There are many different ecological constraints that are collected in the literature under the umbrella of ‘food limitation’ when ecologists try to explain the causes of population changes or conservation problems. ‘Sockeye salmon in British Columbia are declining in abundance because of food limitation in the ocean’. ’Jackrabbits in some states in the western US are increasing because climate change has increased plant growth and thus removed the limitation of their plant food supplies.’ ‘Moose numbers in western Canada are declining because their food plants have shifted their chemistry to cope with the changing climate and now suffer food limitation”. My suggestion here is that ecologists should be careful in defining the meaning of ‘limitation’ in discussing these kinds of population changes in both rare and abundant species.

Perhaps the first principle is that it is the definition of life that food is always limiting. One does not need to do an experiment to demonstrate this truism. So to start we must agree that modern agriculture is built on the foundation that food can be improved and that this form of ‘food limitation’ is not what ecologists who are interested in population changes in the real world are trying to test. The key to explain population differences must come from resource differences in the broad sense, not food alone but a host of other ecological causal factors that may produce changes in birth and death rates in populations.

‘Limitation’ can be used in a spatial or a temporal context. Population density of deer mice can differ in average density in 2 different forest types, and this spatial problem would have to be investigated as a search for the several possible mechanisms that could be behind this observation. Often this is passed off too easily by saying that “resources” are limiting in the poorer habitat, but this statement takes us no closer to understanding what the exact food resources are. If food resources carefully defined are limiting density in the ‘poorer’ habitat, this would be a good example of food limitation in a spatial sense. By contrast if a single population is increasing in one year and declining in the next year, this could be an example of food limitation in a temporal sense.

The more difficult issue now becomes what evidence you have that food is limiting in either time or space. Growth in body size in vertebrates is one clear indirect indicator but we need to know exactly what food resources are limiting. The temptation is to use feeding experiments to test for food limitation (reviewed in Boutin 1990). Feeding experiments in the lab are simple, in the field not simple. Feeding an open population can lead to immigration and if your response variable is population density, you have an indirect effect of feeding. If animals in the experimentally fed area grow faster or have a higher reproductive output, you have evidence of the positive effect of the feeding treatment. You can then claim ‘food limitation’ for these specific variables. If population density increases on your feeding area relative to unfed controls, you can also claim ‘food limitation of density’. The problems then come when you consider the temporal dimension due to seasonal or annual effects. If the population density falls and you are still feeding in season 2 or year 2, then food limitation of density is absent, and the change must have been produced by higher mortality in season 2 or higher emigration.

Food resources could be limiting because of predator avoidance (Brown and Kotler 2007). The ecology of fear from predation has blossomed into a very large literature that explores the non-consumptive effects of predators on prey foraging that can lead to food limitation without food resources being in short supply (e.g., Peers et al. 2018, Allen et al. 2022).

All of this seems to be terribly obvious but the key point is that if you examine the literature about “food limitation” look at the evidence and the experimental design. Ecologists like medical doctors at times have a long list of explanations designed to sooth the soul without providing good evidence of what exact mechanism is operating. Economists are near the top with this distinguished approach, exceeded only by politicians, who have an even greater art in explaining changes after the fact with limited evidence.

As a footnote to defining this problem of food limitation, you should read Boutin (1990). I have also raved on about this topic in Chapter 8 of my 2013 book on rodent populations if you wish more details.

Allen, M.C., Clinchy, M. & Zanette, L.Y. (2022) Fear of predators in free-living wildlife reduces population growth over generations. Proceedings of the National Academy of Sciences (PNAS), 119, e2112404119. doi: 10.1073/pnas.2112404119.

Boutin, S. (1990). Food supplementation experiments with terrestrial vertebrates: patterns, problems, and the future. Canadian Journal of Zoology 68(2): 203-220. doi: 10.1139/z90-031.

Brown, J.S. & Kotler, B.P. (2007) Foraging and the ecology of fear. Foraging: Behaviour and Ecology (eds. D.W. Stephens, J.S. Brown & R.C. Ydenberg), pp. 437-448.University of Chicago Press, Chicago. ISBN: 9780226772646

Krebs, C.J. (2013) Chapter 8, The Food Hypothesis. In Population Fluctuations in Rodents. University of Chicago Press, Chicago. ISBN: 978-0-226-01035-9

Why Ecological Understanding Progresses Slowly

I begin with a personal observation spanning 65 years of evaluating ecological and evolutionary science – we are making progress but very slowly. This problem would be solved very simply in the Middle Ages by declaring this statement a heresy, followed by a quick burning at the stake. But for the most part we are more civil now, and we allow old folks to rant and rave without listening much.

By a stroke of luck, Betts et al. (2021) have reached the same conclusion, but in a more polite and nuanced way than I. So, for the whole story please read their paper, to which I will only add a footnote of a tirade to make it more personal. The question is simple and stark: Should all ecological research be required to follow the hypothetico-deductive framework of science? Many excellent ecologists have argued against this proposal, and I will offer only an empirical, inductive set of observations to make the contrary view in support of H-D science.  

Ecological and evolutionary papers can be broadly categorized as (1) descriptive natural history, (2) experimental hypothesis tests, and (3) future projections. The vast bulk of papers falls into the first category, a description of the world as it is today and in the past. The h-word never appears in these publications. These papers are most useful in discovering new species, new interactions between species, and the valuable information about the world of the past through paleoecology and the geological sciences. Newspapers and TV thrive on these kinds of papers and alert the public to the natural world in many excellent ways. Descriptive natural history in the broad sense fully deserves our support, and it provides information essential to category (2), experimental ecology, by asking questions about emerging problems, introduced pests, declining fisheries, endangered mammals and all the changing components of our natural world. Descriptive papers typically provide ideas that need follow up by experimental studies. 

Public support for science comes from the belief that scientists solve problems, and if the major effort of ecologists and evolutionary biologists is to describe nature, it is not surprising that financial support is minimal in these areas of study. The public is entertained but ecological problems are not solved. So, I argue we need more of papers (2). But we can get these only if we attack serious problems with experimental means, and this requires long-term thinking and long-term funding on a scale we rarely see in ecology. The movement at present is in the direction of big-data, technological methods of gathering data remotely to investigate landscape scale problems. If big data is considered only observational, we remain in category (1) and there is a critical need to make sure that big data projects are truly experimental, category (2) science (Lindenmayer, Likens and Franklin 2018). That this change is not happening so far is clear in Betts et al. (2021) Figure 2, which shows that very few papers in ecology journals in the last 25 years provide a clear set of multiple alternative hypotheses that they are attempting to test. If this criterion is a definition of good science, there is far less being done than we might think from the explosion of papers in ecology and evolution.

The third category of ecological and evolution papers is focused on future predictions with a view to climate change. In my opinion most of these papers should be confined to a science fiction journal because they are untestable model extrapolations for a future beyond our lifetimes. A limited subset of these could be useful is they were projecting a 5-10 year scenario that scientists could possibly test in the short term. If they are to be printed, I would suggest an appendix in all these papers of the list of assumptions that must be made to reach their future predictions.

There is of course the fly in the ointment that even when ecologists diagnose a conservation problem with good experiments and analysis the policy makers will not follow their advice (e.g. Palm et al. 2020). The world is not yet perfect.

Betts, M.G., Hadley, A.S., Frey, D.W., Frey, S.J.K., Gannon, D., et al. (2021). When are hypotheses useful in ecology and evolution? Ecology and Evolution. doi: 10.1002/ece3.7365.

Lindenmayer, D.B., Likens, G.E., and Franklin, J.F. (2018). Earth Observation Networks (EONs): Finding the Right Balance. Trends in Ecology & Evolution 33, 1-3. doi: 10.1016/j.tree.2017.10.008.

Palm, E. C., Fluker, S., Nesbitt, H.K., Jacob, A.L., and Hebblewhite, M. (2020). The long road to protecting critical habitat for species at risk: The case of southern mountain woodland caribou. Conservation Science and Practice 2: e219. doi: 10.1111/csp2.219.

On an Experimental Design Mafia for Ecology

Ecologist A does an experiment and publishes Conclusions G and H. Ecologist B reads this paper and concludes that A’s data support Conclusions M and N and do not support Conclusions G and H. Ecologist B writes to Journal X editor to complain and is told to go get stuffed because Journal X never makes a mistake with so many members of the Editorial Board who have Nobel Prizes. This is an inviting fantasy and I want to examine one possible way to avoid at least some of these confrontations without having to fire all the Nobel Prize winners on the Editorial Board.

We go back to the simple question: Can we agree on what types of data are needed for testing this hypothesis? We now require our graduate students or at least our Nobel colleagues to submit the experimental design for their study to the newly founded Experimental Design Mafia for Ecology (or in French DEME) who will provide a critique of the formulation of the hypotheses to be tested and the actual data that will be collected. The recommendations of the DEME will be nonbinding, and professors and research supervisors will be able to ignore them with no consequences except that the coveted DEME icon will not be able to be published on the front page of the resulting papers.

The easiest part of this review will be the data methods, and this review by the DEME committee will cover the current standards for measuring temperature, doing aerial surveys for elephants, live-trapping small mammals, measuring DBH on trees, determining quadrat size for plant surveys, and other necessary data collection problems. This advice alone should hypothetically remove about 25% of future published papers that use obsolete models or inadequate methods to measure or count ecological items.

The critical part of the review will be the experimental design part of the proposed study. Experimental design is important even if it is designated as undemocratic poppycock by your research committee. First, the DEME committee will require a clear statement of the hypothesis to be tested and the alternative hypotheses. Words which are used too loosely in many ecological works must be defended as having a clear operational meaning, so that idea statements that include ‘stability’ or ‘ecosystem integrity’ may be questioned and their meaning sharpened. Hypotheses that forbid something from occurring or allow only type Y events to occur are to be preferred, and for guidance applicants may be referred to Popper (1963), Platt (1964), Anderson (2008) or Krebs (2019). If there is no alternative hypothesis, your research plan is finished. If you are using statistical methods to test your hypotheses, read Ioannidis (2019).

Once you have done all this, you are ready to go to work. Do not be concerned if your research plan goes off target or you get strange results. Be prepared to give up hypotheses that do not fit the observed facts. That means you are doing creative science.

The DEME committee will have to be refreshed every 5 years or so such that fresh ideas can be recognized. But the principles of doing good science are unlikely to change – good operational definitions, a set of hypotheses with clear predictions, a writing style that does not try to cover up contrary findings, and a forward look to what next? And the ecological world will slowly become a better place with fewer sterile arguments about angels on the head of a pin.

Anderson, D.R. (2008) ‘Model Based Inference in the Life Sciences: A Primer on Evidence.‘ (Springer: New York.) ISBN: 978-0-387-74073-7.

Ioannidis, J.P.A. (2019). What have we (not) learnt from millions of scientific papers with P values? American Statistician 73, 20-25. doi: 10.1080/00031305.2018.1447512.

Krebs, C.J. (2020). How to ask meaningful ecological questions. In Population Ecology in Practice. (Eds D.L. Murray and B.K. Sandercock.) Chapter 1, pp. 3-16. Wiley-Blackwell: Amsterdam. ISBN: 978-0-470-67414-7

Platt, J. R. (1964). Strong inference. Science 146, 347-353. doi: 10.1126/science.146.3642.347.

Popper, K. R. (1963) ‘Conjectures and Refutations: The Growth of Scientific Knowledge.’ (Routledge and Kegan Paul: London.). ISBN: 9780415285940

On the Use of Statistics in Ecological Research

There is an ever-deepening cascade of statistical methods and if you are going to be up to date you will have to use and cite some of them in your research reports or thesis. But before you jump into these methods, you might consider a few tidbits of advice. I suggest three rules and a few simple guidelines:

Rule 1. For descriptive papers keep to descriptive statistics. Every good basic statistics book has advice on when to use means to describe “average values”, when to use medians, or percentiles. Follow their advice and do not in your report generate any hypotheses except in the discussion. And follow the simple advice of statisticians not to generate and then test a hypothesis with the same set of data. Descriptive papers are most valuable. They can lead us to speculations and suggest hypotheses and explanations, but they do not lead us to strong inference.

Rule 2. For explanatory papers, the statistical rules become more complicated. For scientific explanation you need 2 or more alternative hypotheses that make different, non-overlapping predictions. The predictions must involve biological or physical mechanisms. Correlations alone are not mechanisms. They may help to lead you to a mechanism, but the key is that the mechanism must involve a cause and an effect. A correlation of a decline in whale numbers with a decline in sunspot numbers may be interesting but only if you can tie this correlation into an actual mechanism that affects birth or death rates of the whales.

Rule 3. For experimental papers you have access to a large variety of books and papers on experimental design. You must have a control or unmanipulated group, or for a comparative experiment a group A with treatment X, and a group B with treatment Y. There are many rules in the writings of experimental design that give good guidance (e.g. Anderson 2008; Eberhardt 2003; Johnson 2002; Shadish et al. 2002; Underwood 1990).

For all these ecology papers, consider the best of the recent statistical admonitions. Use statistics to enlighten not to obfuscate the reader. Use graphics to illustrate major results. Avoid p-values (Anderson et al. 2000; Ioannidis 2019a, 2019b). Measure effect sizes for different treatments (Nakagawa and Cuthill 2007). Add to these general admonitions the conventional rules of paper or report submission – do not argue with the editor, argue a small amount with the reviewers (none are perfect), and put your main messages in the abstract. And remember that it is possible there was some interesting research done before the year 2000.

Anderson, D.R. (2008) ‘Model Based Inference in the Life Sciences: A Primer on Evidence.’ (Springer: New York.). 184 pp.

Anderson, D.R., Burnham, K.P., and Thompson, W.L. (2000). Null hypothesis testing: problems, prevalence, and an alternative. Journal of Wildlife Management 64, 912-923.

Eberhardt, L.L. (2003). What should we do about hypothesis testing? Journal of Wildlife Management 67, 241-247.

Ioannidis, J.P.A. (2019a). Options for publishing research without any P-values. European Heart Journal 40, 2555-2556. doi: 10.1093/eurheartj/ehz556.

Ioannidis, J. P. A. (2019b). What have we (not) learnt from millions of scientific papers with P values? American Statistician 73, 20-25. doi: 10.1080/00031305.2018.1447512.

Johnson, D.H. (2002). The importance of replication in wildlife research. Journal of Wildlife Management 66, 919-932.

Nakagawa, S. and Cuthill, I.C. (2007). Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological Reviews 82, 591-605. doi: 10.1111/j.1469-185X.2007.00027.x.

Shadish, W.R, Cook, T.D., and Campbell, D.T. (2002) ‘Experimental and Quasi-Experimental Designs for Generalized Causal Inference.’ (Houghton Mifflin Company: New York.)

Underwood, A. J. (1990). Experiments in ecology and management: Their logics, functions and interpretations. Australian Journal of Ecology 15, 365-389.

On Random Sampling and Generalization in Ecology

Virtually every introduction to statistics book makes the point that random sampling is a critical assumption that underlies all statistical inferences. It is assumption #1 of statistical inference and it carries with it an often-hidden assumption that in trying to make your inference, you have clearly defined what the statistical population is that you are sampling. Defining the ‘population’ under consideration should perhaps be rule # 1, but that is usually left as a vague understanding in many statistical studies. As an exercise consult a series of papers on ecological field studies and see if you can find a clear statement of what the ‘population’ under consideration is. An excellent example of this kind of analysis is given by Ioannidis (2003, 2005).

The problem of random sampling does not occur in theoretical statistics and all effort is concentrated on mathematical correctness. This is illustrated well in the polls we are subjected to on political or social issues, and in the medical studies that we hear about daily. The social sciences have considered sampling for polls in much more detail that have biologists. In a historical overview (Lusinchi 2017) provides an interesting and useful analysis of how pollsters have over the years bent the science of statistical inference to their methods of polling to provide an unending flow of conclusions about who will be elected, or which coffee is better tasting. By confounding sample size with an approach to Truth and ignoring the problem of random sampling, the public has been brainwashed to believe what should be properly labeled as ‘fake news’.

What has all of this got to do with the science of ecology? Much of the data we accumulate is uncertain when we ask what is the ‘population’ to which it applies. If you are concerned about the ecology of sharks, you face the problem that most species of shark have never been studied (Ducatez 2019). If you are interested in fish populations, for example, you may find that the fish you catch with hooks are not a random sample of the fish population (Lennox et al. 2017). If you are studying the trees in a large woodlot, that may be your universe for statistical purposes. Interest then shifts to the question of how much you will generalize to other woodlots over what geographical space, a question too rarely discussed in data papers. In an ideal world we would sample several woodlots randomly selected from a larger sample of similar woodlots, so that we could infer processes that were common to woodlots in general.

There are a couple of problems that confound ecologists at this point. No series of woodlots or study sites in general are identical, so we assume they are a collective of ‘very similar’ woodlots about which we could make an inference. Alternatively, we can simply state that we wish to make inferences about only this single woodlot, it is our total population. At this point your supervisor/boss will say that he or she is not interested only in this one woodlot but much more general conclusions, so you will be cut from research funding for having too narrow an interest.

The solution is in general to study one ‘woodlot’ and then generalize to all ‘woodlots’ with no further study on your part, so that it will be up to the next generation to find out if your generalization is right or wrong. While this way of proceeding will perhaps not matter to people interested in ‘woodlots’, it might well matter greatly if your ‘population of interest’ was composed of humans considering a drug for disease treatment. We are further confounded in this era of climate change in dealing with changing ecosystems, so that a study in 2000 about coral reef fish communities could be completely different if it were repeated in 2040 as oceans warm.

Back to random sampling. I would propose that random sampling in ecological systems is impossible and cannot be achieved in a global sense. Be concerned about local processes and sample accordingly. Descriptive ecology must come to the rescue here, so that we know as background information (for example) that trees grow slower as they age, that tree growth varies from year to year, that insect attacks vary with summer temperature, and so on, and sample accordingly following your favourite statistician. There are many very useful statistical techniques and sampling designs you can use as an ecologist to achieve random sampling on a local scale, and statisticians are most useful to consult to validate the design of your field studies.

But it is important to remember that your results and conclusions even though carried out with a perfect statistical design cannot ensure that your generalizations are correct in time or in space. The use of meta-analysis can assist in validating generalizations when enough replicated studies are available, but there are problems even with this approach (Siontis and Ioannidis 2018). Continued discussion of p-values in ecology could benefit much from similar discussions in medicine where funding is higher, and replication is more common (Ioannidis 2019b; Ioannidis 2019a).

All these statistical issues provide a strong argument as to why ecological field studies and experiments should never stop, and all our studies and conclusions are temporary signposts along a path that is never ending.

Ducatez, S. (2019). Which sharks attract research? Analyses of the distribution of research effort in sharks reveal significant non-random knowledge biases. Reviews in Fish Biology and Fisheries 29, 355-367. doi: 10.1007/s11160-019-09556-0.

Ioannidis, J.P.A. (2005). Contradicted and initially stronger effects in highly cited clinical research. Journal of the American Medical Association 294, 218-228. doi: 10.1001/jama.294.2.218.

Ioannidis, J.P.A. (2005). Why most published research findings are false. PLOS Medicine 2, e124. doi: 10.1371/journal.pmed.0020124.

Ioannidis, J.P.A. (2019a). What have we (not) learnt from millions of scientific papers with p values? American Statistician 73, 20-25. doi: 10.1080/00031305.2018.1447512.

Ioannidis, J.P.A. (2019b). The importance of predefined rules and prespecified statistical analyses: do not abandon significance. Journal of the American Medical Association 321, 2067-2068. doi: 10.1001/jama.2019.4582.

Lennox, R.J., et al. (2017). What makes fish vulnerable to capture by hooks? A conceptual framework and a review of key determinants. Fish and Fisheries 18, 986-1010. doi: 10.1111/faf.12219.

Lusinchi, D. (2017). The rhetorical use of random sampling: crafting and communicating the public image of polls as a science (1935-1948). Journal of the History of the Behavioral Sciences 53, 113-132. doi: 10.1002/jhbs.21836.

Siontis, K.C. and Ioannidis, J.P.A. (2018). Replication, duplication, and waste in a quarter million systematic reviews and meta-analyses. Circulation Cardiovascular Quality and Outcomes 11, e005212. doi: 10.1161/circoutcomes.118.005212.

On Caribou and Hypothesis Testing

Mountain caribou populations in western Canada have been declining for the past 10-20 years and concern has mounted to the point where extinction of many populations could be imminent, and the Canadian federal government is asking why this has occurred. This conservation issue has supported a host of field studies to determine what the threatening processes are and what we can do about them. A recent excellent summary of experimental studies in British Columbia (Serrouya et al. 2017) has stimulated me to examine this caribou crisis as an illustration of the art of hypothesis testing in field ecology. We teach all our students to specify hypotheses and alternative hypotheses as the first step to solving problems in population ecology, so here is a good example to start with.

From the abstract of this paper, here is a statement of the problem and the major hypothesis:

“The expansion of moose into southern British Columbia caused the decline and extirpation of woodland caribou due to their shared predators, a process commonly referred to as apparent competition. Using an adaptive management experiment, we tested the hypothesis that reducing moose to historic levels would reduce apparent competition and therefore recover caribou populations. “

So the first observation we might make is that much is left out of this approach to the problem. Populations can decline because of habitat loss, food shortage, excessive hunting, predation, parasitism, disease, severe weather, or inbreeding depression. In this case much background research has narrowed the field to focus on predation as a major limitation, so we can begin our search by focusing on the predation factor (review in Boutin and Merrill 2016). In particular Serrouya et al. (2017) focused their studies on the nexus of moose, wolves, and caribou and the supposition that wolves feed preferentially on moose and only secondarily on caribou, so that if moose numbers are lower, wolf numbers will be lower and incidental kills of caribou will be reduced. So they proposed two very specific hypotheses – that wolves are limited by moose abundance, and that caribou are limited by wolf predation. The experiment proposed and carried out was relatively simple in concept: kill moose by allowing more hunting in certain areas and measure the changes in wolf numbers and caribou numbers.

The experimental area contained 3 small herds of caribou (50 to 150) and the unmanipulated area contained 2 herds (20 and 120 animals) when the study began in 2003. The extended hunting worked well, and moose in the experimental area were reduced from about 1600 animals down to about 500 over the period from 2003 to 2014. Wolf numbers in the experimental area declined by about half over the experimental period because of dispersal out of the area and some starvation within the area. So the two necessary conditions of the experiment were satisfied – moose numbers declined by about two-thirds from additional hunting and wolf numbers declined by about half on the experimental area. But the caribou population on the experimental area showed mixed results with one population showing a slight increase in numbers but the other two showing a slight loss. On the unmanipulated area both caribou populations showed a continuing slow decline. On the positive side the survival rate of adult caribou was higher on the experimental area, suggesting that the treatment hypothesis was correct.

From the viewpoint of caribou conservation, the experiment failed to change the caribou population from continuous slow declines to the rapid increase needed to recover these populations to their former greater abundance. At best it could be argued that this particular experiment slowed the rate of caribou decline. Why might this be? We can make a list of possibilities:

  1. Moose numbers on the experimental area were not reduced enough (to 300 instead of to 500 achieved). Lower moose would have meant much lower wolf numbers.
  2. Small caribou populations are nearly impossible to recover because of chance events that affect small numbers. A few wolves or bears or cougars could be making all the difference to populations numbering 10-20 individuals.
  3. The experimental area and the unmanipulated area were not assigned treatments at random. This would mean to a pure statistician that you cannot make statistical comparisons between these two areas.
  4. The general hypothesis being tested is wrong, and predation by wolves is not the major limiting factor to mountain caribou populations. Many factors are involved in caribou declines and we cannot determine what they are because they change for area to area, year to year.
  5. It is impossible to do these landscape experiments because for large landscapes it is impossible to find 2 or more areas that can be considered replicates.
  6. The experimental manipulation was not carried out long enough. Ten years of manipulation is not long for caribou who have a generation time of 15-25 years.

Let us evaluate these 6 points.

#1 is fair enough, hard to achieve a population of moose this low but possible in a second experiment.

#2 is a worry because it is difficult to deal experimentally with small populations, but we have to take the populations as a given at the time we do a manipulation.

#3 is true if you are a purist but is silly in the real world where treatments can never be assigned at random in landscape experiments.

#4 is a concern and it would be nice to include bears and other predators in the studies but there is a limit to people and money. Almost all previous studies in mountain caribou declines have pointed the finger at wolves so it is only reasonable to start with this idea. The multiple factor idea is hopeless to investigate or indeed even to study without infinite time and resources.

#5 is like #3 and it is an impossible constraint on field studies. It is a common statistical fallacy to assume that replicates must be identical in every conceivable way. If this were true, no one could do any science, lab or field.

#6 is correct but was impossible in this case because the management agencies forced this study to end in 2014 so that they could conduct another different experiment. There is always a problem deciding how long a study is sufficient, and the universal problem is that the scientists or (more likely) the money and the landscape managers run out of energy if the time exceeds about 10 years or more. The result is that one must qualify the conclusions to state that this is what happened in the 10 years available for study.

This study involved a heroic amount of field work over 10 years, and is a landmark in showing what needs to be done and the scale involved. It is a far cry from sitting at a computer designing the perfect field experiment on a theoretical landscape to actually carrying out the field work to get the data summarized in this paper. The next step is to continue to monitor some of these small caribou populations, the wolves and moose to determine how this food chain continues to adjust to changes in prey levels. The next experiment needed is not yet clear, and the eternal problem is to find the high levels of funding needed to study both predators and prey in any ecosystem in the detail needed to understand why prey numbers change. Perhaps a study of all the major predators – wolves, bears, cougars – in this system should be next. We now have the radio telemetry advances that allow satellite locations, activity levels, timing of mortality, proximity sensors when predators are near their prey, and even video and sound recording so that more details of predation events can be recorded. But all this costs money that is not yet here because governments and people have other priorities and value the natural world rather less than we ecologists would prefer. There is not yet a Nobel Prize for ecological field research, and yet here is a study on an iconic Canadian species that would be high up in the running.

What would I add to this paper? My curiosity would be satisfied by the number of person-years and the budget needed to collect and analyze these results. These statistics should be on every scientific paper. And perhaps a discussion of what to do next. In much of ecology these kinds of discussions are done informally over coffee and students who want to know how science works would benefit from listening to how these informal discussions evolve. Ecology is far from simple. Physics and chemistry are simple, genetics is simple, and ecology is really a difficult science.

Boutin, S. and Merrill, E. 2016. A review of population-based management of Southern Mountain caribou in BC. {Unpublished review available at: http://cmiae.org/wp-content/uploads/Mountain-Caribou-review-final.pdf

Serrouya, R., McLellan, B.N., van Oort, H., Mowat, G., and Boutin, S. 2017. Experimental moose reduction lowers wolf density and stops decline of endangered caribou. PeerJ  5: e3736. doi: 10.7717/peerj.3736.

 

On Defining a Statistical Population

The more I do “field ecology” the more I wonder about our standard statistical advice to young ecologists to “random sample your statistical population”. Go to the literature and look for papers on “random environmental fluctuations”, or “non-random processes”, or “random mating” and you will be overwhelmed with references and biology’s preoccupation with randomness. Perhaps we should start with the opposite paradigm, that nothing in the biological world is random in space or time, and then the corollary that if your data show a random pattern or random mating or whatever random, it means you have not done enough research and your inferences are weak.

Since virtually all modern statistical inference rests on a foundation of random sampling, every statistician will be outraged by any concerns that random sampling is possible only in situations that are scientifically uninteresting. It is nearly impossible to find an ecological paper about anything in the real world that even mentions what their statistical “population” is, what they are trying to draw inferences about. And there is a very good reason for this – it is quite impossible to define any statistical population except for those of trivial interest. Suppose we wish to measure the heights of the male 12-year-olds that go to school in Minneapolis in 2017. You can certainly do this, and select a random sample, as all statisticians would recommend. And if you continued to do this for 50 years, you would have a lot of data but no understanding of any growth changes in 12-year-old male humans because the children of 2067 in Minneapolis would be different in many ways from those of today. And so, it is like the daily report of the stock market, lots of numbers with no understanding of processes.

Despite all these ‘philosophical’ issues, ecologists carry on and try to get around this by sampling a small area that is considered homogeneous (to the human eye at least) and then arm waving that their conclusions will apply across the world for similar small areas of some ill-defined habitat (Krebs 2010). Climate change may of course disrupt our conclusions, but perhaps this is all we can do.

Alternatively, we can retreat to the minimalist position and argue that we are drawing no general conclusions but only describing the state of this small piece of real estate in 2017. But alas this is not what science is supposed to be about. We are supposed to reach general conclusions and even general laws with some predictive power. Should biologists just give up pretending they are scientists? That would not be good for our image, but on the other hand to say that the laws of ecology have changed because the climate is changing is not comforting to our political masters. Imagine the outcry if the laws of physics changed over time, so that for example in 25years it might be that CO2 is not a greenhouse gas. Impossible.

These considerations should make ecologists and other biologists very humble, but in fact this cannot be because the media would not approve and money for research would never flow into biology. Humility is a lost virtue in many western cultures, and particularly in ecology we leap from bandwagon to bandwagon to avoid the judgement that our research is limited in application to undefined statistical populations.

One solution to the dilemma of the impossibility of random sampling is just to ignore this requirement, and this approach seems to be the most common solution implicit in ecology papers. Rabe et al. (2002) surveyed the methods used by management agencies to survey population of large mammals and found that even when it was possible to use randomized counts on survey areas, most states used non-random sampling which leads to possible bias in estimates even in aerial surveys. They pointed out that ground surveys of big game were even more likely to provide data based on non-random sampling simply because most of the survey area is very difficult to access on foot. The general problem is that inference is limited in all these wildlife surveys and we do not know the ‘population’ to which the numbers derived are applicable.

In an interesting paper that could apply directly to ecology papers, Williamson (2003) analyzed research papers in a nursing journal to ask if random sampling was utilized in contrast to convenience sampling. He found that only 32% of the 89 studies he reviewed used random sampling. I suspect that this kind of result would apply to much of medical research now, and it might be useful to repeat his kind of analysis with a current ecology journal. He did not consider the even more difficult issue of exactly what statistical population is specified in particular medical studies.

I would recommend that you should put a red flag up when you read “random” in an ecology paper and try to determine how exactly the term is used. But carry on with your research because:

Errors using inadequate data are much less than those using no data at all.

Charles Babbage (1792–1871

Krebs CJ (2010). Case studies and ecological understanding. Chapter 13 in: Billick I, Price MV, eds. The Ecology of Place: Contributions of Place-Based Research to Ecological Understanding. University of Chicago Press, Chicago, pp. 283-302. ISBN: 9780226050430

Rabe, M. J., Rosenstock, S. S. & deVos, J. C. (2002) Review of big-game survey methods used by wildlife agencies of the western United States. Wildlife Society Bulletin, 30, 46-52.

Williamson, G. R. (2003) Misrepresenting random sampling? A systematic review of research papers in the Journal of Advanced Nursing. Journal of Advanced Nursing, 44, 278-288. doi: 10.1046/j.1365-2648.2003.02803.x

 

On Indices of Population Abundance

A discussion with Murray Efford last week stimulated me to raise again this issue of using indices to measure population changes. One could argue that this issue has already been fully aired by Anderson (2003) and Engemann (2003) and I discussed it briefly in a blog about 2 years ago. The general agreement appears to be that mark-recapture estimation of population size is highly desirable if the capture procedure is clearly understood in relation to the assumption of the model of estimation. McKelvey and Pearson (2001) made this point with some elegant simulations. The best procedure then, if one wishes to replace mark-recapture methods with some index of abundance (track counts, songs, fecal pellets, etc.), is to calibrate the index with absolute abundance information of some type and show that the index and absolute abundance are very highly correlated. This calibration is difficult because there are few natural populations on which we know absolute abundance with high accuracy. We are left hanging with no clear path forward, particularly for monitoring programs that have little time or money to do extensive counting of any one species.

McKelvey and Pearson (2001) laid out a good guide for the use of indices in small mammal trapping, and showed that for many sampling programs the use of the number of unique individuals caught in a sampling session was a good index of population abundance, even though it is negatively biased. The key variable in all these discussions of mark-recapture models is the probability of capture of an individual animal living on the trapping area per session. Many years ago Leslie et al. (1953) considered this issue and the practical result was the recommendation that all subsequent work with small rodents should aim for a maximum probability of capture of individuals. The simplest way to do this was with highly efficient traps and large numbers of traps (single catch traps) so that there was always an excess of traps available for the population being censused. Krebs and Boonstra (1984) presented an analysis of trappability for several Microtus populations in which these recommendations were typically followed (Longworth traps in excess), and they found that the average per session detection probability ranged from about 0.6 to 0.9 for the four Microtus species studied. In all these studies live traps were present year round in the field, locked open when not in use, so the traps became part of the local environment for the voles. Clean live traps were much less likely to catch Microtus townsendii than dirty traps soiled with urine and feces (Boonstra and Krebs 1976). It is clear that minor behavioural quirks of the species under study may have significant effects on the capture data obtained. Individual heterogeneity in the probability of capture is a major problem in all mark-recapture work. But in the end natural history is as important as statistics.

There are at least two take home messages that can come from all these considerations. First, there are many statistical decisions that have to be made before population size can be estimated from mark-recapture data or any kind of quadrat based data. Second, there is also much biological information that must be well known before starting out with some kind of sampling design. Detectability may vary greatly with observers, with types of traps used, and observer skills so that again the devil is in the details. A third take home message given to me by someone who must remain nameless is that mark-recapture is hopeless as an ecological method because even after much work, the elusive population size that one wishes to know is lost in a pile of assumptions. But we cannot accept such a negative view without trying very hard to overcome the problems of sampling and estimation.

One way out of the box we find ourselves in (if we want to estimate population size) is to use an index of abundance and recognize its limitations. We cannot use quantitative population modelling on indices but we may find that indices are the best we can do for now. In particular, monitoring with little money must rely on indices of many populations of both plants and animals. Some data are better than no data for the management of populations and communities.

For the present time spatially explicit capture-recapture (SECR) methods of population estimation have provided a most useful approach to estimating density (Efford et al. 2009, 2013) and much future work will be needed to tell us how useful this relatively new approach is for accurately estimating population density (Broekhuis and Gopalaswamy 2016).

And a final reminder that even if you study community or ecosystem ecology, you must rely on getting measures of abundance for many quantitative models of system performance. So methods that provide accuracy for population sizes are just as essential for the vast array of ecological studies.

Anderson, D.R. 2003. Index values rarely constitute reliable information. Wildlife Society Bulletin 31(1): 288-291.

Boonstra, R. and Krebs, C.J. 1976. The effect of odour on trap response in Microtus townsendii. Journal of Zoology (London) 180(4): 467-476. Doi: 10.1111/j.1469-7998.1976.tb04692.x.

Broekhuis, F. and Gopalaswamy, A.M. 2016. Counting cats: Spatially explicit population estimates of cheetah (Acinonyx jubatus) using unstructured sampling data. PLoS ONE 11(5): e0153875. Doi: 10.1371/journal.pone.0153875.

Efford, M.G. and Fewster, R.M. 2013. Estimating population size by spatially explicit capture–recapture. Oikos 122(6): 918-928. Doi: 10.1111/j.1600-0706.2012.20440.x.

Efford, M.G., Dawson, D.K., and Borchers, D.L. 2009. Population density estimated from locations of individuals on a passive detector array. Ecology 90(10): 2676-2682. Doi: 10.1890/08-1735.1

Engeman, R.M. 2003. More on the need to get the basics right: population indices. Wildlife Society Bulletin 31(1): 286-287.

Krebs, C.J. and Boonstra, R. 1984. Trappability estimates for mark-recapture data. Canadian Journal of Zoology 62 (12): 2440-2444. Doi: 10.1139/z84-360

Leslie, P.H., Chitty, D., and Chitty, H. 1953. The estimation of population parameters from data obtained by means of the capture-recapture method. III. An example of the practical applications of the method. Biometrika 40 (1-2): 137-169. Doi:10.1093/biomet/40.1-2.137

McKelvey, K.S. & Pearson, D.E. (2001) Population estimation with sparse data: the role of estimators versus indices revisited. Canadian Journal of Zoology, 79(10): 1754-1765. Doi: 10.1139/cjz-79-10-1754

On Critical Questions in Biodiversity and Conservation Ecology

Biodiversity can be a vague concept with so many measurement variants to make one wonder what it is exactly, and how to incorporate ideas about biodiversity into scientific hypotheses. Even if we take the simplest concept of species richness as the operational measure, many questions arise about the importance of the rare species that make up most of the biodiversity but so little of the biomass. How can we proceed to a better understanding of this nebulous ecological concept that we continually put before the public as needing their attention?

Biodiversity conservation relies on community and ecosystem ecology for guidance on how to advance scientific understanding. A recent paper by Turkington and Harrower (2016) articulates this very clearly by laying out 7 general questions for analyzing community structure for conservation of biodiversity. As such these questions are a general model for community and ecosystem ecology approaches that are needed in this century. Thus it would pay to look at these 7 questions more closely and to read this new paper. Here is the list of 7 questions from the paper:

  1. How are natural communities structured?
  2. How does biodiversity determine the function of ecosystems?
  3. How does the loss of biodiversity alter the stability of ecosystems?
  4. How does the loss of biodiversity alter the integrity of ecosystems?
  5. Diversity and species composition
  6. How does the loss of species determine the ability of ecosystems to respond to disturbances?
  7. How does food web complexity and productivity influence the relative strength of trophic interactions and how do changes in trophic structure influence ecosystem function?

Turkington and Harrower (2016) note that each of these 7 questions can be asked in at least 5 different contexts in the biodiversity hotspots of China:

  1. How do the observed responses change across the 28 vegetation types in China?
  2. How do the observed responses change from the low productivity grasslands of the Qinghai Plateau to higher productivity grasslands in other parts of China?
  3. How do the observed responses change along a gradient in the intensity of human use or degradation?
  4. How long should an experiment be conducted given that the immediate results are seldom indicative of longer-term outcomes?
  5. How does the scale of the experiment influence treatment responses?

There are major problems in all of this as Turkington and Harrower (2016) and Bruelheide et al. (2014) have discussed. The first problem is to determine what the community is or what the bounds of an ecosystem are. This is a trivial issue according to community and ecosystem ecologists, and all one does is draw a circle around the particular area of interest for your study. But two points remain. Populations, communities, and ecosystems are open systems with no clear boundaries. In population ecology we can master this problem by analyses of movements and dispersal of individuals. On a short time scale plants in communities are fixed in position while their associated animals move on species-specific scales. Communities and ecosystems are not a unit but vary continuously in space and time, making their analysis difficult. The species present on 50 m2 are not the same as those on another plot 100 m or 1000 m away even if the vegetation types are labeled the same. So we replicate plots within what we define to be our community. If you are studying plant dynamics, you can experimentally place all plant species selected in defined plots in a pre-arranged configuration for your planting experiments, but you cannot do this with animals except in microcosms. All experiments are place specific, and if you consider climate change on a 100 year time scale, they are also time specific. We can hope that generality is strong and our conclusions will apply in 100 years but we do not know this now.

But we can do manipulative experiments, as these authors strongly recommend, and that brings a whole new set of problems, outlined for example in Bruelheide et al. (2014, Table 1, page 78) for a forestry experiment in southern China. Decisions about how many tree species to manipulate in what size of plots and what planting density to use are all potentially critical to the conclusions we reach. But it is the time frame of hypothesis testing that is the great unknown. All these studies must be long-term but whether this is 10 years or 50 years can only be found out in retrospect. Is it better to have, for example, forestry experiments around the world carried out with identical protocols, or to adopt a laissez faire approach with different designs since we have no idea yet of what design is best for answering these broad questions.

I suspect that this outline of the broad questions given in Turkington and Harrower (2016) is at least a 100 year agenda, and we need to be concerned how we can carry this forward in a world where funding of research questions has a 3 or 5 year time frame. The only possible way forward, until we win the Lottery, is for all researchers to carry out short term experiments on very specific hypotheses within this framework. So every graduate student thesis in experimental community and ecosystem ecology is important to achieving the goals outlined in these papers. Even if this 100 year time frame is optimistic and achievable, we can progress on a shorter time scale by a series of detailed experiments on small parts of the community or ecosystem at hand. I note that some of these broad questions listed above have been around for more than 50 years without being answered. If we redefine our objectives more precisely and do the kinds of experiments that these authors suggest we can move forward, not with the solution of grand ideas as much as with detailed experimental data on very precise questions about our chosen community. In this way we keep the long-range goal posts in view but concentrate on short-term manipulative experiments that are place and time specific.

This will not be easy. Birds are probably the best studied group of animals on Earth, and we now have many species that are changing in abundance dramatically over large spatial scales (e.g. http://www.stateofcanadasbirds.org/ ). I am sobered by asking avian ecologists why a particular species is declining or dramatically increasing. I never get a good answer, typically only a generally plausible idea, a hand waving explanation based on correlations that are not measured or well understood. Species recovery plans are often based on hunches rather than good data, with few of the key experiments of the type requested by Turkington and Harrower (2016). At the moment the world is changing rather faster than our understanding of these ecological interactions that tie species together in communities and ecosystems. We are walking when we need to be running, and even the Red Queen is not keeping up.

Bruelheide, H. et al. 2014. Designing forest biodiversity experiments: general considerations illustrated by a new large experiment in subtropical China. Methods in Ecology and Evolution, 5, 74-89. doi: 10.1111/2041-210X.12126

Turkington, R. & Harrower, W.L. 2016. An experimental approach to addressing ecological questions related to the conservation of plant biodiversity in China. Plant Diversity, 38, 1-10. Available at: http://journal.kib.ac.cn/EN/volumn/current.shtml

Hypothesis testing using field data and experiments is definitely NOT a waste of time

At the ESA meeting in 2014 Greg Dwyer (University of Chicago) gave a talk titled “Trying to understand ecological data without mechanistic models is a waste of time.” This theme has recently been reiterated on Dynamic Ecology Jeremy Fox, Brian McGill and Megan Duffy’s blog (25 January 2016 https://dynamicecology.wordpress.com/2016/01/25/trying-to-understand-ecological-data-without-mechanistic-models-is-a-waste-of-time/).  Some immediate responses to this blog have been such things as “What is a mechanistic model?” “What about the use of inappropriate statistics to fit mechanistic models,” and “prediction vs. description from mechanistic models”.  All of these are relevant and interesting issues in interpreting the value of mechanistic models.

The biggest fallacy however in this blog post or at least the title of the blog post is the implication that field ecological data are collected in a vacuum.  Hypotheses are models, conceptual models, and it is only in the absence of hypotheses that trying to understand ecological data is a “waste of time”. Research proposals that fund field work demand testable hypotheses, and testing hypotheses advances science. Research using mechanistic models should also develop testable hypotheses, but mechanistic models are certainly are not the only route to hypothesis creation of testing.

Unfortunately, mechanistic models rarely identify how the robustness and generality of the model output could be tested from ecological data and often fail comprehensively to properly describe the many assumptions made in constructing the model. In fact, they are often presented as complete descriptions of the ecological relationships in question, and methods for model validation are not discussed. Sometimes modelling papers include blatantly unrealistic functions to simplify ecological processes, without exploring the sensitivity of results to the functions.

I can refer to my own area of research expertise, population cycles for an example here.  It is not enough for example to have a pattern of ups and downs with a 10-year periodicity to claim that the model is an acceptable representation of cyclic population dynamics of for example a forest lepidopteran or snowshoe hares. There are many ways to get cyclic dynamics in modeled systems. Scientific progress and understanding can only be made if the outcome of conceptual, mechanistic or statistical models define the hypotheses that could be tested and the experiments that could be conducted to support the acceptance, rejection or modification of the model and thus to inform understanding of natural systems.

How helpful are mechanistic models – the gypsy moth story

Given the implication of Dwyer’s blog post (or at least blog post title) that mechanistic models are the only way to ecological understanding, it is useful to look at models of gypsy moth dynamics, one of Greg’s areas of modeling expertise, with the view toward evaluating whether model assumptions are compatible with real-world data Dwyer et al.  2004  (http://www.nature.com/nature/journal/v430/n6997/abs/nature02569.html)

Although there has been considerable excellent work on gypsy moth over the years, long-term population data are lacking.  Population dynamics therefore are estimated by annual estimates of defoliation carried out by the US Forest Service in New England starting in 1924. These data show periods of non-cyclicity, two ten-year cycles (peaks in 1981 and 1991 that are used by Dwyer for comparison to modeled dynamics for a number of his mechanistic models) and harmonic 4-5 year cycles between 1943 and1979 and since the 1991 outbreak. Based on these data 10-year cycles are the exception not the rule for introduced populations of gypsy moth. Point 1. Many of the Dwyer mechanistic models were tested using the two outbreak periods and ignored over 20 years of subsequent defoliation data lacking 10-year cycles. Thus his results are limited in their generality.

As a further example a recent paper, Elderd et al. (2013)  (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3773759/) explored the relationship between alternating long and short cycles of gypsy moth in oak dominated forests by speculating that inducible tannins in oaks modifies the interactions between gypsy moth larvae and viral infection. Although previous field experiments (D’Amico et al. 1998) http://onlinelibrary.wiley.com/doi/10.1890/0012-9658(1998)079%5b1104:FDDNAW%5d2.0.CO%3b2/abstract concluded that gypsy moth defoliation does not affect tannin levels sufficiently to influence viral infection, Elderd et al. (2013) proposed that induced tannins in red oak foliage reduces variation in viral infection levels and promotes shorter cycles. In this study, an experiment was conducted using jasmonic acid sprays to induce oak foliage. Point 2 This mechanistic model is based on experiments using artificially induced tannins as a mimic of insect damage inducing plant defenses. However, earlier fieldwork showed that foliage damage does not influence virus transmission and thus does not support the relevance of this mechanism.

In this model Elderd et al. (2013) use a linear relationship for viral transmission (transmission of infection on baculovirus density) based on two data points and the 0 intercept. In past mechanistic models and in a number of other systems the relationship between viral transmission and host density is nonlinear (D’Amico et al. 2005, http://onlinelibrary.wiley.com/doi/10.1111/j.0307-6946.2005.00697.x/abstract;jsessionid=D93D281ACD3F94AA86185EFF95AC5119.f02t02?userIsAuthenticated=false&deniedAccessCustomisedMessage= Fenton et al. 2002, http://onlinelibrary.wiley.com/doi/10.1046/j.1365-2656.2002.00656.x/full). Point 3. Data are insufficient to accurately describe the viral transmission relationship used in the model.

Finally the Elderd et al. (2013) model considers two types of gypsy moth habitat – one composed of 43% oaks that are inducible and the other of 15% oaks and the remainder of the forest composition is in adjacent blocks of non-inducible pines. Data show that gypsy moth outbreaks are limited to areas with high frequencies of oaks. In mixed forests, pines are only fed on by later instars of moth larvae when oaks are defoliated. The pines would be interspersed amongst the oaks not in separate blocks as in the modeled population. Point 4.  Patterns of forest composition in the models that are crucial to the result are unrealistic and this makes the interpretation of the results impossible.

Point 5 and conclusion. Because it can be very difficult to critically review someone else’s mechanistic model as model assumptions are often hidden in supplementary material and hard to interpret, and because relationships used in models are often arbitrarily chosen and not based on available data, it could be easy to conclude that “mechanistic models are misleading and a waste of time”. But of course that wouldn’t be productive. So my final point is that closer collaboration between modelers and data collectors would be the best way to ensure that the models are reasonable and accurate representations of the data.  In this way understanding and realistic predictions would be advanced. Unfortunately the great push to publish high profile papers works against this collaboration and manuscripts of mechanistic models rarely include data savvy referees.

D’Amico, V., J. S. Elkinton, G. Dwyer, R. B. Willis, and M. E. Montgomery. 1998. Foliage damage does not affect within-season transmission of an insect virus. Ecology 79:1104-1110.

D’Amico, V. D., J. S. Elkinton, P. J.D., J. P. Buonaccorsi, and G. Dwyer. 2005. Pathogen clumping: an explanation for non-linear transmission of an insect virus. Ecological Entomology 30:383-390.

Dwyer, G., F. Dushoff, and S. H. Yee. 2004. The combined effects of pathogens and predators on insect outbreaks. Nature 430:341-345.

Elderd, B. D., B. J. Rehill, K. J. Haynes, and G. Dwyer. 2013. Induced plant defenses, host–pathogen interactions, and forest insect outbreaks. Proceedings of the National Academy of Sciences 110:14978-14983.

Fenton, A., J. P. Fairbairn, R. Norman, and P. J. Hudson. 2002. Parasite transmission: reconciling theory and reality. Journal of Animal Ecology 71:893-905.