Category Archives: Experimental Design in Ecology

On Tipping Points and Regime Shifts in Ecosystems

Leave a reply

A new important paper raises red flags about our preoccupation with tipping points, alternative stable states and regime shifts (I’ll call them collectively sharp transitions) in ecosystems (Capon et al. 2015). I do not usually call attention to papers but this paper and a previous review (Mac Nally et al. 2014) seem to me to be critical for how we think about ecosystem changes in both aquatic and terrestrial ecosystems.

Consider an oversimplified example of how a sharp transition might work. Suppose we dumped fertilizer into a temperate clear-water lake. The clear water soon turns into pea soup with a new batch of algal species, a clear shift in the ecosystem, and this change is not good for many of the invertebrates or fish that were living there. Now suppose we stop dumping fertilizer into the lake. In time, and this could be a few years, the lake can either go back to its original state of clear water or it could remain as a pea soup lake for a very long time even though the pressure of added fertilizer was stopped. This second outcome would be a sharp transition, “you cannot go back from here” and the question for ecologists is how often does this happen? Clearly the answer is of great interest to natural resource managers and restoration ecologists.

The history of this idea for me was from the 1970s at UBC when Buzz Holling and Carl Walters were modelling the spruce budworm outbreak problem in eastern Canadian coniferous forests. They produced a model with a manifold surface that tipped the budworm from a regime of high abundance to one of low abundance (Holling 1973). We were all suitably amazed and began to wonder if this kind of thinking might be helpful in understanding snowshoe hare population cycles and lemming cycles. The evidence was very thin for the spruce budworm, but the model was fascinating. Then by the 1980s the bandwagon started to roll, and alternative stable states and regime change seemed to be everywhere. Many ideas about ecosystem change got entangled with sharp transition, and the following two reviews help to unravel them.

Of the 135 papers reviewed by Capon et al. (2015) very few showed good evidence of alternative stable states in freshwater ecosystems. They highlighted the use and potential misuse of ecological theory in trying to predict future ecosystem trajectories by managers, and emphasized the need of a detailed analysis of the mechanisms causing ecosystem change. In a similar paper for estuaries and near inshore marine ecosystems, Mac Nally et al. (2014) showed that of 376 papers that suggested sharp transitions, only 8 seemed to have sufficient data to satisfy the criteria needed to conclude that a transition had occurred and was linkable to an identifiable pressure. Most of the changes described in these studies are examples of gradual ecosystem changes rather than a dramatic shift; indeed, the timescale against which changes are assessed is critical. As always the devil is in the details.

All of this is to recognize that strong ecosystem changes do occur in response to human actions but they are not often sharp transitions that are closely linked to human actions, as far as we can tell now. And the general message is clearly to increase rigor in our ecological publications, and to carry out the long-term studies that provide a background of natural variation in ecosystems so that we have a ruler to measure human induced changes. Reviews such as these two papers go a long way to helping ecologists lift our game.

Perhaps it is best to end with part of the abstract in Capon et al. (2015):

“We found limited understanding of the subtleties of the relevant theoretical concepts and encountered few mechanistic studies that investigated or identified cause-and-effect relationships between ecological responses and nominal pressures. Our results mirror those of reviews for estuarine, nearshore and marine aquatic ecosystems, demonstrating that although the concepts of regime shifts and alternative stable states have become prominent in the scientific and management literature, their empirical underpinning is weak outside of a specific environmental setting. The application of these concepts in future research and management applications should include evidence on the mechanistic links between pressures and consequent ecological change. Explicit consideration should also be given to whether observed temporal dynamics represent variation along a continuum rather than categorically different states.”

 

Capon, S.J., Lynch, A.J.J., Bond, N., Chessman, B.C., Davis, J., Davidson, N., Finlayson, M., Gell, P.A., Hohnberg, D., Humphrey, C., Kingsford, R.T., Nielsen, D., Thomson, J.R., Ward, K., and Mac Nally, R. 2015. Regime shifts, thresholds and multiple stable states in freshwater ecosystems; a critical appraisal of the evidence. Science of The Total Environment 517(0): in press. doi:10.1016/j.scitotenv.2015.02.045.

Holling, C.S. 1973. Resilience and stability of ecological systems. Annual Review of Ecology and Systematics 4: 1-23. doi:10.1146/annurev.es.04.110173.000245.

Mac Nally, R., Albano, C., and Fleishman, E. 2014. A scrutiny of the evidence for pressure-induced state shifts in estuarine and nearshore ecosystems. Austral Ecology 39: 898-906. doi:10.1111/aec.12162.

The Anatomy of an Ecological Controversy – Dingos and Conservation in Australia

3 Replies

Conservation is a most contentious discipline, partly because it is ecology plus a moral stance. As such you might compare it to discussions about religious truths in the last several centuries but it is a discussion among scientists who accept the priority of scientific evidence. In Australia for the past few years there has been much discussion of the role of the dingo in protecting biodiversity via mesopredator release of foxes and cats (Allen et al. 2013; Colman et al. 2014; Hayward and Marlow 2014; Letnic et al. 2011, and many more papers). I do not propose here to declare a winner in this controversy but I want to dissect it as an example of an ecological issue with so many dimensions it could continue for a long time.

Dingos in Australia are viewed like wolves in North America – the ultimate enemy that must be reduced or eradicated if possible. When in doubt about what to do, killing dingos or wolves has become the first commandment of wildlife management and conservation. The ecologist would like to know, given this socially determined goal, what are the ecological consequences of reduction or eradication of dingos or wolves. How do we determine that?

The experimentalist suggests doing a removal experiment (or conversely a re-introduction experiment) so we have ecosystems with and without dingos (Newsome et al. 2015). This would have to be carried out on a large scale dependent on the home range size of the dingo and for a number of years so that the benefits or the costs of the removal would be clear. Here is the first hurdle, this kind of experiment cannot be done, and only a quasi-experiment is possible by finding areas that have dingos and others that do not have any (or a reduced population) and comparing ecosystems. This decision immediately introduces 5 problems:

  1. The areas with- and without- the dingo are not comparable in many respects. Areas with dingos for example may be national parks placed in the mountains or in areas that humans cannot use for agriculture, while areas with dingo control are in fertile agricultural landscapes with farming subsidies.
  2. Even given areas with and without dingos there is the problem of validating the usual dingo reduction carried out by poison baits or shooting. This is an important methodological issue.
  3. One has to census the mesopredators, in Australia foxes and cats, with further methodological issues of how to achieve that with accuracy.
  4. In addition one has to census the smaller vertebrates presumed to be possibly affected by the mesopredator offtake.
  5. Finally one has to do this for several years, possibly 5-10 years, particularly in variable environments, and in several pairs of areas chosen to represent the range of ecosystems of interest.

All in all this is a formidable research program, and one that has been carried out in part by the researchers working on dingos. And we owe them our congratulations for their hard work. The major part of the current controversy has been how one measures population abundance of all the species involved. The larger the organism, paradoxically the more difficult and expensive the methods of estimating abundance. Indirect measures, often from predator tracks in sand plots, are forced on researchers because of a lack of funding and the landscape scale of the problem. The essence of the problem is that tracks in sand or mud measure both abundance and activity. If movements increase in the breeding season, tracks may indicate activity more than abundance. If old roads are the main sampling sites, the measurements are not a random sample of the landscape.

This monumental sampling headache can be eliminated by the bold stroke of concluding with Nimmo et al. (2015) and Stephens et al. (2015) that indirect measures of abundance are sufficient for guiding actions in conservation management. They may be, they may not be, and we fall back into the ecological dilemma that different ecosystems may give different answers. And the background question is what level of accuracy do you need in your study? We are all in a hurry now and want action for conservation. If you need to know only whether you have “few” or “many” dingos or tigers in your area, indirect methods may well serve the purpose. We are rushing now into the “Era of the Camera” in wildlife management because the cost is low and the volume of data is large. Camera ecology may be sufficient for occupancy questions, but may not be enough for demographic analysis without detailed studies.

The moral issue that emerges from this particular dingo controversy is similar to the one that bedevils wolf control in North America and Eurasia – should we remove large predators from ecosystems? The ecologist’s job is to determine the biodiversity costs and benefits of such actions. But in the end we are moral beings as well as ecologists, and for the record, not the scientific record but the moral one, I think it is poor policy to remove dingos, wolves, and all large predators from ecosystems. Society however seems to disagree.

 

Allen, B.L., Allen, L.R., Engeman, R.M., and Leung, L.K.P. 2013. Intraguild relationships between sympatric predators exposed to lethal control: predator manipulation experiments. Frontiers in Zoology 10(39): 1-18. doi:10.1186/1742-9994-10-39.

Colman, N.J., Gordon, C.E., Crowther, M.S., and Letnic, M. 2014. Lethal control of an apex predator has unintended cascading effects on forest mammal assemblages. Proceedings of the Royal Society of London, Series B 281(1803): 20133094. doi:DOI: 10.1098/rspb.2013.3094.

Hayward, M.W., and Marlow, N. 2014. Will dingoes really conserve wildlife and can our methods tell? Journal of Applied Ecology 51(4): 835-838. doi:10.1111/1365-2664.12250.

Letnic, M., Greenville, A., Denny, E., Dickman, C.R., Tischler, M., Gordon, C., and Koch, F. 2011. Does a top predator suppress the abundance of an invasive mesopredator at a continental scale? Global Ecology and Biogeography 20(2): 343-353. doi:10.1111/j.1466-8238.2010.00600.x.

Newsome, T.M., et al. (2015) Resolving the value of the dingo in ecological restoration. Restoration Ecology, 23 (in press). doi: 10.1111/rec.12186

Nimmo, D.G., Watson, S.J., Forsyth, D.M., and Bradshaw, C.J.A. 2015. Dingoes can help conserve wildlife and our methods can tell. Journal of Applied Ecology 52. (in press, 27 Jan. 2015). doi:10.1111/1365-2664.12369.

Stephens, P.A., Pettorelli, N., Barlow, J., Whittingham, M.J., and Cadotte, M.W. 2015. Management by proxy? The use of indices in applied ecology. Journal of Applied Ecology 52(1): 1-6. doi:10.1111/1365-2664.12383.

A Survey of Strong Inference in Ecology Papers: Platt’s Test and Medawar’s Fraud Model

10 Replies

In 1897 Chamberlin wrote an article in the Journal of Geology on the method of multiple working hypotheses as a way of experimentally testing scientific ideas (Chamberlin 1897 reprinted in Science). Ecology was scarcely invented at that time and this has stimulated my quest here to see if current ecology journals subscribe to Chamberlin’s approach to science. Platt (1964) formalized this approach as “strong inference” and argued that it was the best way for science to progress rapidly. If this is the case (and some do not agree that this approach is suitable for ecology) then we might use this model to check now and then on the state of ecology via published papers.

I did a very small survey in the Journal of Animal Ecology for 2015. Most ecologists I hope would classify this as one of our leading journals. I asked the simple question of whether in the Introduction to each paper there were explicit hypotheses stated and explicit alternative hypotheses, and categorized each paper as ‘yes’ or ‘no’. There is certainly a problem here in that many papers stated a hypothesis or idea they wanted to investigate but never discussed what the alternative was, or indeed if there was an alternative hypothesis. As a potential set of covariates, I tallied how many times the word ‘hypothesis’ or ‘hypotheses’ occurred in each paper, as well as the word ‘test’, ‘prediction’, and ‘model’. Most ‘model’ and ‘test’ words were used in the context of statistical models or statistical tests of significance. Singular and plural forms of these words were all counted.

This is not a publication and I did not want to spend the rest of my life looking at all the other ecology journals and many issues, so I concentrated on the Journal of Animal Ecology, volume 84, issues 1 and 2 in 2015. I obtained these results for the 51 articles in these two issues: (number of times the word appeared per article, averaged over all articles)

Explicit hypothesis and alternative hypotheses

“Hypothesis”

“Test”

“Prediction”

“Model”

Yes

22%

Mean

3.1

7.9

6.5

32.5

No

78%

Median

1

6

4

20

No. articles

51

Range

0-23

0-37

0-27

0-163

There are lots of problems with a simple analysis like this and perhaps its utility may lie in stimulating a more sophisticated analysis of a wider variety of journals. It is certainly not a random sample of the ecology literature. But maybe it gives us a few insights into ecology 2015.

I found the results quite surprising in that many papers failed Platt’s Test for strong inference. Many papers stated hypotheses but failed to state alternative hypotheses. In some cases the implied alternative hypothesis is the now-discredited null hypothesis (Johnson 2002). One possible reason for the failure to state hypotheses clearly was discussed by Medawar many years ago (Howitt and Wilson 2014; Medawar 1963). He pointed out that most scientific papers were written backwards, analysing the data, finding out what it concluded, and then writing the introduction to the paper knowing the results to follow. A significant number of papers in these issues I have looked at here seem to have been written following Medawar’s “fraud model”.

But make of such data as you will, and I appreciate that many people write papers in a less formal style than Medawar or Platt would prefer. And many have alternative hypotheses in mind but do not write them down clearly. And perhaps many referees do not think we should be restricted to using the hypothetical deductive approach to science. All of these points of view should be discussed rather than ignored. I note that some ecological journals now turn back papers that have no clear statement of a hypothesis in the introduction to the submitted paper.

The word ‘model’ is the most common word to appear in this analysis, typically in the case of a statistical model evaluated by AIC kinds of statistics. And the word ‘test’ was most commonly used in statistical tests (‘t-test’) in a paper. Indeed virtually all of these paper overflow with statistical estimates of various kinds. Few however come back in the conclusions to state exactly what progress has been made by their paper and even less make statements about what should be done next. From this small survey there is considerable room for improvement in ecological publications.

Chamberlin, T.C. 1897. The method of multiple working hypotheses. Journal of Geology 5: 837-848 (reprinted in Science 148: 754-759 in 1965). doi:10.1126/science.148.3671.754

Howitt, S.M., and Wilson, A.N. 2014. Revisiting “Is the scientific paper a fraud?”. EMBO reports 15(5): 481-484. doi:10.1002/embr.201338302

Johnson, D.H. (2002) The role of hypothesis testing in wildlife science. Journal of Wildlife Management 66(2): 272-276. doi: 10.2307/3803159

Medawar, P.B. 1963. Is the scientific paper a fraud? In “The Threat and the Glory”. Edited by P.B. Medawar. Harper Collins, New York. pp. 228-233. (Reprinted by Harper Collins in 1990. ISBN: 9780060391126.)

Platt, J.R. 1964. Strong inference. Science 146: 347-353. doi:10.1126/science.146.3642.347

On Repeatability in Ecology

Leave a reply

One of the elementary lessons of statistics is that every measurement must be repeatable so that differences or changes in some ecological variable can be interpreted with respect to some ecological or environmental mechanism. So if we count 40 elephants in one year and count 80 in the following year, we know that population abundance has changed and we do not have to consider the possibility that the repeatability of our counting method is so poor that 40 and 80 could refer to the same population size. Both precision and bias come into the discussion at this point. Much of the elaboration of ecological methods involves the attempt to improve the precision of methods such as those for estimating abundance or species richness. There is less discussion of the problem of bias.

The repeatability that is most crucial in forging a solid science is that associated with experiments. We should not simply do an important experiment in a single place and then assume the results apply world-wide. Of course we do this, but we should always remember that this is a gigantic leap of faith. Ecologists are often not willing to repeat critical experiments, in contrast to scientists in chemistry or molecular biology. Part of this reluctance is understandable because the costs associated with many important field experiments is large and funding committees must then judge whether to repeat the old or fund the new. But if we do not repeat the old, we never can discover the limits to our hypotheses or generalizations. Given a limited amount of money, experimental designs often limit the potential generality of the conclusions. Should you have 2 or 4 or 6 replicates? Should you have more replicates and fewer treatment sites or levels of manipulation? When we can, we try one way and then another to see if we get similar results.

A looming issue now is climate change which means that the ecosystem studied in 1980 is possibly rather different than the one you now study in 2014, or the place someone manipulated in 1970 is not the same community you manipulated this year. The worst case scenario would be to find out that you have to do the same experiment every ten years to check if the whole response system has changed. Impossible with current funding levels. How can we develop a robust set of generalizations or ‘theories’ in ecology if the world is changing so that the food webs we so carefully described have now broken down? I am not sure what the answers are to these difficult questions.

And then you pile evolution into this mix and wonder if organisms can change like Donelson et al.’s (2012) tropical reef fish, so that climate changes might be less significant than we currently think, at least for some species. The frustration that ecologists now face over these issues with respect to ecosystem management boils over in many verbal discussions like those on “novel ecosystems” (Hobbs et al. 2014, Aronson et al. 2014) that can be viewed as critical decisions about how to think about environmental change or a discussion about angels on pinheads.

Underlying all of this is the global issue of repeatability, and whether our current perceptions of how to manage ecosystems is sufficiently reliable to sidestep the adaptive management scenarios that seem so useful in theory (Conroy et al. 2011) but are at present rare in practice (Keith et al. 2011). The need for action in conservation biology seems to trump the need for repeatability to test the generalizations on which we base our management recommendations. This need is apparent in all our sciences that affect humans directly. In agriculture we release new varieties of crops with minimal long term studies of their effects on the ecosystem, or we introduce new methods such as no till agriculture without adequate studies of its impacts on soil structure and pest species. This kind of hubris does guarantee long term employment in mitigating adverse consequences, but is perhaps not an optimal way to proceed in environmental management. We cannot follow the Hippocratic Oath in applied ecology because all our management actions create winners and losers, and ‘harm’ then becomes an opinion about how we designate ‘winners’ and ‘losers’. Using social science is one way out of this dilemma, but history gives sparse support for the idea of ‘expert’ opinion for good environmental action.

Aronson, J., Murcia, C., Kattan, G.H., Moreno-Mateos, D., Dixon, K. & Simberloff, D. (2014) The road to confusion is paved with novel ecosystem labels: a reply to Hobbs et al. Trends in Ecology & Evolution, 29, 646-647.

Conroy, M.J., Runge, M.C., Nichols, J.D., Stodola, K.W. & Cooper, R.J. (2011) Conservation in the face of climate change: The roles of alternative models, monitoring, and adaptation in confronting and reducing uncertainty. Biological Conservation, 144, 1204-1213.

Donelson, J.M., Munday, P.L., McCormick, M.I. & Pitcher, C.R. (2012) Rapid transgenerational acclimation of a tropical reef fish to climate change. Nature Climate Change, 2, 30-32.

Hobbs, R.J., Higgs, E.S. & Harris, J.A. (2014) Novel ecosystems: concept or inconvenient reality? A response to Murcia et al. Trends in Ecology & Evolution, 29, 645-646.

Keith, D.A., Martin, T.G., McDonald-Madden, E. & Walters, C. (2011) Uncertainty and adaptive management for biodiversity conservation. Biological Conservation, 144, 1175-1178.

On Research Questions in Ecology

Leave a reply

I have done considerable research in arctic Canada on questions of population and community ecology, and perhaps because of this I get e mails about new proposals. This one just arrived from a NASA program called ABoVE that is just now starting up.

“Climate change in the Arctic and Boreal region is unfolding faster than anywhere else on Earth, resulting in reduced Arctic sea ice, thawing of permafrost soils, decomposition of long- frozen organic matter, widespread changes to lakes, rivers, coastlines, and alterations of ecosystem structure and function. NASA’s Terrestrial Ecology Program is in the process of planning a major field campaign, the Arctic-Boreal Vulnerability Experiment (ABoVE), which will take place in Alaska and western Canada during the next 5 to 8 years.“

“The focus of this solicitation is the initial research to begin the Arctic-Boreal Vulnerability Experiment (ABoVE) field campaign — a large-scale study of ecosystem responses to environmental change in western North America’s Arctic and boreal region and the implications for social-ecological systems. The Overarching Science Question for ABoVE is: “How vulnerable or resilient are ecosystems and society to environmental change in the Arctic and boreal region of western North America? “

I begin by noting that Peters (1991) wrote very much about the problems with these kinds of ‘how’ questions. First of all note that this is not a scientific question. There is no conceivable way to answer this question. It contains a set of meaningless words to an ecologist who is interested in testing alternative hypotheses.

One might object that this is not a research question but a broad brush agenda for more detailed proposals that will be phrased in such a way to become scientific questions. Yet it boggles the mind to ask how vulnerable ecosystems are to anything unless one is very specific. One has to define an ecosystem, difficult if it is an open system, and then define what vulnerable means operationally, and then define what types of environmental changes should be addressed – temperature, rainfall, pollution, CO2. And all of that over the broad expanse of arctic and boreal western North America, a sampling problem on a gigantic scale. Yet an administrator or politician could reasonably ask at the end of this program, ‘Well, what is the answer to this question?’ That might be ‘quite vulnerable’, and then we could go on endlessly with meaningless questions and answers that might pass for science on Fox News but not I would hope at the ESA. We can in fact measure how primary production changes over time, how much CO2 is sequestered or released from the soils of the arctic and boreal zone, but how do we translate this into resilience, another completely undefined empirical ecological concept?

We could attack the question retrospectively by asking for example: How resilient have arctic ecosystems been to the environmental changes of the past 30 years? We can document that shrubs have increased in abundance and biomass in some areas of the arctic and boreal zone (Myers-Smith et al. 2011), but what does that mean for the ecosystem or society in particular? We could note that there are almost no data on these questions because funding for northern science has been pitiful, and that raises the issue that if these changes we are asking about occur on a time scale of 30 or 50 years, how will we ever keep monitoring them over this time frame when research is doled out in 3 and 5 year blocks?

The problem of tying together ecosystems and society is that they operate on different time scales of change. Ecosystem changes in terrestrial environments of the North are slow, societal changes are fast and driven by far more obvious pressures than ecosystem changes. The interaction of slow and fast variables is hard enough to decipher scientifically without having many external inputs.

So perhaps in the end this Arctic-Boreal Vulnerability Experiment (another misuse of the word ‘experiment’) will just describe a long-term monitoring program and provide the funding for much clever ecological research, asking specific questions about exactly what parts of what ecosystems are changing and what the mechanisms of change involve. Every food web in the North is a complex network of direct and indirect interactions, and I do not know anyone who has a reliable enough understanding to predict how vulnerable any single element of the food web is to climate change. Like medieval scholars we talk much about changes of state or regime shifts, or tipping points with a model of how the world should work, but with little long term data to even begin to answer these kinds of political questions.

My hope is that this and other programs will generate some funding that will allow ecologists to do some good science. We may be fiddling while Rome is burning, but at any rate we could perhaps understand why it is burning. That also raises the issue of whether or not understanding is a stimulus for action on items that humans can control.

Myers-Smith, I.H., et al. (2011) Expansion of canopy-forming willows over the 20th century on Herschel Island, Yukon Territory, Canada. Ambio, 40, 610-623.

Peters, R.H. (1991) A Critique for Ecology. Cambridge University Press, Cambridge, England. 366 pp.

On Indices of Population Abundance

Leave a reply

I am often surprised at ecological meetings by how many ecological studies rely on indices rather than direct measures. The most obvious cases involve population abundance. Two common criteria for declaring a species as endangered are that its population has declined more than 70% in the last ten years (or three generations) or that its population size is less than 2500 mature individuals. The criteria are many and every attempt is made to make them quantitative. But too often the methods used to estimate changes in population abundance are based on an index of population size, and all too rarely is the index calibrated against known abundances. If an index increases by 2-fold, e.g. from 20 to 40 counts, it is not at all clear that this means the population size has increased 2-fold. I think many ecologists begin their career thinking that indices are useful and reliable and end their career wondering if they are providing us with a correct picture of population changes.

The subject of indices has been discussed many times in ecology, particularly among applied ecologists. Anderson (2001) challenged wildlife ecologists to remember that indices include an unmeasured term, detectability: Anderson (2001, p. 1295) wrote:

“While common sense might suggest that one should estimate parameters of interest (e.g., population density or abundance), many investigators have settled for only a crude index value (e.g., “relative abundance”), usually a raw count. Conceptually, such an index value (c) is the product of the parameter of interest (N) and a detection or encounter probability (p): then c=pN “

He noted that many indices used by ecologists make a large assumption that the probability of encounter is a constant over time and space and individual observers. Much of the discussion of detectability flowed from these early papers (Williams, Nichols & Conroy 2002; Southwell, Paxton & Borchers 2008). There is an interesting exchange over Anderson’s (2001) paper by Engeman (2003) followed by a retort by Anderson (2003) that ended with this blast at small mammal ecologists:

“Engeman (2003) notes that McKelvey and Pearson (2001) found that 98% of the small-mammal studies reviewed resulted in too little data for valid mark-recapture estimation. This finding, to me, reflects a substantial failure of survey design if these studies were conducted to estimate population size. ……..O’Connor (2000) should not wonder “why ecology lags behind biology” when investigators of small-mammal communities commonly (i.e., over 700 cases) achieve sample sizes <10. These are empirical methods; they cannot be expected to perform well without data.” (page 290)

Take that you small mammal trappers!

The warnings are clear about index data. In some cases they may be useful but they should never be used as population abundance estimates without careful validation. Even by small mammal trappers like me.

Anderson, D.R. (2001) The need to get the basics right in wildlife field studies. Wildlife Society Bulletin, 29, 1294-1297.

Anderson, D.R. (2003) Index values rarely constitute reliable information. Wildlife Society Bulletin, 31, 288-291.

Engeman, R.M. (2003) More on the need to get the basics right: population indices. Wildlife Society Bulletin, 31, 286-287.

McKelvey, K.S. & Pearson, D.E. (2001) Population estimation with sparse data: the role of estimators versus indices revisited. Canadian Journal of Zoology, 79, 1754-1765.

O’Connor, R.J. (2000) Why ecology lags behind biology. The Scientist, 14, 35.

Southwell, C., Paxton, C.G.M. & Borchers, D.L. (2008) Detectability of penguins in aerial surveys over the pack-ice off Antarctica. Wildlife Research, 35, 349-357.

Williams, B.K., Nichols, J.D. & Conroy, M.J. (2002) Analysis and Management of Animal Populations. Academic Press, New York.

Back to p-Values

1 Reply

Alas ecology has slipped lower on the totem-pole of serious sciences by an article that has captured the attention of the media:

Low-Décarie, E., Chivers, C., and Granados, M. 2014. Rising complexity and falling explanatory power in ecology. Frontiers in Ecology and the Environment 12(7): 412-418. doi: 10.1890/130230.

There is much that is positive in this paper, so you should read it if only to decide whether or not to use it in a graduate seminar in statistics or in ecology. Much of what is concluded is certainly true, that there are more p-values in papers now than there were some years ago. The question then comes down to what these kinds of statistics mean and how this would justify a conclusion captured by the media that explanatory power in ecology is declining over time, and the bottom line of what to do about falling p-values. Since as far as I can see most statisticians today seem to believe that p-values are meaningless (e.g. Ioannidis 2005), one wonders what the value of showing this trend is. A second item that most statisticians agree about is that R2 values are a poor measure of anything other than the items in a particular data set. Any ecological paper that contains data to be analysed and reported summarizes many tests providing p-values and R2 values of which only some are reported. It would be interesting to do a comparison with what is recognized as a mature science (like physics or genetics) by asking whether the past revolutions in understanding and prediction power in those sciences corresponded with increasing numbers of p-values or R2 values.

To ask these questions is to ask what is the metric of scientific progress? At the present time we confuse progress with some indicators that may have little to do with scientific advancement. As journal editors we race to increase their impact factor which is interpreted as a measure of importance. For appointments to university positions we ask how many citations a person has and how many papers they have produced. We confuse scientific value with some numbers which ironically might have a very low R2 value as predictors of potential progress in a science. These numbers make sense as metrics to tell publication houses how influential their journals are, or to tell Department Heads how fantastic their job choices are, but we fool ourselves if we accept them as indicators of value to science.

If you wish to judge scientific progress you might wish to look at books that have gathered together the most important papers of the time, and examine a sequence of these from the 1950s to the present time. What is striking is that papers that seemed critically important in the 1960s or 1970s are now thought to be concerned with relatively uninteresting side issues, and conversely papers that were ignored earlier are now thought to be critical to understanding. A list of these changes might be a useful accessory to anyone asking about how to judge importance or progress in a science.

A final comment would be to look at the reasons why a relatively mature science like geology has completely failed to be able to predict earthquakes in advance and even to specify the locations of some earthquakes (Steina et al. 2012; Uyeda 2013). Progress in understanding does not of necessity dictate progress in prediction. And we ought to be wary of confusing progress with p-and R2 values.

Ioannidis, J.P.A. 2005. Why most published research findings are false. PLoS Medicine 2(8): e124.

Steina, S., Gellerb, R.J., and Liuc, M. 2012. Why earthquake hazard maps often fail and what to do about it. Tectonophysics 562-563: 1-24. doi: 10.1016/j.tecto.2012.06.047.

Uyeda, S. 2013. On earthquake prediction in Japan. Proceedings of the Japan Academy, Series B 89(9): 391-400. doi: 10.2183/pjab.89.391.

Models need testable predictions to be useful

2 Replies

It has happened again.  I have just been to a seminar on genetic models – something about adaptation of species on the edges of their ranges.  Yes this is an interesting topic of relevance to interpreting species’ responses to changing environments.  It ended by the speaker saying something like, “It would be a lot of work to test this in the field”. How much more useful my hour would have been spent if the talk had ended with “Although it would be difficult to do, this model makes the following predictions that could be tested in the field,” or “The following results would reject the hypothesis upon which this model is based.”

Now it is likely that some found these theoretical machinations interesting and satisfying in some mathematical way, but I feel that it is irresponsible to not even consider how a model could be tested and the possibility (a likely possibility at that) that it doesn’t apply to nature and tells us nothing helpful about understanding what is going to happen to willow or birch shrubs at the edge of their ranges in the warming arctic (for example).

Recommendation – no paper on models should be published or talked about unless it makes specific, testable predictions of how the model can be tested.

Experimental Model Systems in Ecology

Leave a reply

Ecology progresses slowly when we have to study natural populations or communities. It is expensive to manipulate large units of habitat, and there are two approaches that suggest themselves to alleviate this problem. First, study small areas that can be analysed and manipulated by one or two persons. This can be a useful approach, depending on your question and hypotheses, and I do not discuss this approach here. The second approach is through experimental model systems. Typically this means taking the question or problem into a semi-laboratory system. For aquatic studies it may mean putting large cylinders in a lake (Carpenter 1996). For rodent studies it may mean putting populations into small fenced enclosures. For sake of clarity I will discuss this latter example with which I am familiar.

The key question for all experimental model systems in ecology is to know at what spatial and temporal scale the system works. To gain precision we typically want to conduct our studies within an enclosure of some small size. That is, we wish to study an open system with more precision by converting it to a closed system of some much smaller size. But what size allows the system to operate as an open natural population, in this example of rodents? In a sense we wish to know the shape of this generalized curve:

EMS_Drawing1

Assume there is some natural outcome known for the particular study. In the case of small rodents this might be that the population fluctuates in periodic ‘cycles’. The question then is what size of enclosure is needed to observe this same population trend. One simple way of looking at this is to ask for islands, what size of island allows a closed population to fluctuate in ‘cycles’. For this particular problem we know that you cannot observe ‘cycles’ in small rooms in the laboratory or even in 1 ha field enclosures.

Many other examples can be given for this type of question in ecology. For example, we may know that infanticide in a particular species is rare in natural populations. But if we raise the same species in small cages in the laboratory, we may observe infanticide very commonly. We would conclude that this is not the natural state of this system, and thus decide that you could not draw conclusions about the frequency of infanticide by studying it in small cages.

The critical judgement is whether any experimental model system we design will mimic natural processes that occur in open, real world populations or communities. All too often in ecological studies we assume that the size of the enclosure or study area that we are using is “natural” and the conclusions will represent what happens in natural populations or communities. In an ideal world we would examine a series of sizes of our study enclosures to see the best one that mimics natural outcomes. But this cannot always be done for reasons of time and money. In some cases we have no idea what the natural situation is, and in these cases it is most difficult to know if our model system results bear any relationship to reality.

This whole issue is another way of looking at the problem of habitat fragmentation – how small a piece of habitat can we get by with to conserve species X or community Y? These types of conservation questions always involve a temporal as well as a spatial dimension, given the problem of extinction debts (Krauss et al. 2010). In the extreme case we can argue that we can conserve at least some species in zoos, but this is a way of avoiding the main goal of conserving natural environments and processes.

The bottom line is to ask yourself as you are setting up a study using an experimental model system approach whether the process you are investigating can be observed at the spatial and temporal scale you have available. Alternatively it may be important to try to construct the curve shown above for the system of interest. This question is important because some previous studies for any ecological system may have reached invalid conclusions because of a faulty spatial scale of the model system.

Carpenter, S. R. 1996. Microcosm experiments have limited relevance for community and ecosystem ecology. Ecology 77:677-680.

Krauss, J., R. Bommarco, M. Guardiola, R. K. Heikkinen, A. Helm, M. Kuussaari, R. Lindborg, E. Öckinger, M. Pärtel, J. Pino, J. Pöyry, K. M. Raatikainen, A. Sang, C. Stefanescu, T. Teder, M. Zobel, and I. Steffan-Dewenter. 2010. Habitat fragmentation causes immediate and time-delayed biodiversity loss at different trophic levels. Ecology Letters 13:597-605.