Monthly Archives: October 2014

Citation Analysis Gone Crazy

2 Replies

Perhaps we should stop and look at the evils of citation analysis in science. Citation analysis began some 15 or 20 years ago with a useful thought that it might be nice to know if one’s scientific papers were being read and used by others working in the same area. But now it has morphed into a Godzilla that has the potential to run our lives. I think the current situation rests on three principles:

  1. Your scientific ability can be measured by the number of citations you receive. This is patent nonsense.
  2. The importance of your research is determined by which journals accept your papers. More nonsense.
  3. Your long-term contribution to ecological science can be measured precisely by your h–score or some variant.

These principles appeal greatly to the administrators of science and to many people who dish out the money for scientific research. You can justify your decisions with numbers. Excellent job to make the research enterprise quantitative. The contrary view which I might hope is held by many scientists rests on three different principles:

  1. Your scientific ability is difficult to measure and can only be approximately evaluated by another scientist working in your field. Science is a human enterprise not unlike music.
  2. The importance of your research is impossible to determine in the short term of a few years, and in a subject like ecology probably will not be recognized for decades after it is published.
  3. Your long-term contribution to ecological science will have little to do with how many citations you accumulate.

It will take a good historian to evaluate these alternative views of our science.

This whole issue would not matter except for the fact that it is eroding science hiring and science funding. The latest I have heard is that Norwegian universities are now given a large amount of money by the government if they publish a paper in SCIENCE or NATURE, and a very small amount of money if they publish the same results in the CANADIAN JOURNAL OF ZOOLOGY or – God forbid – the CANADIAN FIELD NATURALIST (or equivalent ‘lower class’ journals). I am not sure how many other universities will fall under this kind of reward-based publication scores. All of this is done I think because we do not wish to involve the human judgment factor in decision making. I suppose you could argue that this is a grand experiment like climate change (with no controls) – use these scores for 30 years and then see if they worked better than the old system based on human judgment. How does one evaluate such experiments?

NSERC (Natural Sciences and Engineering Research Council) in Canada has been trending in that direction in the last several years. In the eternal good old days scientists read research proposals and made judgments about the problem, the approach, and the likelihood of success of a research program. They took time to discuss at least some of the issues. But we move now into quantitative scores that replace human judgment, which I believe to be a very large mistake.

I view ecological research and practice much like I think medical research and medical practice operate. We do not know how well certain studies and experiment will work, any more than a surgeon knows exactly whether a particular technique or treatment will work or a particular young doctor will be a good surgeon, and we gain by experience in a mostly non-quantitative manner. Meanwhile we should encourage young scientists to try new ideas and studies, to give them opportunities based on judgments rather than on counts of papers or citations. Currently we want to rank everyone and every university like sporting teams and find out the winner. This is a destructive paradigm for science. It works for tennis but not for ecology.

Bornmann, L. & Marx, W. (2014) How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations. Scientometrics, 98, 487-509.

Leimu, R. & Koricheva, J. (2005) What determines the citation frequency of ecological papers? Trends in Ecology & Evolution, 20, 28-32.

Parker, J., Lortie, C. & Allesina, S. (2010) Characterizing a scientific elite: the social characteristics of the most highly cited scientists in environmental science and ecology. Scientometrics, 85, 129-143.

Todd, P.A., Yeo, D.C.J., Li, D. & Ladle, R.J. (2007) Citing practices in ecology: can we believe our own words? Oikos, 116, 1599-1601.

Large Mammal Conservation

Leave a reply

The conservation problem is largely focused on large things, birds and mammals, with a few pretty things like butterflies thrown in. What concerns me is the current distortion in the conservation knowledge base available for large animal conservation. I will talk largely about mammals but large birds are equally a problem.

The difficulty is this. It is nearly impossible to study large mammals because they are scarce on the ground, so census methods must be spatially extensive and thus very expensive. One needs a big budget to do this properly, and this effectively rules out university scientists unless they can collaborate with government biologists who have large budgets or private consortia who need the large mammals so they can shoot them. But even with a large budget, a large mammal ecologist cannot be very productive as measured in papers per year of research effort. So the universities in general have shied away from hiring young scientists who might be described as large mammal ecologists. This produces positive feedback in the job market so that few young scientists see this as a viable career.

All of this would be changed if governments were hiring large mammal ecologists. But they are not, with few exceptions. Governments at least in Canada and Australia have been shedding ecologists of all varieties while all the time professing how much they are doing for conservation of threatened species. The advantage of this approach for governments is that they shed high cost biologists, and cover their tracks with some hiring of public relations personnel who have no field costs and perhaps a few biologists who concentrate on small creatures and local problems. So we reach a stalemate when it comes to large mammal conservation. Why do we need polar bear scientists when all they do is make trouble? We can escape such trouble easily. Count the polar bears or the caribou every 5 years or so, so there is consequently much less information that scientists can put their fingers on. (Imagine if we counted the stock market once every 5 years.) The consequence is that in many areas we have large scale, long-term problems with few scientists and only small scale funding to find out what is happening in the field. For polar bears this seems to be partly alleviated by private funding from people who care, while the government shirks its duties for future generations.

For caribou in Canada the situation is worse because the problem is spread over more than half of Canada so the funding and person-power needed for conservation is much larger, and this is further compounded by the immediate conflict of caribou with industrial developments in oil, gas, and forestry. When dollars conflict with conservation needs, it is best not to bet on conservation winning. What good has a polar bear or a caribou done for you?

The potential consequence of all this is that we slowly lose populations of these large iconic species. If this loss is slow enough, no one seems to notice save a few concerned conservation biologists who do not own the newspapers and TV stations. And conservation ecologists grow pessimistic that we can save these large species that require much habitat and freedom from disturbance. The solutions seem to be two. First, build a big fence and keep them in a very large zoo (Packer et al. 2013). This will work for some species like caribou, as Kruger Park in South Africa illustrates so well with African large mammals (but some disagree, Creel et al. 2013). But the fence-solution will not work for polar bears, and our best response for their conservation may be to cross our fingers and hope, all the while trying to slow down the losses in the best way we can. A second solution is to decide that these large mammal conservation situations are not scientific but sociological, and progress can best be made by doing good sociological research to change the attitudes of humans about the value of biodiversity. If this is the solution, we do not need to worry that there are no biologists available to investigate the conservation issues of large mammals.

I think perhaps the bottom line is that it takes a spirited soul to aim for a career in large mammal conservation research and we hope that this happens and the conservation future for large mammals in Canada grows brighter.

Creel, S., 2013. Conserving large populations of lions – the argument for fences has holes. Ecology Letters 16 (5): 635-641. doi: 10.1111/ele.12145.

Packer, C., et al. 2013. Conserving large carnivores: dollars and fence. Ecology Letters. 16: 1413-e3. doi: 10.1111/ele.12091.

Pauly, D. 1995. Anecdotes and the shifting baseline syndrome of fisheries. Trends in Ecology and Evolution 10: 430.

The Secretary’s Dilemma

Leave a reply

Back in the good old days when Departments of Biology had secretaries that did the typing of formal letters, one problem always stumped me. Let us say I have a letter of reference that must be typed on departmental stationary and on the average might take about 20 minutes of typing for a good secretary. Now if I took that in today, it could be done in 20 minutes and given back to me to mail say within the hour. But in every case I can remember the turnaround time for a letter was about one week. The puzzle was that it took the same amount of time to type the letter 7 days from now as it would today, so why the delay? If it was backlog, there must be a permanent backlog or the return time would be variable not constant.

No secretaries exist today in modern universities and we all type our own letters on the computer, so why is this puzzle relevant? I suggest that it is the same dilemma that exists over referee reviews of submitted manuscripts for scientific journals. To be more specific I sit now waiting for reviews and a decision on a paper submitted 4 months ago. This is not a record I would presume, but I had another paper submitted for which the review took 6 months. Now go back to the Secretary’s Dilemma. If you are to review a paper, you could do it in say 3-4 hours today when it arrives, or put it aside for 4 months. Whatever you decide, it will take you the same amount of time to do the actual review whether now or later. So we need a set of hypotheses to explain this anomalous situation.

First of all we note that some journals like SCIENCE or PNAS will reject your paper within one day, an extreme example of the-journal-is-overrun hypothesis. If they decide to review it, I would guess you will hear something within a week or two. There are some journals that promise a decision within a short time, 2 to 4 weeks for example. These journals threaten their reviewers if they do not act within a short time. But in some cases it still takes a long time to get a decision letter, and this might be another the-editor-is-overrun hypothesis, no matter how fast the reviewers respond. Finally, many journals do not promise anything in timing, and this might be explained by the hypothesis that our-reviewers-are-overrun. This problem in turn can be a side effect of the last problem I can identify, the I-am-too-important-to-review-papers hypothesis, so that reviews fall on a small subset of ecologists rather than more evenly. One can be sympathetic to these situations since it is my observation that everyone is overrun all the time in the modern university. And everyone must publish many papers to gain a position, with many associated issues discussed by Statzner and Resh (2010).

There are some possible solutions. One is to blackball reviewers who take excessive time to return reviews. I imagine many editors do this already. Another relief valve might be to get rid of paper journals and make everything electronic. This should reduce the cost of journals and allow expanded volumes. I gain the impression that many journals have page limits set by the cost structure, so that one receives a note accompanying the review sheet that states that the journal must reject 85% of papers so that only Nobel Prize papers can be accepted. And to rub the whole process in more, some journals make you pay to publish. You do all the work, get the paper ready, and then they want money to publish it. You can see why some people start their own journal (not a solution for the faint hearted).

And finally I cannot pass on this subject without a comment about civilized behaviour on the part of reviewers. Ad hominem attacks, sarcastic remarks, and blanket condemnations have no place in any review. Journal editors should put such reviews in the garbage can. There are a few simple guidelines for reviewers, and they are summarized in a new paper by Al Glen (2014). Please read it, memorize it, and act on it when you are a reviewer.

Glen, A.S. 2014. A new ‘golden rule’ for peer review? Bulletin of the Ecological Society of America 95(4): 431-434.

Statzner, B., and Resh, V.H. 2010. Negative changes in the scientific publication process in ecology: potential causes and consequences. Freshwater Biology 55(12): 2639-2653. doi: 10.1111/j.1365-2427.2010.02484.x.

 

Is Ecology like Economics?

2 Replies

One statement in Thomas Piketty’s book on economics struck me as a possible description of ecology’s development. On page 32 he states:

“To put it bluntly, the discipline of economics has yet to get over its childish passion for mathematics and for purely theoretical and often highly ideological speculation at the expense of historical research and collaboration with the other social sciences. Economists are all too often preoccupied with petty mathematical problems of interest only to themselves. This obsession with mathematics is an easy way of acquiring the appearance of scientificity without having to answer the far more complex questions posed by the world we live in.”

If this is at least a partially correct summary of ecology’s history, we could argue that finally in the last 20 years ecology has begun to analyze the far more complex questions posed by the ecological world. But it does so with a background of oversimplified models, whether verbal or mathematical, that we are continually trying to fit our data into. Square pegs into round holes.

Part of this problem arises from the hierarchy of science in which physics and in particular mathematics are ranked as the ideals of science to which we should all strive. It is another verbal model of the science world constructed after the fact with little attention to the details of how physics and the other hard sciences have actually progressed over the past three centuries.

Sciences also rank high in the public mind when they provide humans with more gadgets and better cars and airplanes, so that technology and science are always confused. Physics led to engineering which led to all our modern gadgets and progress. Biology has assisted medicine in continually improving human health, and natural history has enriched our lives by raising our appreciation of biodiversity. But ecology has provided a less clearly articulated vision for humans with a new list of commandments that seem to inhibit economic ‘progress’. Much of what we find in conservation biology and wildlife management simply states the obvious that humans have made a terrible mess of life on Earth – extinctions, overharvesting, pollution of lakes and the ocean, and invasive weeds among other things. In some sense ecologists are like the priests of old, warning us that God or some spiritual force will punish us if we violate some commandments or regulations. In our case it is the Earth that suffers from poorly thought out human alterations, and, in a nutshell, CO2 is the new god that will indeed guarantee that the end is near. No one really wants to hear or believe this, if we accept the polls taken in North America.

So the bottom line for ecologists should be to concentrate on the complex questions posed by the biological world, and try first to understand the problems and second to suggest some way to solve them. Much easier said than done, as we can see from the current economic mess in what might be a sister science.

Piketty, T. 2014. Capital in the Twenty-First Century. Belknap Press, Harvard University, Boston. 696 pp. ISBN 9780674430006