Study claiming '97% of climate scientists agree' is flawed

Perhaps the most common argument used when urging action on climate change is the appeal to scientific authority. Previously this was accomplished by pointing at the IPCC, but since they have lost a significant portion of their credibility recently it has become more frequent to point out the scientists themselves. The most common claim that I encounter is a variation on this claim:

97% of climate experts agree humans are causing global warming.

I recently heard this claim on my own threads. I looked at the source (the study Doran and Zimmerman 2009), found some problems, and then wrote back on my threads. However, I have seen this claim so many times that I believe it would be good to make a post about it. I also e-mailed several prominent climate scientists who would be considered ‘skeptics’ to get their opinions on the study. Their responses are displayed at the end of the post. In this post I briefly comment on past responses to the study, then break my post into three sections. The first will focus on the flaw in the study (the second question), the second will look at the motives of the researcher, and the third will be posting responses from prominent ‘skeptical’ climate scientists.

First I’m going to address a common response to this study. In this post at The Hockey Schtick, it is pointed out that the 97% statistic is based on only 79 climatologists, and that those participating were self-selected. There are two concerns here. The first is sample size. While climate science isn’t a massive field, 79 participants is fairly small. To claim definitely that 97% believe this or that you would need to poll significantly more people. The second concern is the fact that the scientists were self-selected by an online survey. This may not have led to a representative sample.

Other concerns with the study deal with numbers behind it, or other reasons to consider it a poor study. However, these aren’t my primary concern. My concern is the actual questions asked in the study, which I will show in a moment.

The study on which these claims are based is available here. It is an paper by Peter Doran and Maggie Kendall Zimmerman written in 2009, entitled “Examining the Scientific Consensus on Climate Change”. Here is the citation:

Doran, P. T., and M. Kendall Zimmerman (2009), Examining the Scientific Consensus on Climate Change, Eos Trans. AGU, 90(3)

The questions

The study is fairly simple. It has a large database of earth scientists, and sends them an invitation to participate in their study. If they accept, then they take an online survey. The survey asks two primary questions:

1. When compared with pre-1800s levels, do you think that mean global temperatures have generally risen, fallen, or remained relatively constant?

2. Do you think human activity is a significant contributing factor in changing mean global temperatures?

The first question is largely irrelevant. I’m unaware of any scientists who don’t believe the planet has gotten warmer when compared with pre-1800s levels. Not surprisingly, 76 of 79 climate scientists answered ‘risen’ to this question. I’m guessing that the other three didn’t consider the increase significant enough to warrant ‘risen’ and picked ‘constant’.

The major problem with this study is the second question. It is not phrased properly. In fact, the phrasing is so poor that I consider the entire study flawed because of it. There are multiple problems with the phrasing, so let me break them down.

1. The phrase “human activity”

Human activity comprises numerous actions which can affect the climate other than greenhouse gases. Agricultural changes and deforestation are two influences that come to mind. Now, any respondent who believes that ANY human activity can change the climate must answer yes to this question.

A better phrasing would be:

Do you think anthropogenic greenhouse gas emissions are a significant contributing factor in changing mean global temperatures?

2. The phrase “significant contributing factor”

The problem with this is obvious. What makes something significant? If 5% of recent temperature change is caused by mankind, is that significant? How about 10%? There is no context for answering the question. There is no way of knowing whether or not the respondents consider human activity the primary factor in temperature change.

A better phrasing would be:

Do you think that anthropogenic greenhouse gas emissions are the primary factor (50% or more) in changing mean global temperatures?

3. The phrase “changing mean global temperature”

This is the most problematic part of the question, because there is no indication of how much temperature change is considered worth answering ‘yes’. For example, if a respondent believed that human activities had increased the temperature of the planet by 1/10th of a degree, the answer would still be yes. Even so for 1/100th. There is no useful context here. Many climate skeptics believe that human activities have increased the temperature of the planet, but not by any significant amount. The survey should specifically ask if the warming is a statistically significant amount. Also, the word “changing” should be changed to “increasing”, because otherwise a respondent could consider human activities as cooling the planet and still answer yes.

Actually, I’d change the ending altogether. In my mind, the following is a much better question:

Do you believe that anthropogenic greenhouse gas emissions were the primary factor (50% or more) in the observed mean global temperature increase since the mid-20’th century?

I believe this question is more appropriate then theirs to determine if a consensus exists among climate scientists regarding AGW. My guess is less than 97% would agree with this statement. But I didn’t create the survey.

The second question they ask isn’t able to determine if climate scientists believe that GHGs have caused significant global warming. The poor phrasing and the ambiguity of certain essential terms renders the entire study useless. All it proves is that 97% climatologists agree that the planet has warmed since 1800, and that human activity has caused an unspecified amount of that warming. Let me be very clear and break out the bold:

The survey does not ask climate scientists if they believe global warming is primarily driven by human activity. Because of this, the survey responses cannot answer this question.

Instead of asking if human activity is the primary driver, they asked if it was a “significant contributing factor”, which is completely subjective. A respondent could believe that 10% of current warming is due to human activities, and might consider this significant enough to answer yes to the question.

Another problem with citing this study is that is only asks the opinion of scientists about whether or not global warming is occurring and humans are having an impact, but it does not address whether or not scientists are concerned about climate change. The study is being used to urge action, yet it makes no claims about whether or not scientists are urging action. This isn’t a problem with the study itself, only with how AGW proponents are using it.

Motives of the researcher

There is another aspect to this study which hasn’t been mentioned; the motives of the researcher. I think this is a valid question, especially considering that climate skeptics are constantly having their motives questioned. Anytime a skeptical paper is published the accusations of ‘big-oil funding’ start to fly, implying that the researchers have impure motives. What was the motivation behind this study? Let’s listen to the researcher himself. In combination with this press release, Peter Doran gave an interview on January 19th, 2009 to a University of Illinois at Chicago news program called Research News. It is available here. There is no transcript so I’ve written a section myself. It starts about 3/4’s of the way through the interview:

Some people have asked me since this paper came out, “Barack Obama is in office now, the Democrats are in control, do we have to worry about this anymore?”, and the answer is yes, because the general public is still about 50% convinced that global warming is an issue that’s real, let alone do we have to do something about it. And so the public needs convincing, and also, there are still people in government that need convincing. As recently as December there was a senate minority report put forward that said exactly the opposite of what our paper said, and was trying to convince people in the senate that scientists don’t agree on global warming. So there is a still a battle, if you will, to be fought here, and I hope our paper pushes the numbers towards more people believing that global warming is a reality. I think if people don’t believe that scientists agree then they can use that as an excuse for inaction, and that’s a dangerous thing.

Clearly this researcher wants his paper to change the public’s mind and politicians’ minds about the scientific consensus. I am not claiming that his paper is invalid solely because of this reason, but I do want to make the motivations of this particular study clear: the study is intended to convince the public, and politicians, that global warming is real and we have no excuse for inaction.

Response from climate scientists

In looking at the phrasing of the survey questions, I kept thinking to myself “I’m fairly certain that most climate scientists would agree with these even if they aren’t concerned about future climate change.” In order to verify if this were actually true, I e-mailed multiple ‘skeptical’ climate scientists. I included the two survey questions, and then asked the two following questions to them:

1. What are your answers to these two questions?

2. Do you believe the second question is phrased correctly to determine if climatologists consider AGW a concern?

I asked the first because I was curious to see if they did agree with the survey questions even though they are not considered part of the consensus opinion. I asked the second question to see if they agreed with my assessment that the phrasing of the second question was poor. Here are their responses.

Patrick Michaels

Yes and yes. The second question is phrased precisely to NOT determine whether or not the respondent feels it is a pressing concern.

Anyone with experience in survey development (and I know people who do this) would recognize the hidden motive here. It is telling that such a paper would be accepted with such poor design and such a foregone conclusion.

Pat Michaels

Richard Lindzen

As you know, polling is a dicey business. With respect to your first question, my answer to (1) is probably, but the amount is surprisingly small — suggesting that global mean temperature anomaly is not a particularly good index. My answer to (2) would be yes, but dependent on what is meant by significant. As to your second question, I agree that one can answer yes without any implication of alarm. Remember, according to the IPCC, we have already reached a level of radiative forcing that is almost as large as one would expect from a doubling of CO2. Even if climate sensitivity were 0.5C (which is generally considered to be of no concern) we would still be making a significant contribution to the small observed ‘warming.’

John Christy

1. What are your answers to these two questions?

Generally temperatures have risen from the little-ice age minimum in the 19th century to the present.

No one knows how much of this warming is due to human effects. In my opinion, most of the warming since the 19th century is due to natural variations.

2. Do you believe the second question is phrased correctly to determine if climatologists consider AGW a concern?

It was not phrased properly. For example someone might think that 10 percent of any warming constitutes a “significant” contribution, and so would answer yes to that question, even though the proportion of warming due to any human effect might in fact be small.

Bob Carter

Both the questions that you report from Doran’s study are (scientifically) meaningless because they ask what people “think”. Science is not about opinion but about factual or experimental testing of hypotheses – in this case the hypothesis that dangerous global warming is caused by human carbon dioxide emissions.


When tested against empirical data, this global warming hypothesis fails. For example, there has been no increase in global temperature for more than 10 years despite an increase in atmospheric carbon dioxide of more than 5%.


Rephrasing them appropriately, the scientific answers to the two questions are therefore


“1. When compared with pre-1800s levels, have mean global temperatures generally risen, fallen, or remained relatively constant?”


The answer depends (a) on what dataset you use (MSU satellites, ground thermometers, radiosondes), (b) the ways in which you plot and/or average the data points, and (c) the precise choice of endpoints.


For all datasets, however, a true statement is that there has been no significant (i.e. within the bounds of error) global warming since 1998, and some of the datasets demonstrate cooling.


“2. Is human activity a significant contributing factor in changing mean global temperatures?”


It is unchallenged that human activity has an effect (in different places either cooling or warming) on local and regional temperatures, not least as a result of land-use changes. When averaged across the globe, however, the net human effect on the global average temperature statistic is indeterminable, presumably because it is so small that it is lost in the noise of natural variation.


In addition, because carbon dioxide is a greenhouse gas and because the gas mixes globally in the atmosphere, human emissions must exert a prima facie global warming effect. In actuality, both positive and negative feedbacks then occur, which are poorly known, so there is an ongoing debate as to the magnitude of the net human greenhouse effect (the climate sensitivity issue). In any case, and again, the empirical data fail to demonstrate an unequivocal warming trend of human origin against the background of natural climate variation.


Therefore, the null hypothesis that the temperature changes that have been measured since the advent of thermometers are natural remains unchallenged. The onus of providing substantive evidence for a dangerous human-caused greenhouse effect therefore rests with the proponents of that hypothesis, and to date they have failed utterly to provide it, basing their arguments instead on speculative deterministic computer models that are known to be inaccurate.


[Dr. Carter also mentioned that he addresses these issues in more detail in his recent book on pp. 38-70, available here].

I’d like to thank these scientists for responding to my questions. I appreciate it.

On the whole, I would say that these four climatologists agree with my assessment of this study. Patrick Michaels believes the study’s results were a foregone conclusion, Richard Lindzen points out that even at a very low climate sensitivity the second answer is yes, John Christy agrees that the second question was phrased incorrectly, and Bob Carter admits that human activity has changed climate but asserts that it is too small to even ascertain. If scientists such as Patrick Michaels or Richard Lindzen, often called ‘deniers’, can agree with the survey questions asked, how can the study claim to prove any consensus? This study is seriously flawed.

Conclusion

This survey should not be cited as evidence that a consensus exists among climate scientists regarding AGW. This is due to the fact that it does not ask the scientists if human activities are the primary cause of increasing temperatures. The questions asked only pertained to ascertaining whether or not climate scientists agree that the earth has warmed and humans have played any role, and it did a poor job at ascertaining these facts as well. Anyone using this study to claim that 97% of climate scientists agree that humans are the primary cause of global warming is ignoring the ambiguous and poor phrasing of this survey questions. The survey does not ask if global warming is primarily driven by human activity, so the survey responses cannot answer this question.

[wp_campaign_2]