What the IPCC Citizen's Audit really means

15 Apr

Yesterday, the IPCC Citizen’s Audit results were released. They show that out of 18,531 references in the AR4, 5,587 (30%) are not peer-reviewed. These results are making their rounds around the internet, but what do they really mean?

First, I need to state that I was involved in the audit. I was an auditor, and I also helped Donna Laframboise in a few other ways, such as creating a guide to auditing the report. If you’ve ever visited this site in the past, you know that I have looked fairly in depth at the AR4 and found several interesting things, such as the boot cleaning guide, the master’s student story, the master’s students citations, the false wildfire-tourism claim, all the news articles, the issues with the authors falsely citing themselves, and other interesting stuff. I already know that the AR4 isn’t entirely based on peer-reviewed material.

Within only a few hours of this audit being released, some have attacked it by citing the fact that the IPCC does allow ‘gray literature’ to be cited in the AR4. This is true, but they are missing the point. The audit wasn’t being done in order to claim the report isn’t valid because it relies on non-peer-reviewed literature. The audit was done…..well, let’s read what Donna wrote (emphasis mine):

The IPCC chairman has declared that non-peer-reviewed research sources belong in the dustbin (see the last lines of this newspaper article) but this project does not necessarily take that position. Its primary goal was to determine whether the chairman’s claim (frequently repeated by journalists) that this report is based only and solely on peer-reviewed literature is accurate.

She also says here:

The chairman of the IPCC has repeatedly said the report relies solely on peer-reviewed literature to support its findings. He has said research that hasn’t appeared in peer-reviewed journals should be thrown “into the dustbin” (see the last line of this newspaper article). But our audit has discovered almost 5,600 non-peer-reviewed references in this report.

Just in case you don’t understand yet, one of her main bullet points is here:

IPCC chairman’s claim that the report relies solely on peer-reviewed sources is not supported

And in her original blog post asking for volunteers, she said:

Recent examinations of two random chapters found only 25 percent and 58 percent of the sources cited were peer-reviewed journal articles.

These numbers conflict sharply with declarations by IPCC chairman, Rajendra Pachauri, that only peer-reviewed literature is relied on. Are these two chapters unrepresentative outliers? It is important that we find out. This, therefore, is a call for volunteers.

The audit was done in order to expose the false claims of Pachauri and the media that the IPCC relies solely on peer-reviewed literature. That claim can now definitively be put to rest, because of the excellent work of Donna and others.

I’m not saying that the audit doesn’t show us something more about the IPCC AR4. You can determine its value yourself, especially because it was done so openly. You can look at every individual audit of every chapter yourself. This is an excellent resource for those who are either skeptical of the audit results, and wish to verify them, or for those who want to dig deeper into the references themselves. Transparency was very important in this project. Make your own conclusions.

The findings of this project cannot be dismissed simply because the AR4 allows for ‘gray literature’. As the above quotes indicate, that wasn’t the point of the audit. This audit has dealt the final blow to the claim that the IPCC relies solely on peer-reviewed literature. As I said when the audit started:

This project should be welcomed from all sides of the climate debate. Supporters of the IPCC should be glad to have their claims vindicated, detractors of the IPCC should determine if their criticisms have been valid, and those on the fence will be able to find out for themselves who has made the false claims.

The results are in: Chairman Pachauri and media outlets have made the false claims.


Posted by on April 15, 2010 in Uncategorized


Tags: ,

4 responses to “What the IPCC Citizen's Audit really means

  1. Donna Laframboise

    April 15, 2010 at 1:25 pm

    Thanks for this post. It is an astonishing thing that the chairman of one of the most important organizations in the world can go around describing his own report inaccurately and

    a) he still keeps his job
    b) we’re still expected to believe him
    c) we’re still expected to believe in his organization

    Surely reasonable people can agree that there’s a huge credibility problem with the IPCC at the moment – whatever one’s other views on climate might be.

  2. JohnWho

    April 15, 2010 at 5:10 pm

    Donna –

    that is exactly the way Politics works, which is why he keeps his job.

    If this was actually science, he’d be gone a long time ago.

  3. Todd

    April 21, 2010 at 10:41 am

    Why did you and all these other people perform this extensive audit just to prove that Pachauri and other media outlets were wrong when you could just pointed out Annex 2 in the IPCC Procedures (the part that allows for non peer reviewed literature)? What a waste of time.

  4. Sam

    April 21, 2010 at 10:31 pm


    Pointing to a policy that allows for gray literature doesn’t prove gray literature was used.

    Pointing to an audit done of every single chapter in the report, which shows nearly a third of references were gray, does prove it.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: