Yesterday, the IPCC Citizen’s Audit results were released. They show that out of 18,531 references in the AR4, 5,587 (30%) are not peer-reviewed. These results are making their rounds around the internet, but what do they really mean?
First, I need to state that I was involved in the audit. I was an auditor, and I also helped Donna Laframboise in a few other ways, such as creating a guide to auditing the report. If you’ve ever visited this site in the past, you know that I have looked fairly in depth at the AR4 and found several interesting things, such as the boot cleaning guide, the master’s student story, the master’s students citations, the false wildfire-tourism claim, all the news articles, the issues with the authors falsely citing themselves, and other interesting stuff. I already know that the AR4 isn’t entirely based on peer-reviewed material.
Within only a few hours of this audit being released, some have attacked it by citing the fact that the IPCC does allow ‘gray literature’ to be cited in the AR4. This is true, but they are missing the point. The audit wasn’t being done in order to claim the report isn’t valid because it relies on non-peer-reviewed literature. The audit was done…..well, let’s read what Donna wrote (emphasis mine):
The IPCC chairman has declared that non-peer-reviewed research sources belong in the dustbin (see the last lines of this newspaper article) but this project does not necessarily take that position. Its primary goal was to determine whether the chairman’s claim (frequently repeated by journalists) that this report is based only and solely on peer-reviewed literature is accurate.
She also says here:
The chairman of the IPCC has repeatedly said the report relies solely on peer-reviewed literature to support its findings. He has said research that hasn’t appeared in peer-reviewed journals should be thrown “into the dustbin” (see the last line of this newspaper article). But our audit has discovered almost 5,600 non-peer-reviewed references in this report.
Just in case you don’t understand yet, one of her main bullet points is here:
IPCC chairman’s claim that the report relies solely on peer-reviewed sources is not supported
And in her original blog post asking for volunteers, she said:
Recent examinations of two random chapters found only 25 percent and 58 percent of the sources cited were peer-reviewed journal articles.
These numbers conflict sharply with declarations by IPCC chairman, Rajendra Pachauri, that only peer-reviewed literature is relied on. Are these two chapters unrepresentative outliers? It is important that we find out. This, therefore, is a call for volunteers.
The audit was done in order to expose the false claims of Pachauri and the media that the IPCC relies solely on peer-reviewed literature. That claim can now definitively be put to rest, because of the excellent work of Donna and others.
I’m not saying that the audit doesn’t show us something more about the IPCC AR4. You can determine its value yourself, especially because it was done so openly. You can look at every individual audit of every chapter yourself. This is an excellent resource for those who are either skeptical of the audit results, and wish to verify them, or for those who want to dig deeper into the references themselves. Transparency was very important in this project. Make your own conclusions.
The findings of this project cannot be dismissed simply because the AR4 allows for ‘gray literature’. As the above quotes indicate, that wasn’t the point of the audit. This audit has dealt the final blow to the claim that the IPCC relies solely on peer-reviewed literature. As I said when the audit started:
This project should be welcomed from all sides of the climate debate. Supporters of the IPCC should be glad to have their claims vindicated, detractors of the IPCC should determine if their criticisms have been valid, and those on the fence will be able to find out for themselves who has made the false claims.
The results are in: Chairman Pachauri and media outlets have made the false claims.