IPCC first makes claim, then adds reference later

11 Mar

In other posts I have claimed that the IPCC process appears to be along these lines: Make a claim, then find a citation. While it was fairly obvious they did this often, I hadn’t found a slam-dunk case I could point at to prove it. I now have that case.

I am participating in the Audit of the IPCC’s AR4 conducted over at (I recommend you spare an hour or two to help if you can). I was given Chapter 12 of WG III, which I’ve completed. Afterwords, I decided to look at the expert reviewers comments for my chapter. I found quite a few biased remarks, which will make another post coming soon. I also found one particular comment interesting. Comment 12-189 (page 78):

Seems a bold claim – can you substantiate it? ENGOs in most cases are supported through financial contributions from individuals as well as from foundation funding. They are also governed through Boards that in principle represent their consituency. If you leave this, you need to provide a citation.
(Jan Corfee-Morlot, University College London (on
leave from OECD))

Some background is necessary. In the Second Order Draft, here is the claim he is referring to (page 43, lines 34-36):

Relatively few NGOs are directly accountable to members in the same way that governments are to voters or businesses are to shareholders, raising further questions about the extent to which their claims to the mantle of civil society are justified.

Notice this claim has no reference. So, the reviewer thinks that this claim is bold, and seems to imply it is false. He ends “If you leave this, you need to provide a citation.” So what does the reviewer say?

Noted. Will look for references.

There we go. There is no talk of changing the claim or removing it, he simply says he will look for references. Again, let me clarify what is happening here. The author made a claim, it was challenged, then he says ‘alright, now I will go find literature to support what I’ve already decided.’ See the problem here?

Maybe you will point out that the claim does indeed have a citation (now), so as long as the source supports the claim, who cares? Let’s look at the original claim again:

Relatively few NGOs are directly accountable to members in the same way that governments are to voters or businesses are to shareholders, raising further questions about the extent to which their claims to the mantle of civil society are justified.

And now here is the same section in the final report:

Relatively few NGOs are directly accountable to members in the same way that governments are to voters or businesses are to shareholders, raising further questions about the extent to which their claims to the mantle of civil society are justified (Najam, 1996).

That’s right, they are the exact same, except for the (Najam, 1996) tacked on the end. Let’s look at the reference:

Najam, A., 1996: Understanding the Third Sector: Revisiting the prince, the merchant and the citizen. Nonprofit Management and Leadership, 7(2), pp. 203-19.

I didn’t want to pay for the full article, but I found an abstract:

An attempt to provide a definitive explanation of the nonprofit, charitable, or philanthropic third sector, defining its boundaries & growing influence in local, national, & international politics. A typology of third-sector organizations highlights the difficulty in arriving at a common definition. Social science & other disciplines have tended to describe the rationale of the third sector as a response to catastrophes, eg, market or government failure. It is argued here that the third sector should be viewed as equal to other sectors of society rather than dependent on them. The imagery of the prince (political society), the merchant (market), & the citizen (civil) is used to represent three sectors of power in society. In this model, the rationale for the third sector comes from demands arising from the civil sector to provide services, empower the underprivileged, & advocate social change. It is noted, however, that many service or contracting agencies called nonprofits for taxation purposes are better understood as market organizations. 1 Exhibit, 59 References. Adapted from the source document.

If someone wants to buy this paper (or donate to help me afford it), or can find it for free let me know. I’d be interested in knowing if the full text does support their unaltered claim. My guess is, based on the abstract, they are only loosely related.

This claim is not based in any literature. It is based on whichever author at the IPCC wrote it. When this was pointed out by a reviewer, the solution was to simply find and tack on a paper that appears to only be related in the subject matter. This is shoddy, sloppy, and biased work.



Posted by on March 11, 2010 in IPCC



6 responses to “IPCC first makes claim, then adds reference later

  1. ben

    March 11, 2010 at 1:11 pm

    Hmmm, interesting point. My reading is different. I have done a lot of literature reviewing in my time, and one thing that can happen is that you read a lot of stuff, then write it up. Not every point you make gets a reference, and it is only in the review process where bits that really do need a reference get highlighted.

    By this time, you know that point you made is somewhere in the literature you’ve read, and you’re confident what you wrote fairly represents what you read – but where is it? I’ve had times when I simply haven’t been able to recover the reference and had to delete the claim, which is usually pretty frustrating.

    There is, however, no excuse at all for passing off references as saying something they do not – if that is what has happened.

  2. ben

    March 11, 2010 at 1:23 pm

    I have the paper if you want it, mail me benwinters22 at

  3. hro001

    March 12, 2010 at 6:01 pm

    This certainly isn’t the only problem with this polluted “peer review” process. There have been a number of documented instances in which the responses to the Reviewer Comments by the “chapter team” would seem to indicate that if the Reviewer has not followed the party-line, his/her comments are simply rejected – or at best “partially accepted” without any indication of which “part” and/or don’t make it into the Final version regardless of “acceptance”.

    For an example of this “process” in action, you might want to take a look at an analysis I did to Briffa’s responses to comments in the Second Order Draft of WG 1, Chapter 6, in particular the paragraph in which the infamous “hockey-stick” was discussed:

  4. Marion Morrison

    March 13, 2010 at 7:48 pm

    This is a fairly typical response from the denier camp – pick a relatively unimportant comment – one which shouldn’t really need a “citation” for the sort of observation that seems fairly apparent (do you really need an expert to tell you that NGOs do not respond in the same what that democratic entities do?). Probably the author had read about the subject beforehand and did not anticipate the level of paranoid, politically driven pedantry encountered in the denosphere. I did notice that Climategate did not actually read the article in question, and I did notice that, in fact, the original author was able to provide a citation.

    Dead end, folks. Pointless too.

  5. Baa Humbug

    March 15, 2010 at 5:18 am


    At your link (fascinating read) I noticed you took a few screen shots.
    I downloaded (took hours) Chapter 1 and chapter 9 1st and 2nd Order drafts as well as the reviewer comments. Plus the Govt expert review of SPM 1st 2nd and final drafts.
    if you wish to have copies, my email is

    I plan on downloading other chapters when I get time.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: