Cook's survey not only meaningless but also misleading; fails to mention they found more papers rejecting AGW than say humans are primarily responsible

Cooking the books: John Cook's new survey is not only meaningless, but is also highly misleading because it fails to mention that they found more scientific publications whose abstracts reject global warming than say humans are primarily responsible! Only 65 out of a total of 12,271 abstracts endorsed the position that man is primarily responsible for climate change. That would indicate a 0.5% non-consensus on man-made global warming.

Reblogged from De staat van het klimaat:

Cook’s survey not only meaningless but also misleading

DOOR MARCEL CROK OP 17 MAY 2013

The new Cook et al. survey of the scientific literature is attracting worldwide media attention and even the American president tweeted (“Ninety-seven percent of scientists agree:#climate change is real, man-made and dangerous.”) about it. Yesterday my own first reaction on twitter was: “All this talking about consensus http://iopscience.iop.org/1748-9326/8/2/024024/pdf/1748-9326_8_2_024024.pdf … doesn’t improve our understanding of climate even a tiny bit” And I added that “it shows again how meaningless this kind of consensus analysis is”.

Now I still support the label “meaningless” but today I will add “misleading” to the list.

So what’s all the fuzz about? Cook et al. selected around 12,000 scientific abstracts that contained the words “global warming” or “global climate change” published in the period 1991-2011. With a large group of volunteers they then rated the papers using 7 categories. Around 8000 of the abstracts (2/3) take no clear position on Anthropogenic Global Warming (AGW). Of the remaining ~4000 abstracts more than 97% “endorse AGW” according to the paper. Only a tiny amount (78 papers) “reject AGW”. Hence they claim again that there is a consensus, that the debate is over and also that there is a gap between scientists and the public (see graph above). A much larger percentage of the scientists “endorses AGW” than the public at large.

Misleading
Now here comes the misleading part. If an abstract/paper “endorses AGW”, what would this mean for most people? Let’s look again at the tweet of Obama: “#climate change is real, man-made and dangerous”. If this is what it means for the president of the US, it probably means the same for many citizens who heard the news in the media. However, can this be sustantiated by the survey results? In no way.

To the credit of the researchers they made all their results available in a searchable database. Their rating system is online as well. There are 7 levels of endorsement, going from quantified endorsement of AGW all the way down to a quantified rejection of AGW. Seems fair enough. But here  is the issue. Only the first category can be regarded as a real or strong endorsement of AGW. Here is the description of category 1:

1. Explicit Endorsement of AGW with quantification
1.1 Mention that human activity is a dominant influence or has caused most of recent climate change (>50%).
1.2 Endorsing the IPCC without explicitly quantifying doesnt count as explicit endorsement – that would be implicit.

Now specifically look at 1.1. This comes close to the iconic statement from the IPCC AR4 report which said that “Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations.” Now if 97% of the abstracts would repeat in slightly varying terms this major conclusion, than at least the conclusion of the survey would be more or less fair. However the survey doesn’t come even close.

Brandon Shollenberger, who is guest blogger at The Blackboard, was the first who reported that actually only 65 papers have been rated “category 1″. Yes that’s right, only 65 abstracts clearly “mention that human activity is a dominant influence or has caused most of recent climate change (>50%)”. 65 on a total of 12,000 is 0.5%. So a completely fair conclusion from their survey is that only 1 in 200 abstracts explicitly mentioned that humans are dominating climate. If you ignore the 8000 papers that were labelled category 4 (neutral, meaning having no position on AGW) the 65 would be 1.6%. The paper reported that only 78 papers (1.9% if you ignore the 8000 neutral abstracts) rejected AGW.

Now where is the 97% endorsement of AGW coming from? What the authors did is to add up the numbers of category 1 to 3 and of the category 5 to 7 which I show below*:

Category 1: 65
Category 2: 934
Category 3: 2933
Category 4: 8261
Category 5: 53
Category 6: 15
Category 7: 10
Total:         12271

As you can see the 78 “rejection of AGW” abstracts are the added number of category 5-7. Category 1-3 together adds up to 3932 papers. This 3932 divided by 4010 (the total of category 1-3 + 5-7) gives their impressive 97% (with the above numbers it is even 98%, see * for more explanation about how these numbers were found). However of these 3932 abstracts 2933 (75%) fall in category 3. Now how strong is the endorsement of AGW in this category? Here is the description:

3. Implicit Endorsement of AGW
3.1 Mitigation papers that examine GHG emission reduction or carbon sequestration, linking it to climate change
3.2 Climate modelling papers that talks about emission scenarios and subsequent warming or other climate impacts from increased CO2 in the abstract implicitly endorse that GHGs cause warming
3.3 Paleoclimate papers that link CO2 to climate change
3.4 Papers about climate policy (specifically mitigation of GHG emissions) unless they restrict their focus to non-GHG issues like CFC emissions in which case neutral

3.5 Modelling of increased CO2 effect on regional temperature – not explicitly saying global warming but implying warming from CO2
3.6 Endorsement of IPCC findings is usually an implicit endorsement. (updated this so it is more than just reference to IPCC but actual endorsement of IPCC)

I like 3.2: “endorse that GHG’s cause warming”. I also strongly agree with this part of 3.5: “implying warming from CO2″. The meaningless result of their whole exercise is that 75% of the abstracts that say something about AGW at all “link CO2 to climate change” or “imply warming from CO2″.

The misleading part is that they didn’t specify this result in their paper. Nowhere in their paper or in the supplementary material they even mentioned the total numbers in the different categories like I did in the simple table above. They only showed the total of category 1-3 in their figure 1(a):

Even the other >24% of AGW endorsement (based on the 4010) in category 2 is pretty meaningless:

2. Explicit Endorsement of AGW without quantification
2.1 Mention of anthropogenic global warming or anthropogenic climate change as a given fact.
2.2 Mention of increased CO2 leading to higher temperatures without including anthropogenic or reference to human influence/activity relegates to implicit endorsement.

In a comment under his post Shollenberger nicely explains who the different categories skew the result towards “endorsement of AGW”:

To give an example, if we say, “Humans are responsible for 40% of global warming,” that puts us in the bottom category. Change the number to 60%, and suddenly we’re in the top category. But what if we don’t give a number at all? 

If we just say “Humans cause some global warming,” we could be supporting a value 20% or 90%. Despite being able to support either position, we’d land in the top categories. That means the results will automatically be skewed toward the top.

Self confirming Now for anyone who reads climate papers frequently this is totally obvious. Climate scientists have to frame their research in the abstract and there wouldn’t be so many climate papers if there was no concern for CO2.

So the whole result of this survey is completely self confirming. Because there is a concern for CO2 there is a lot of funding of climate science. This then generates a lot of climate science papers (they surveyed 12,000 but mention there are many more). In the abstracts scientists refer to the concern about CO2. The abstract then falls into category 2 or 3 and therefore almost all the papers “endorse AGW”.

* This is how you can replicate the numbers. Go to http://www.skepticalscience.com/tcp.php?t=search and type just a single letter in the search term box (I used “a”). This generates the total of 12,271 abstracts. Then select the whole period 1991-2011 and just search the 7 different categories. This generates the table from the blog post above.