Tuesday, May 21, 2013

The Lewandowsky Papers

The Lewandowsky Papers

This essay was written for Spiked-Online, and will be published on Spiked at some point.

As the influence of environmental thinking has increased its hold over the political establishment, the failure to win the public support that might create the basis for decisive action to save the planet has also increasingly been blamed on climate sceptics operating on the internet. On this view, bloggers have thwarted international and domestic action to prevent climate change. Accordingly, the nature of the blogosphere and the workings of the minds of climate sceptics have become the focus of academic research, just as the mechanics of the climate system have been the subject of climate scientists. But this attempt to form a pathological view of a complex debate says much more about the researchers than the objects of their study.

A recent study by academic psychologist, Stephan Lewandowsky at the University of Western Australia aimed to identify climate change sceptics’ tendency towards conspiracy theories. According to Lewandowsky’s paper published in Psychological Science, links were placed on climate blogs, inviting readers to take part in a survey that measured each respondents’ political orientation, attitude to climate change science, and adherence to popular conspiracy theories such as those about the deaths of JFK, Martin Luther King and Diana, the 911 attacks, aliens in Roswell, and the 1969 moon landing. Hence, the title of the paper NASA faked the moon landing — Therefore (Climate) Science is a Hoax: An Anatomy of the Motivated Rejection of Science, which claimed to find a correlation between belief in the principle of a free-market, rejection of climate change science and ‘conspiracy ideation’.

However, in spite of the title of the paper linking the idea that the moon landings were faked with scepticism of climate science, just 10 of 1145 respondents either ‘agreed’ or ‘strongly agreed’ with the statement ‘The Apollo moon landings never happened and were staged in a Hollywood film studio’. Of those 10, six of them either ‘agreed’ or ‘strongly agreed’ with statements representing the scientific consensus on climate change. Of the remaining four who disagreed with the climate consensus, two ‘strongly agreed’ with every conspiracy theory, another agreed either ‘strongly’ or ‘very strongly’ with each of the 14 conspiracy theories listed in survey. The single remaining climate sceptic either genuinely believed that the moon landings were faked, or was another person seeking to game the survey’s results with a more sophisticated approach than his colleagues’.

A title that better reflected the survey’s results, then, might have been NASA faked the moon landing — Therefore (Climate) Science is Real. But the problems with the study don’t stop with the researcher’s haste to give his work a compelling name.
The paper, now known as LOG12, was published on Lewandowsky’s website last July, well ahead of its publication in the journalPsychological Science in March. In spite of not yet having been published,  a ‘pre-press’ version drew a great deal of attention from people across the climate debate. For environmentalists it was proof that their counterparts were textbook nut-jobs. For sceptics, it was a poorly-conceived and improperly-executed smear job with only superficial academic credibility. The paper was circulated by the green media and blogosphere and taken apart by sceptics on the web.

Many problems with the paper emerged. The authors had claimed to have asked the operators of blogs popular with people at different ends of the climate debate to invite their readers to take part in the survey. As the research admits, none of the sceptic blogs responded, leading to the criticism that sceptics had not in fact been asked to participate. More fatally for the paper, it seemed that the researchers had let slip their intention to link climate scepticism with a tendency to produce conspiracy theories, and some blog commenters openly discussed their intentions to respond as sceptics to influence the outcome. Even worse, the paper drew criticism from Lewandowsky’s own colleagues.

Lewandowsky amplified these problems by refusing to answer questions about the survey, choosing instead to write a series of blog posts attacking his critics who were trying to understand his method and analysis. Through the acrimony, it emerged that Lewandowsky had anticipated that as he was well known for his own hostility to climate scepticism, sceptical bloggers would not be willing to participate in his survey. In order to overcome the problem, he asked the University’s ethics committee for permission to send the invitation from other researchers. However, though these emails were found, they had been ignored by the blog owners as the kind of spam bloggers often get, and were discarded, meaning that, nonetheless, sceptics hadn’t really been invited to participate.

Consequently, only 127 people that could be described as sceptics of climate science responded to the survey against the 1018 respondents that Lewandowsky categorises as ‘pro-science’. This should strike us immediately as a ridiculous starting point for an investigation. We would not survey the users of a Star Wars fan website to find out why some people don’t like science fiction movies. Nonetheless, from just 127 suspect responses, Lewandowsky proceeded to make statements about the phenomenon of climate scepticism.

But worse than Lewandowsky’s approach to gathering data is his method for analysing it. Whereas most surveys give results than can be understood fairly simply — eight out of ten cats prefer… — Lewandowsky’s approach was more complicated than his task warranted. And whereas we might expect significant differences between different groups of people to show up when their responses to questions are averaged, in fact few differences between the groups emerge when the data from the surveys is analysed through simpler methods than those deployed by Lewandowsky.

For instance, if we divide the respondents into ‘sceptics’ and ‘warmists’ on the basis of their assent to/dissent from the statement, ‘I believe that burning fossil fuels increases atmospheric temperature to some measurable degree’, and then compare those groups’ assent to/dissent from popular conspiracy theories, we get the following result:

As this table demonstrates, the average (and on this test, it must be noted, fairly militant) sceptic is not much more prone towards conspiracy theories than his putatively ‘pro-science’ counterpart. In fact, the ‘warmist’ is more likely (though only just) to buy into conspiracy theories relating to the 911 and Oklahoma attacks and the assassination of Martin Luther King. Both groups are broadly inclined away from conspiracy theories, with the exception being the only positive assent to the conspiracy theory that — ‘The claim that the climate is changing due to emissions from fossil fuels is a hoax perpetrated by corrupt scientists who wish to spend more taxpayer money on climate research’.

Similarly, a graph showing sceptics’ attitudes to consensus science reveals that not much separates the two groups.

As we can see, and as we might expect, sceptics and warmists only really disagree on matters relating to climate science.
The third variable that Lewandowsky sought to relate to sceptics’ attitude to climate science and conspiracy theories was their views on the economy. (In some of these statements, the scoring was reversed, so that 1 = agreement, 4 = disagreement, denoted by ‘R’.) Here we see slightly more disagreement.

However, this part of the survey asks questions that, in the main, in fact ask the respondent to prioritise the economy versus the environment, rather than to express his views on the economy independently of his views on the environment. This is a problem for Lewandowsky’s subsequent claim that a person’s views on the economy can ‘predict’ his view on the environment. Yet nonetheless, the survey results simply do not suggest that a big difference exists between sceptics and warmists. The average sceptic is, after all, with a score of 2.51, precisely undecided about whether or not ‘I support the free market system but not at the expense of the environmental quality’. Meanwhile, if these statements really reflect opposite ends of some kind of eco-economic axis, we would expect people who are ‘pro-science’ (i.e. ‘warmists’) to dissent more strongly from the statement ‘Free and unregulated markets pose important threats to sustainable development’.

Across these three groups of statements, there is very little disagreement on much other than on the environment. Yet Lewandowsky claimed that ‘Rejection of climate science was strongly associated with endorsement of a laissez-faire view of unregulated free markets‘, and that ‘A second variable that was associated with rejection of climate science as well as other scientific propositions was conspiracist ideation‘.

It seems fairly obvious that Lewandowsky was at best mistaken. The data simply do not support his conclusions. His claims differ remarkably from what the difference between an averaged profile of a ‘sceptic’ and his counterpart look like.
What produced Lewandowsky’s result is a statistical technique called structural-equation modelling (SEM). SEM is a complex process used mainly in the social and behavioural sciences to test and explore assumptions about relationships in observational data. Though this may be an appropriate tool in some cases, its usefulness to the job of shedding light on what people think about complex political and scientific issues is debateable, and may in fact reveal more about Lewandowsky than sceptics. First, there is the problem of Lewandowsky’s assumptions: he confuses categories of economic and environmental ideas; he fails to test for conspiracy theories that might be more coincident with environmentalism than the ones he chose, (for example the idea that climate scepticism is a phenomenon produced by covert PR operations, paid for by fossil fuel companies); and he fails to achieve a robust definition of climate scepticism. Second, a much more simple method — averaging — show that, no matter what the slight statistical tendency of sceptics is, on aggregate, there is no clear line dividing lunatic sceptics from the enlightened climate scientists and their disciples. The more complex method, though, has the virtue of being a larger fig leaf for Lewandowsky’s bad faith.

This would not have been Lewandowsky’s only experiment with statistics-abuse. In June last year, using what he called ‘simple mathematics’, but which in fact depended on Bayesian statistical methods, he made a series of extraordinary claims, concluding that,‘greater uncertainty about the evolution of the climate should give us even greater cause for concern‘, that ‘greater uncertainty means that things could be worse than we thought’ and that ‘greater uncertainty means that the expected damages from climate change will necessarily be greater than anticipated’. This astonishing claim barely needs unpacking to demonstrate as so much nonsense cloaked in mathematical jargon.

This should raise questions about exactly what it is that Lewandowsky is trying to achieve through the use of dubious statistical methods, and his attempt to understand the minds of climate sceptics: is the point to advance understanding and knowledge, or is it a strategic move in a political battle that there can be no doubt he has taken a stand in.
But to suggest that either bad faith or incompetence has driven Lewandowsky would be, on his view, a conspiracy theory. This was his claim in a subsequent paper published in March by open access journal, Frontiers in Personality Science and Individual Differences, in which Lewandowsky and colleagues attempted to draw conclusions about responses from the climate sceptic blogosphere to his previous paper, LOG12.

In the paper, Recursive fury: Conspiracist ideation in the blogosphere in response to research on conspiracist ideation, Lewandowsky claimed to, ‘identify and trace the hypotheses that emerged in response to LOG12 and that questioned the validity of the paper’s conclusions. Using established criteria to identify conspiracist ideation, we show that many of the hypotheses exhibited conspiratorial content and counterfactual thinking.’

This involved compiling a database of the criticisms made against the first paper and categorising them. For example, one such comment posted by Richard Betts at the Bishop Hill website run by author of the Hockey Stick Illusion, Andrew Montford read as follows:
The thing I don’t understand is, why didn’t they just make a post on sceptic blogs themselves, rather than approaching blog owners. They could have posted as a Discussion topic here at Bishop Hill without even asking the host, and I very much doubt that [Montford] would have removed it. Climate Audit also has very light-touch moderation and I doubt whether Steve McIntyre would have removed such an unsolicited post. Same probably goes for many of the sceptic blogs, in my experience. So it does appear to that they didn’t try very hard to solicit views from the climate sceptic community.

This comment was put into a table with about 110 others that Lewandowsky et al reckoned to be evidence that their authors ‘Espouse conspiracy theories’. Each of the comments were put under different category of ‘conspiracy theory’, such as ‘didn’t email deniers’,‘Warmists faked data’, and ‘Emailed warmists before deniers’. However, this raises the problem, much as with the previous study, in the way categories are defined before they are tabulated and analysed.

If Betts’s comment is evidence of climate sceptics doing ‘conspiracy theory ideation’, then the test for it is set very low indeed — the comment was a straightforward criticism of Lewandowsky’s attempt to gather data, not speculation about why he had taken such liberties. Saying that Lewandowsky’s attempts to get responses from sceptics was inadequate is nothing like saying that the CIA killed Martin Luther King. Bogus categories and a seemingly objective method allowed Lewandowsky’s prejudices to prevail — a statistical technique serving as a fig leaf, again.

Even more unfortunate for Lewandowsky, however, the comment in question did not belong to a climate change sceptic at all. Richard Betts is a climate scientist, an IPCC lead author, and head of climate impacts research at the UK Meteorological Office.

The fact that Lewandowsky could put Betts into a category of ‘climate sceptic’, and his comment into a category of ‘conspiracy theory’ should be an object lesson about letting prejudice influence research for those seeking to understand and explain the climate debate. Lewandowsky’s confusion is owed to the fact of Betts’s very different and unusual approach to overcoming the climate debate: actually having the debate. Betts was able to criticise Lewandowsky after taking the opinions and discussion between climate sceptics seriously, rather than by taking it for granted that they were wrong.

Lewandowsky worked from his prejudice — that all sceptics are, a priori, wrong. His objective was to expose the ‘motivated reasoning’ that lies behind climate scepticism. But in doing so, he managed only to expose his own bad faith. This raises serious questions, not only about the categories and perceptions of ‘climate scepticism’ that dominate the language of anti-scepticism, but also broader questions about how such naked politicking can be passed off as academic research.
Across three papers based on the same data (a third appearing in the journal, Nature Climate Change), Lewandowsky, either unwittingly or deliberately allowed his prejudices to form the basis from which his study proceeded. On his view, climate scepticism is a ‘rejection of climate science’, which sits in contrast to ‘pro science’ opinion. But debunking this claim is easy. We can find climate scientists who give lower estimates of climate’s sensitivity to CO2 whose arguments are better grounded in science than any number of eco-warriors whose arguments are irrational, emotional, and lack any sense of proportion. One can either believe or disbelieve in the idea of anthropogenic climate change independently of science.

Second, Lewandowsky wanted to claim that this rejection of science is ‘motivated reasoning’ — that something else prefigures in the minds of sceptics, that causes them to reject climate science. This too, is easily countered, not just with a more straightforward analysis of Lewandowsky’s own data, which shows otherwise, but also by questioning this conception of ‘motivated reasoning’ itself.

All reasoning is, to a greater or lesser extent, ‘motivated’ — why would we reason, were it not as a means to some ends or other? However, the implication of Lewandowsky’s claim is that those who believe in climate change have transcended such human faults. Yet we can see that a great deal of presupposition and ‘ideology’ exists prior to the science in the arguments put forward in the debate about the environment. As I have argued previously on Spiked, in order to make the claim that climate change is dangerous, many environmentalists have to presuppose that society’s sensitivity to climate is equivalent to climate’s sensitivity to CO2 — a highly deterministic claim which is not borne out by history, let alone has any grounding in science, and which necessarily precludes the possibility of development mediating the vulnerability of society to climate.

Environmentalism’s presuppositions prefigure much more in Lewandowsky’s analyses than anything he can identify working in the minds of sceptics. This calls into question the ability of psychology to offer insight into the climate debate. For instance, much is made of the scientific consensus on climate change by researchers hoping to understand the phenomenon of climate scepticism, for example by asking whether or not respondents agree or disagree with positions that are assumed to represent the consensus position. However, this method is highly sensitive to the researcher’s ability to accurately represent the consensus, and to elicit from the respondent an accurate picture of his opinion. But as we have already seen, psychologists can quite easily put climate scientists into the category of climate sceptics. Researchers with misconceived ideas about what the science pertains to at best measures the public’s agreement with their own misconceptions.

Unfortunately for Lewandowsky and others engaged in the same attempts to form an understanding of climate scepticism from the psychological perspective, the objects of their study may well have a much better understanding of climate science and its problems than they possess. In this way, complex arguments and nuances are lost in the researcher’s desire to reduce multi-dimensional arguments into categories of right and wrong, good and bad, pro-science and anti-science.

Thus, the consensus on climate science is removed from its scientific context, and becomes a consensus without an object: it can mean whatever climate change psychologists want it to mean. Research into the public understanding of science, where that science has implications for public policy, becomes a vehicle for researchers to manifest their own prejudices as policy.
So what might appear at first pass as an inconsequential squabble on the internet between an arrogant researcher and recalcitrant bloggers in fact has significance for the debate about science’s role in public life. How was it that such shoddy, prejudiced, and partial research passed the seemingly objective tests of peer-review, and that such errors of category, method and analysis were nonetheless deemed worthy of publication by editors of scientific publications?

The answer here is twofold. First, there is the insidiousness of the objectless consensus: the mythology of the climate debate precedes the climate debate. The idea of there being scientists on the one hand, opposed by irrational sceptics on the other has been established so concretely that few editors, peer-reviewers or journalists even bother to ask questions about the content of the consensus, much less about how it is contradicted by the substance of climate sceptics’ arguments. Climate change orthodoxyallowed Lewandowsky’s work to go unchallenged by the checks and balances we might expect to catch out, or at least, criticise, such bare-faced framing of the debate.

Second, there is the political utility of the scientific consensus. The desire for policies to have a grounding in science is ubiquitous amongst a class of ‘policymakers’ (PKA politicians) and institutional science. Research such as Lewandowsky’s would not be significant, were it not for the (wrong) belief that climate scepticism’s influence over public opinion is the chief impediment to climate policies. For instance, in February, Parliament’s Science and Technology Select Committee called for submissions to an inquiry into the public’s understanding of climate change, following a report that had advised that ‘should scepticism continue to increase, democratic governments are likely to find it harder to convince voters to support costly environmental policies aimed at mitigation of, or adaptation to, climate change.’

In the era of ‘evidence-based policy-making’, public opinion is an afterthought rather than the measure of a democratic mandate. Only once a political consensus has been achieved between political parties do today’s ‘policymakers’ seek ways of convincing the public that their policies are a good idea. The extent of this upside-down form of politics is revealed by one of the questions asked by the select committee: ‘Does the Government have sufficient expertise in social and behavioural sciences to understand the relationship between public understanding of climate science and the feasibility of relevant public policies?

The academy increasingly replaces the ballot box in public affairs. And in particular, the science academy. On the face of it, it looks like a good idea. Expertise, is of course, almost always better placed to answer technical questions than is the man-on-the-street. But when the man-on-the-street becomes the object of the ‘social and behavioural sciences’, which are, in turn, employed to elicit his obedience, politics undergoes a radical transformation.

Material and social scientists have not been forthcoming in their criticism of the growing compact between the academy and the state. Indeed it much better serves those who we might expect to be the best critics of power to serve it instead. Lewandowsky, who has now moved from Australia to be the Chair in Cognitive Psychology in the School of Experimental Psychology, at the University of Bristol, has been given the Royal Society’s Wolfson Research Merit Award — a scheme designed ‘for outstanding scientists who would benefit from a five year salary enhancement to help recruit them to or retain them in the UK’. Says the university:

‘Professor Lewandowsky receives the award for his project entitled ‘The (mis)information revolution: information seeking and knowledge transmission’, which addresses how people navigate the blizzard of information with which we are faced on a daily basis, not all of which is accurate or truthful. The project emphasises how people update their memories and under what conditions they are able to discount information that turns out to be false. The project also examines how people interact with, and influence, each other to understand how information spreads through a society.’

Lewandowsky is well placed to speak about filling the public sphere with information ‘not all of which is accurate or truthful’. But what contribution to either science, or society has he really made that is worthy of such an award? And why would the Royal Society be so keen to chuck £tens of thousands at what can be called at best, cod psychology? If this award reflects the Royal Society’s priorities, it says a great deal about them.

As I’ve reported previously on Spiked, the Royal Society has sought an ever expanding role in policy-making, mostly in environmental matters. In particular, the academy published the results of a two year study into population and the environment last year, which coincided with their award to another researcher notable for his failures — population environmentalist and neo-malthusian doomsayer, Paul Ehrlich. It would seem that the closer the Royal Society get to policy-making, the more distance is put between itself and science. In spite of the failures of Ehrlich and Lewandowsky, their claims still have political utility.

It follows that the science academy’s growing desire for influence in the public sphere causes it to seek evidence that the public aren’t capable of managing their own affairs without it. The premise of a technocracy is, after all, the inadequacies of democracy. Thus we see in the Apocalyptic rants of the likes of Paul Ehrlich FRS and Lewandowsky the beneficiary of their generosity not simply claims about the material world, but claims about the shortcomings of human faculties. Between them, these men paint a picture, from the psychology of the individual, through to the functioning of the planet’s natural processes, in which humanity simply cannot help but steer a course for catastrophe without their intervention.

To point this out, though — to point to the problems with the science, or to ask questions about the political assumptions and consequences of these arguments — is to seem to be ‘anti-science’, or to betray a mindset that is preoccupied with conspiracy theories. To point out that the science is, in fact, groundless and speculative crap, and that it is intended, not to advance knowledge, but to serve a political function is to seem to stand out as a denier of science.

Scientists such as Lewandowsky are better at self-justification than scientific research. Rather than being an investigation into the workings of the material world, Lewandosky’s ‘research’ — a poorly executed and error-prone online survey, seen through dodgy statistical methods and bogus categories — is a naked attempt to explain why people dare challenge scientific authority. But there are good reasons for challenging it. Science has turned its gaze on the public as politicians have sought to remedy their diminishing public support by recruiting the academy. It is not a coincidence that the scientific agenda increasingly reflects the prejudices and problems of elite politics.

This would be an anti-science conspiracy theory if it were a claim that Lewandowsky and the Royal Society were engaged in science proper, and were aware of what they were doing. But what Lewandowsky reveals is the consequence of confusing science as a process — a method — and science as an institution. A shonky web survey is given respectability by its author’s professorship and tenure at a university, by peer-review, and by publication. The institutional apparatus of science allowed prejudice and politics to be passed off as objective study. Criticism of the survey — i.e. about the research’s adherence to (or departure from) the scientificmethod –  was defended against on the basis that it challenged, not the scientific argument (i.e. science as a process), but scientific authority (i.e institutional science).

A culture of intransigence has developed in the shadow of the compact between politics and science, which can be seen in the Lewandowsky affair in microcosm. Lewandowsky’s work unwittingly demonstrates that what is passed off as peer-reviewed and published ‘science’, even in today’s world, is no more scientific than the worst ramblings of the least qualified and nuttiest climate change denier on the internet. It looks like science, certainly, but the product only survives a superficial inspection. The only difference being the institutional muscle that Lewandowsky has access to, but which unhinged climate change deniers do not. The object of the Professor’s study is his really his own refusal to debate with his lessers.

The consequence of this should be alarming to everyone who takes an interest in the climate and other scientific debates, no matter what their view on climate change. Lewandowsky demonstrates that the academic institutions do not produce dialogue that has any more merit than the petty exchanges — flame wars –that the internet is famous for. Dressing political arguments up in scientific terminology risks the value of science being lost to society — its potential squandered for an edge in a political fight. After all, if Lewandowsky’s work is representative of the quality of scientific research in general and the standards the academy expects of academics, what does that say about climate science and the quality of the scientific consensus on climate change? If the scientific argument about the link between anthropogenic CO2 and climate change is only as good as Lewandowsky’s claim that ‘Rejection of climate science [is] strongly associated with endorsement of a laissez-faire view of unregulated free markets’, then perhaps climate sceptics should be taken more seriously.

1 comment:

  1. http://www.spiked-online.com/site/article/13716/