WSJ Op-Ed on the corruption of peer-review in science, including climate science

The Frogs and the Gulls
Nicholas Kristof's confirmation bias.

By JAMES TARANTO
July 14, 2014    THE WALL STREET JOURNAL

In a July 8, 2004, email, one scientist assured another that the hypothesis they shared would prevail "even if we have to redefine what the peer-review literature is!" Exactly 10 years later, RetractionWatch.com reported that Peter Chen, a researcher at Taiwan's National Pingtung University of Education, had undertaken such a redefinition. "SAGE Publishers is retracting 60 articles from the Journal of Vibration and Control after an investigation revealed a 'peer review and citation ring,' " noted RetractionWatch's Ivan Oransky.

According to a statement from SAGE, "it was discovered that the author had created various aliases on SAGE Track, providing different email addresses to set up more than one account. Consequently, SAGE scrutinised further the co-authors of and reviewers selected for Peter Chen's papers, [and] these names appeared to form part of a peer review ring. The investigation also revealed that on at least one occasion, the author Peter Chen reviewed his own paper under one of the aliases he had created."

Corruption of the peer-review process is a widespread problem in scientific research, argues Hank Campbell of Science 2.0 in an op-ed for today's Wall Street Journal. "Even the most rigorous peer review can be effective only if authors provide the data they used to reach their results, something that many still won't do and that few journals require for publication," Campbell notes. He offers this example:

In 2002 and 2010, papers published in the Proceedings of the National Academy of Sciences claimed that a pesticide called atrazine was causing sex changes in frogs. As a result the Environmental Protection Agency set up special panels to re-examine the product's safety. Both papers had the same editor, David Wake of the University of California, Berkeley, who is a colleague of the papers' lead author, Tyrone Hayes, also of Berkeley. 

In keeping with National Academy of Sciences policy, Prof. Hayes preselected Prof. Wake as his editor. Both studies were published without a review of the data used to reach the finding. No one has been able to reproduce the results of either paper, including the EPA. . . . As the agency investigated, it couldn't even use those papers about atrazine's alleged effects because the research they were based on didn't meet the criteria for legitimate scientific work. The authors refused to hand over data that led them to their claimed results--which meant no one could run the same computer program and match their results.

That sounded familiar. In a May 2012 column, the New York Times's Nicholas Kristof had written: "A widely used herbicide acts as a female hormone and feminizes male animals in the wild. Thus male frogs can have female organs, and some male fish actually produce eggs."

A red-eyed tree frog in Costa Rica. Getty Images

"What herbicide exactly?" asked science journalistDeborah Blum in a critical blog post for the Public Library of Science. "Here, reader, you are just out of luck. Because he is just not going to tell you that. Not in that [para]graph or anywhere in the piece."

She went on: "I'm going to guess that it's (a) the herbicide Atrazine which was linked ten years ago to "hermaphroditic, demasculinized frogs." But there were at least two other possibilities, which she designated (b) and (c). "Or it could be (d) all of the above. You tell me, reader, because the New York Times column doesn't." (A terminological note: Herbicides, or weed killers, are considered a type of pesticide.)

Blum opened her post by describing herself as a "long-time fan" of Kristof, specifically of "his work in social justice journalism, his passionate reporting of problems [that] others ignore, his dedication to helping people in traumatized regions of Africa":

It's outstanding work and, oh, how I wish he would stick to it. Because his secondary crusade of the last few years, you know, the one against evil industrial chemicals, is really starting to annoy me.

It's true that Kristof has done a good deal of worthwhile reporting on Third World human-rights abuses. But there, too, he has sometimes proved to be gullible. As Slate's Amanda Hess noted last month, Kristof's 2009 reporting on sex slavery in Cambodia relied heavily on the claims of an activist named Somaly Mam and of Mam's associate Long Pross, "a Cambodian teenager who said she was kidnapped, beaten, tortured with electric currents, tied up, and sold in a brothel at 13, 'where her brothel owner gouged out her right eye.' " That story fell apart in May:

Newsweek revealed that the most horrific sections of Mam's backstory had been inflated and fabricated, and that she had enlisted Pross--who had actually lost the eye after undergoing surgery for a nonmalignant tumor--to do the same. Responding to revelations about Mam's deception, Kristof said in a column [actually a blog post] last week, "I wish I had never written about her."

Similarly, in an April 2011 column Kristof had to distance himself from the work of Gerg Mortenson, who runs a charity called the Central Asia Institute and whose memoir, "Three Cups of Tea," described his work building schools in Afghanistan and Pakistan. Mortenson had just been the subject of a "60 Minutes" exposé, as Kristof wrote:

Greg is accused of many offenses: misstating how he got started building schools; lying about a dramatic kidnapping; exaggerating how many schools he has built and operates; and using his charity, the Central Asia Institute, "as his personal A.T.M." The attorney general of Montana, where his charity is based, has opened an inquiry into the allegations.

A year later, the Montana Department of Justice announced a settlement in which "Mortenson agreed to repay in excess of $1 million to the charity, including credit for some payments already made, for book royalties, speaking and travel fees, promotional costs and inappropriate personal charges." He remained on the institute's payroll, but "through the settlement, he agrees to no longer oversee financial aspects of the charity or serve as a voting member of the board of directors so long as he remains an employee." No criminal charges were filed.

In 2011, Kristoff was withholding judgment:

I don't know what to make of these accusations. Part of me wishes that all this journalistic energy had been directed instead to ferret out abuses by politicians who allocate government resources to campaign donors rather than to the neediest among us, but that's not a real answer. The critics have raised serious questions that deserve better answers: we need to hold school-builders accountable as well as fat cats.

The conclusion is right, but Kristof's wish is revealing. Moral crusaders are especially vulnerable to confirmation bias, the tendency to be insufficiently rigorous about testing information that bolsters their preconceptions. That's a problem in science as well, as Campbell notes:

Absent rigorous peer review, we get the paper published in June in the Proceedings of the National Academy of Sciences. Titled "Female hurricanes are deadlier than male hurricanes," it concluded that hurricanes with female names cause more deaths than male-named hurricanes--ostensibly because implicit sexism makes people take the storms with a woman's name less seriously. The work was debunked once its methods were examined, but not before it got attention nationwide.

Among those who gave it uncritical attention was Nicholas Kristof, in his June 12 column.

Speaking of weather, the email quote with which we opened this column did not come from Peter Chen but from Phil Jones, a climate researcher at Britain's University of East Anglia. It was part of the "Climategate" trove released in 2009, and Jones was referring to a paper that questioned elements of the purported consensus on anthropogenic global warming.

"The scientific consensus is stronger than ever," wrote a New York Times columnist this past January. You can probably guess which one.