Friday, December 16, 2011

New paper: Popular method to compare models to observations is 'highly misleading'

From the annals of The Settled Science, a paper published today in Geophysical Research Letters states, "Comparison of [computer] model outputs with observations of the climate system forms an essential component of model assessment and is crucial for building our confidence in model predictions." However, the paper finds that the "methods for undertaking this comparison are not always clearly justified and understood" and that the popular approach of comparing the spread of model outputs to constrained observations can be "highly misleading." Furthermore, the method of comparison recommended by the authors "may lead to very different...conclusions."


GEOPHYSICAL RESEARCH LETTERS, VOL. 38, L24702, 5 PP., 2011
doi:10.1029/2011GL049812
Key Points
  • We present an alternative paradigm for ensemble evaluation
  • Previous assessments of CMIP3 ensemble spread may be misleadingly pessimistic
J. D. Annan
Research Institute for Global Change,, Yokohama,, Japan
J. C. Hargreaves
Research Institute for Global Change,, Yokohama,, Japan
K. Tachiiri
Research Institute for Global Change,, Yokohama,, Japan
Comparison of model outputs with observations of the climate system forms an essential component of model assessment and is crucial for building our confidence in model predictions. Methods for undertaking this comparison are not always clearly justified and understood. Here we show that the popular approach of comparing the ensemble spread to a so-called “observationally-constrained pdf” can be highly misleading. Such a comparison will almost certainly result in disagreement, but in reality tells us little about the performance of the ensemble. We present an alternative approach, and show how it may lead to very different, and rather more encouraging, conclusions. We additionally present some necessary conditions for an ensemble (or more generally, a probabilistic prediction) to be challenged by an observation.

1 comment:

  1. Thirty years ago I already knew the main problem with modern science, across the board, was a failure to understand and abide by the basic rules of probability, in order to properly confront the many problems with modern theories. The reason for this is because modern theories are unquestioned dogmas (they are like the "favorite team" of an obsessive sports fan, to be defended against any and all criticism), and the evidence that should overturn them in an instant is blunted and ignored by being turned into a "mere probability", which, however decidedly against the popular dogma (or "consensus"), is then routinely, and incompetently, dismissed as "not proof". In that larger context, which scientists have yet to even begin to honestly face, statistical science (and the judging of theories) at bottom is just understanding probability. What is wrong with climate science cannot be cured just using a different statistical method, upon models that contain bad theory, and thus can never give true results; climate scientists need to confront the definitive evidence that invalidates the greenhouse theory, at the root of the incompetent climate consensus. Instead they label that evidence a "coincidence", and dismiss it. They need to face the fact that a "coincidence" is something that is ridiculously improbable, by chance alone, and DEMANDS AN EXPLANATION before they can say that they know the truth, or that "the science is settled". Those of us not bound to the climate models already know they are all wrong, and it doesn't take complex statistical arguments to know that. So any paper that seeks to wring the truth out of those models is wrong-headed, from the start. The problem is deeper, and more basic, fellow scientists, than you are yet willing to face. Your fundamental theories, your supposed basic understandings, are incompetent, and the most obvious evidence of that is that they are ridiculously improbable (which you have been taught to ignore, and so you make those theories practically unfalsifiable).

    ReplyDelete