Paper: Models lead to overly confident climate predictions

A paper published today in the Journal of Climate finds that ensembles of climate models used by the IPCC to predict future climate change "may lead to overly confident climate predictions." The authors find that many models share the same computer code, have the same limitations, and "tend to be fairly similar," resulting in confirmation bias. Indeed, empirical observations have shown far less warming than the "90% confident" IPCC models in AR4, as shown in this poster by John Christy:

Journal of Climate doi: 10.1175/2010JCLI3814.1 

On the Effective Number of Climate Models

Christopher Pennell and Thomas Reichler

Department of Atmospheric Sciences, University of Utah, Salt Lake City, UT

Abstract: Projections of future climate change are increasingly based on the output of many different models. Typically, the mean over all model simulations is considered as the optimal prediction, with the underlying assumption that different models provide statistically independent information evenly distributed around the true state. However, there is reason to believe that this is not the best assumption. Coupled models are of comparable complexity and are constructed in similar ways. Some models share parts of the same code and some models are even developed at the same center. Therefore, the limitations of these models tend to be fairly similar, contributing to the well-known problem of common model biases and possibly to an unrealistically small spread in the outcomes of model predictions.

This study attempts to quantify the extent of this problem by asking how many models there effectively are and how to best determine this number. Quantifying the effective number of models is achieved by evaluating 24 state-of-the-art models and their ability to simulate broad aspects of 20th century climate. Using two different approaches, we calculate the amount of unique information in the ensemble and find that the effective ensemble size is much smaller than the actual number of models. As more models are included in an ensemble the amount of new information diminishes in proportion. Furthermore, we find that this reduction goes beyond the problem of “same-center” models and that systemic similarities exist across all models. We speculate that current methodologies for the interpretation of multi-model ensembles may lead to overly confident climate predictions.