You are here

Analyzing precipitation projections: A comparison of different approaches to climate model evaluation

TitleAnalyzing precipitation projections: A comparison of different approaches to climate model evaluation
Publication TypeManual Entry
Year of Publication2011
AuthorsSchaller, N., I. Mahlstein, J. Cermak, and R. Knutti
Journal of Geophysical Research-Atmospheres

Complexity and resolution of global climate models are steadily increasing, yet the uncertainty of their projections remains large, particularly for precipitation. Given the impacts precipitation changes have on ecosystems, there is a need to reduce projection uncertainty by assessing the performance of climate models. A common way of evaluating models is to consider global maps of errors against observations for a range of variables. However, depending on the purpose, feature-based metrics defined on a regional scale and for one variable may be more suitable to identify the most accurate models. We compare three different ways of ranking the CMIP3 climate models: errors in a broad range of climate variables, errors in global field of precipitation, and regional features of modeled precipitation in areas where pronounced future changes are expected. The same analysis is performed for temperature to identify potential differences between variables. The multimodel mean is found to outperform all single models in the global field-based rankings but performs only averagely for the feature-based ranking. Selecting the best models for each metric reduces the absolute spread in projections. If anomalies are considered, the model spread is reduced in a few regions, while the uncertainty can be increased in others. We also demonstrate that the common attribution of a lack of model agreement in precipitation projections to different model physics may be misleading. Agreement is similarly poor within different ensemble members of the same model, indicating that the lack of robust trends can be attributed partly to a low signal-to-noise ratio.

Citation Key99
Community Notes

Schaller et al. investigate several approaches to climate model evaluation.  Climate model evaluation is an important task in climate science and there is currently no standard means to perform evaluations.  There are also many different climate applications that require different levels of evaluation based on what measurements are appropriate to their work.  This study evaluated the CMIP3 climate models using methods of investigating errors in climate variables, errors in global fields, and regional features.  One limiting factor of evaluation is the availability of observed climate records.  Many times there are substantially more records over land compared to over water.  Of these three methods, no CMIP3 model stood out in performance to the rest.  Instead, the multimodel mean is often favored over any individual model because it has relatively small biases over the whole globe and a good representation of spatial patterns.  This approach is less useful for regional climate studies that require additional representation of local physical mechanisms.  This article is useful for describing how others are evaluation climate models and how there is yet to be defined a standard practice.