GCM Climate Metrics

Background

Numerous studies have documented biases in general circulation models of climate, including their simulations of cloud, water vapor, and rainfall. However these biases tell you little about how accurately the model will simulate the response to small changes in radiative forcing or other boundary conditions. It is often possible to "tune" a model in such a way as to remove or shift these biases without greatly affecting its sensitivity to forcings. By the same token, several models whose climatic biases are similar may exhibit very different senstivities to forcings. These latter differences are due largely to behavior of clouds, especially those in the lower troposphere. A recent focus of the GEWEX working group on cloud processes and climate is the development of metrics for climate models that are more relevant to the models' ability to simulate changes, or to the fidelity of model processes.

LTMI metric of Sherwood et al. (2104) related to global climate sensitivity

In January 2014 I published, with Sandrine Bony and Jean-Louis Dufresne, a paper providing evidence that the amount of mixing between the lower and middle troposphere controlled the strength of a positive low-cloud feedback in models (see also my Explainer of this paper in The Conversation). This paper presented a metric called LTMI, which was the sum of two measures S and D. The small-scale measure S is based on simple averages of fields in a restricted domain, and is sufficiently straightforward that I have not provided a code to calculate it.

The code I used for calculating the parameter D, and quantities used to compute resolved upward humidity and mass fluxes, can be found here. It is written in IDL. You will also need a land-sea mask data file used by the routine. This code requires only the omega field (vertical pressure velocity) but has options to compute humidity fluxes as well, in which case specific humidity data are also needed.

Evaluation metrics based on the joint distributions of climatological mean cloud and moisture variables

Map

Upper left: Joint density (histogram) of precipitable water below 700 hPa (PW)
and upper-tropospheric humidity with respect to ice (UTH). Other three panels:
average net (CRF), shortwave (SW), and longwave (LW) cloud forcings for
each combination of PW and UTH. All data are annual means.
(After Fig. 4 of Bennhold and Sherwood 2008).

Map

Same as previous figure, except based on output from the CCSM climate model.

Overall model assessment efforts — particularly those focusing on cloud-related processes &mdash have recently shifted focus from climatological mean fields to model behaviour on shorter time scales closer to those of key processes such as convection. This valuable approach is generally more difficult from the point of view both of available observations and typical model output. However, we have discovered a surprisingly effective (and easier) alternative that clearly reveals process-relevant model-data discrepancies, described in Bennhold and Sherwood (2008). It is based on simply examining the (a) joint density (histogram) of column precipitable water against relative humidity in the upper troposphere, and (b) the conditional mean of different cloud-forcing variables conditioned on pairs of these variables, where statistics are computed using the long-term annual means of all model grid points in low and midlatitudes. It seems crazy (for example, averaging over the seasonal cycle) but it works. The resulting discrepancies (see figure) stand out far more clearly than would be evident from previous model-data comparisons. Bennhold and Sherwood (2008) hypothesize some physical interpretations of these discrepancies and how model convective process errors might be responsible for them, but further work is needed to test these suggestions. We encourage model developers to compare their models to the observations reported here. Satellite-observed upper-troposheric humidity (UTH) is not a straightforward quantity to calculate in a GCM, since it is a vertical integral whose weighting function depends on temperature and humidity. Fortunately, we found that (at least for the three models we examined) results are nearly identical if one approximates UTH as the mean of the relative humidities with respect to ice at the 300 and 500 hPa levels. The data appearing in this figure and described in the paper are provided here as a NetCDF file: Download NetCDF Data file.

F. Bennhold and S. C. Sherwood, Erroneous relationships among humidity and cloud forcing variables in three global climate models. Journal of Climate, Vol. 21, 2008, 4190-4206
view abstract / print version