Climategate II: Phil Jones Schooled in Statistics

4091

30 Oct 2009 05:26:48
from Don McNeil
subject: Re: statistical methods
to p.jones...

Dear Phil:

The treatment of anomalies versus absolute temperatures is not quite as simple as you say.  It might be if there were only 12 monthly constants in all, but in fact there are 12 for each region, and there are a lot of regions, and as we know the regions have different patterns of temperature increase.  So creating anomalies is essentially the same as fitting a model with a large number of parameters (12 times the number of regions) to the data for the 30-year period 1961-1990 and then using this model to adjust ALL the data.  As a result, the variability of the data outside the period used to fit the model is generally not the same as the variability within this period.  The extent of this difference will depend on the distribution of the data, but for the hadCRUT3 data the variability increases with year after 1961.  The reason for this is that the temperatures are generally increasing with different rates for different regions, so they are increasingly “getting away” from the model based on the 1961-1990 data.
If you are just doing descriptive analyses based on anomalies this increasing volatility is not obvious and might not matter too much, and we didn’t see it at first.  It was only after we fitted the “Lee-Carter” regression model (a model that allows for different temperature increases in different regions) to the anomalies that we realised the problem, which disappeared when we reclaimed the absolute data by adding absTem3 to the anomalies.  That’s why statisticians like myself prefer to start with absolute data, rather than “anomalies” that already have been adjusted.  We can make the adjustments ourselves, ensuring that these adjustments apply equally to the whole period.  (I guess this is what you meant when you said “this is always a problem for statisticians, but ones like Peter Bloomfield, Richard Smith and Rick Katz (who have all worked extensively with climate data) understand why.”)
Peter Bloomfield was my close colleague in the Statistics Dept at Princeton from 1970-1976 (with John Tukey who suggested the Winsorizing you use for your data cleaning), and he suggested I contact you when I told him that my students and colleagues in Thailand were interested in looking at global warming data.  He also suggested I contact the NCAR people, and they put me on to Richard Smith's work. (But Richard’s a Bayesian and therefore on a different planet to us mainstream statisticians!)
I’m puzzled by your comment that precip[it]ation is more likely an effect than a cause of temperature increase.  Isn’t precip[it]ation due to atmospheric water vapour, one of the greenhouse gases that cause global warming?
Cheers....Don

See also “CRU’s Dr Phil Jones, world renowned climatologist, can’t even plot a trend in excel”, at Watts Up With That, and “0435: Briffa: Pathetic GRL Hockey Stick Paper a Step Backwards in Time and- Understanding”, at The Unbearable Nakedness of Climate Change (wherein Edward Cook is quoted conceding, “we know with certainty that we know fuck-all”), and Steve Goddard’s “Phil Jones Gets Slammed by Reviewer for GCM Nonsense”, at Real Science.