But I Was Probably Right About Climate Models
I try not to practice false modesty (those of you who know me well probably just did a spit take at that understatement). So while I try to stand up and admit when I’m wrong, I also like to stand up and point out where I’m right.
It shouldn’t be a surprise to any of you that I came to the conclusion that climate models are pretty much total bullshit. My problem with them is that they are incomplete, overfitted, and unproven. It turns out that one of the foremost experts on forecasting in general also thinks that these models have no predictive value. In fact, items (6) and (7) of their statement shows that you can predict the future temperature really well simply by saying it will be the same as the current temperature.
You can read their more formal indictment of climate forecasting methods here.
Oh snap!
Possible Insight 
I’m curious as to what arguments have been put forth as to the forecastability of global climate in principle. That is, what can we even hope to know about the global climate system in the future, if anything? And what kinds of forecasts are a priori to be thrown out? Is there a time horizon before which we can make meaningful claims and after which it is truly chaotic? And if so what does it look like in terms of predictability as a function of time?
Relatedly, if one is prepared to say that the global climate system is not predictable even in principle (due to complexity), then how does that jive with macroeconomics being somewhat predictable?
rafefurst
January 29, 2009 at 3:06 pm
If you go read the links I put in there, you will see that a good forecast of the mean surface temperature 100 years from now is “somewhere within 0.5 deg C of this year’s MST”.
So the problem is not that the climate is not forecast-able in principle. It’s just really hard to beat the most naive forecast possible.
Oh, and I never said macroeconomics was good at forecasting. I just said I think it’s a “coherent study of a complex system.” Whether there’s any forecasting value to be extracted from it, I do not know. Before, I thought it “wasn’t even wrong”. Now it might be right or wrong.
kevindick
January 29, 2009 at 3:24 pm
I can’t find what you are referring to, but if you are saying that the past is a good indicator of the future, i.e. the null theory is the best predictive model we have. I’m not asking for the best model we have, I’m asking what work has been done on the epistemological question of how good can such a theory in principle get.
Some systems might be so complex that short of running the system you can’t get any meaningful prediction whatsoever; the system itself is it’s only viable model. This of course begs the question of what is “meaningful”, but presumably that would be defined in such an epistemological argument about the climate.
Regarding macro, it is an interesting question as to whether we should consider models that are descriptive “coherent studies” but not at all predictive as being useful models or not. It gets to Popperian falsifiability. Or by coherent do you just mean self-consistent?
rafefurst
January 29, 2009 at 4:10 pm
I’m talking about the first document I link to, points (6) and (7). You really should read the whole thing. It’s only a couple of pages.
His point isn’t that we can’t forecast climate in principle. It’s that, in principle, we know the global climate to vary so little that it’s not worth building forecasting models more complicated than the naive one. My take is that it may be worth estimating the probabilities of discrete events that destabilizing the climate. But building a continuous forecasting model is a waste of resources.
As for macroeconomics, I agree that models should be predictive. The modern macro models are predictive models. I just don’t know how good they are. The old school models might have been statistically predictive but they were so divorced from a coherent conception of the underlying processes, that it was hard to evaluate whether they were worthwhile.
kevindick
January 29, 2009 at 7:38 pm
“My take is that it may be worth estimating the probabilities of discrete events that destabilizing the climate.”
Meh, You may be missing 99% of the major destabilizing events. And who says you can even predict the ones you know about.
Daniel
January 30, 2009 at 9:35 am
Regarding predictability of climate, I suspect it’s more predictable than the economy (even though it includes the economy) for the simple fact of much slower feedback.
rafefurst
January 30, 2009 at 10:16 am
@Daniel. Note that I said “might”. I was specifically thinking of the case where we want to allocate resources to responding to climate mediated disasters. In that case, you have to do some assessment of the relative likelihoods of things like volcanoes and unusual solar minima. Otherwise, you don’t know how much money to put in relative to other disaster preparedness.
@Rafe. That’s pretty much a tautology given that our experience with climate is that it varies much less than the economy.
kevindick
January 30, 2009 at 10:49 am
One thing that the debate lacks is a proper framing of the problem. Are we just talking about average global temperature? Surely there are other factors that matter when we are talking about “climate change”.
If the claim is that the climate is not changing, and we see graphs of C02 content in the atmosphere rising precipitously, and we see evidence of different local trends (e.g. greater frequency and magnitude of hurricanes, ice caps melting, etc.) and we see phenomena which may be related (e.g. severe drop in species diversity in the oceans), then we can be as pedantic as we want about “climate stability”, but that’s not really what the debate is about.
The debate is about (a) what in principle can we know about the future, and (b) what can we effectively do to shape the future to our liking.
Regarding (a), it does no good to simply look back 150 years; the climate system has been known to go through periods of long relative stability punctuated by periods of massive change and instability. It is worth questioning whether we are not at one of those transition points, man-made or not.
To even address (b), we need an answer to (a). If the future is too unpredictable, then it certainly is foolish to try to change the climate. You don’t know what effects your actions will have (could be the exact opposite of your intentions). In that case, we’d be better off working on ways to ameliorate negative effects and be as resilient as possible in the face of whatever may come. I think this is where Kevin prefers we spend our energy.
But, it’s still not clear to me that the future is so unpredictable that we need to throw up our hands at effecting it. I understand that if you believe it is stable then the action you should choose is do nothing. But if we admit that it is a non-linear system with punctuated equilibrium (based on the long view of history), perhaps there are some general predictions we can make that have feasible policy consequences for attempting to change or stabilize the climate if we are heading into a period of destabilization.
rafefurst
January 30, 2009 at 1:49 pm
@Rafe. I will ignore the first half of your last comment. I don’t see anything productive to debate about it. I think “climate change” is a red herring and you don’t.
The second half is where things get interesting. We believe, _in principle_, that the climate is a non-linear system. Unfortunately, we’ve only observed it in its punctuated equilibrium. This is Armstrong’s point. You can’t say anything meaningful about what it will do in response to impulses we haven’t observed.
Now, you can say we have paleoclimate data. But our historical climate reconstructions aren’t very precise. So until our confidence in them gets better, there are huge error bars in the _inputs_ we would use to construct a model of this non-linear system. That’s not a recipe for any kind of forecasting success as you well know.
In principle, if we had a fine enough grid of sensors generating enough different kids of data points with enough precision, we could calculate the effects of different impulses. But I don’t hold out much hope of this working out in practice.
The bottom line is that we know in general what the shape of the climate distribution is 100 years in the future. It’s very narrow around the current temperature, but the tails (on both ends) are very long. It’s hard to see us understanding the system well enough to establish the parameters of this distribution with a level of confidence that justifies different policies.
kevindick
January 30, 2009 at 2:31 pm
[…] But I Was Probably Right About Climate Models […]
Micro –> Macro –> Micro, etc. « The Emergent Fool
April 10, 2009 at 3:49 pm