Freakonomics NY Times column evaluates forecasting performace of Kansas City TV stations and NOAA office

[Ryan McGinnis]

Quite an interesting read. In some ways, it says some things we already know -- like the fact that a "10 day forecast" is about as reliable as the Farmers Almanac on the 10th day.

http://freakonomics.blogs.nytimes.com/2008/04/21/how-valid-are-tv-weather-forecasts/?rss=true


The graph above shows that stations get their precipitation predictions correct about 85 percent of the time one day out and decline to about 73 percent seven days out.
On the surface, that would not seem too bad. But consider that if a meteorologist always predicted that it would never rain, they would be right 86.3 percent of the time. So if a viewer was looking for more certainty than just assuming it will not rain, a successful meteorologist would have to be better than 86.3 percent. Three of the forecasters were about 87 percent at one day out — a hair over the threshold for success.
Other than that, no forecaster is ever better than just assuming it won’t rain. If you think that’s bad, sadly it gets worse.

[Paul Knightley]

Persistence forecasting (i.e. what the weather is like today is what it'll be like tomorrow) is pretty accurate most of the time. However, that's the key - it's absolutely no good when a change is expected, and especially if that change contains severe weather...it's in tricky/dangerous weather periods that us forecasters earn our keep!

[Mike Peregrine]

That was really interesting. It would also be interesting to see the math for a known severe weather interval, during spring months. I'm sure the long-term results would be the same just about anywhere. It would have been good for them to at least mention the process of model guidance in the article as well, since it is based on mathematics. Something as dynamic as the atmosphere just can't be contained in a 10 minute bottle on the evening news.

[John Cameron]

Interesting indeed. For many years I've heard people speak unfavorably of the weatherman, but I've never heard any of those naysayers specifically mention the National Weather Service. I will guess that's because their forecasters are not on television.

[Ryan Wichman]

I agree with Mike, it is so hard to sum up everything happening in the atmosphere in a simple few minute presentation that you must also water down so the public can understand. I think most weather forecasters would feel more comfortable only giving a 3 day forecast because of the increased accuracy, but now that the 7 day forecasts have been established it would be impossible to take away the extended outlook from the public.

[James Siler]

Here in OKC all of the weathermen on tv were starting to do anywhere from the 7 day forcast plus 4, 5, and i think news nine tried 9 days out. They did it for a very short period and haven't done it since.

[Chad Cowan]

Thanks for sharing Ryan, Freakonomics is one of my top-5 books. I would definitely recommend it to those who found this article interesting. There are a quite a few topics in the book in which the authors bring together statistics and facts through data mining to come up with mind blowing conclusions (widespread teacher-led cheating on standardized tests, business-like hierarchy of drug dealing operations, real estate agents only look out for themselves, etc).

I would like to see the same study done with GFS and ECMWF forecast temps.

[Chris Bowman]

I don't understand the critique of changing the forecast when according to his own stats it has improved the forecast. Do people really want a less accurate forecast if it means that forecast hasn't been changed. :confused:

[Ryan McGinnis]

I don't understand the critique of changing the forecast when according to his own stats it has improved the forecast. Do people really want a less accurate forecast if it means that forecast hasn't been changed. :confused:

I think it was just a roundabout way of saying that if you need to change your forecast that much as the day draws nearer, then the original forecast probably wasn't very useful to begin with.