What’s the chance of rain?

by on October 9, 2012 at 12:57 pm in Books, Science | Permalink

…the for-profit weather forecasters rarely predict exactly a 50 percent chance of rain, which might seem wishy-washy and indecisive to consumers.  Instead, they’ll flip a coin and round up to 60, or down to 40, even though this makes the forecasts both less accurate and less honest.

That is from Nate Silver’s new and excellent The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t.  The profile of Robin Hanson appears on p.201.

1 Enrique October 9, 2012 at 1:05 pm

Perhaps we should create “weather markets” in which people buy and sell predictions about the weather, or extend the “TruthMarket” concept to weather.

2 abby October 9, 2012 at 1:47 pm

Weather derivatives exist as a type of insurance and are used often. Simple formula. Given any location and specific parameters, we look at 10 years worth of historical data. The premium for the derivative (option or swap) would be the percentage of likelihood plus markup.

For example:
say, chance of 1″+ of rain at Location A on a given date
if historical data shows 25% chance
plus a 1% markup
a $10,000 payout swap would cost you $2,600.

You could easily form some kind of greater than/less than swap deal to simulate this.

3 RR October 9, 2012 at 1:29 pm

A similar example was the first measurement of the height of Mt. Everest. From Wikipedia:
“Peak XV (measured in feet) was calculated to be exactly 29,000 ft (8,839.2 m) high, but was publicly declared to be 29,002 ft (8,839.8 m). The arbitrary addition of 2 ft (61 cm) was to avoid the impression that an exact height of 29,000 feet (8,839.2 m) was nothing more than a rounded estimate.”

4 Thor October 9, 2012 at 8:30 pm

Damn, do I have to go back up there and jump 61 cm into the (thin) air?

5 Urso October 9, 2012 at 1:32 pm

Thanks for the newfound knowledge. But how to arbitrage it?

6 Yancey Ward October 9, 2012 at 1:59 pm

There is only a 60% chance Silver is right.

7 Mark Thorson October 9, 2012 at 2:21 pm

The ideal job is stock market commentator. You just look at what happened on a given day and the stock market movement, and blame one on the other. I’ve heard a rise in oil prices being blamed for a fall in the stock market because a higher price is like a tax on the entire economy. I’ve also heard a rise in oil prices being credited for a rise in the stock market because oil company stocks are a big part of the Dow. As a weatherman, you’d never get away with that kind of thing because everybody would know you’re just making it up.

8 NL7 October 9, 2012 at 3:17 pm

I sold my car after moving to Chicago, so I have to walk anywhere even to get to the bus stop. Any nonzero chance of rain is enough to make me carry an umbrella and/or wear a hoodie; a 10% chance of sprinkles has me reaching for the mini umbrella I keep hooked next to my jacket. To my needs, 40% and 60% are functionally the same.

9 Tom West October 9, 2012 at 3:21 pm

If consumers are happier being lied to, then isn’t the company’s responsibility to do so? After all, the companies ultimate job is to ensure the customers are happy enough to continue paying them. Providing weather reports is simply a mechanism for keeping them happy.

I think pretty much anyone who works in business has seen that on the odd occasions when satisfying a client and satisfying their business needs differ, the job is to satisfy the client.

10 Greg G October 9, 2012 at 3:36 pm

I once asked a weatherman about this. He said people want to either plan a picnic or cancel the picnic. They don’t want to plan half a picnic. They want the weatherman to make a call and they prefer a bad call to a no decision.

11 Andrew' October 9, 2012 at 3:38 pm

“Reports are that 50% of people are still planning to have their picnic.”

12 Urso October 9, 2012 at 4:45 pm

Schrodinger’s picnic

13 ladderff October 9, 2012 at 7:38 pm

haha

14 Andrew' October 10, 2012 at 8:30 am

We need a clearinghouse for the picnic shelter options market.

15 careless October 9, 2012 at 7:57 pm

I feel like I’d still cancel at 40% but not 30.

16 JRPtwo October 9, 2012 at 4:07 pm

So, 60% reported really means about 56.66%?

(Assume that 40%, 50% and 60% chances of rain come up equally often. When we hear 60%, we should think there’s a 1/3 chance it’s a misreported 50%.)

17 mulp October 10, 2012 at 10:31 pm

On Fresh Air, he noted for profit forecasters predict more rain or snow, not less, so 60% means 50% chance of rain, 70% means 60%,…

18 Bill October 9, 2012 at 4:25 pm

We have some long term for profit weather forecasters predicting no climate change as well.

Exxon-Mobil

19 Brian Donohue October 9, 2012 at 5:12 pm

Really? No climate change? Source?

20 Bill October 9, 2012 at 6:54 pm

Am traveling, but google Exxon climate change denial for sources.

21 careless October 9, 2012 at 8:00 pm

Sorry, you’re wrong. They do say that they think it will be “manageable”

22 Bill October 9, 2012 at 8:26 pm

Challenge to others with Internet access. Print results of google search with links below. Am unable to do so with iPad.

23 LemmusLemmus October 10, 2012 at 7:30 am

To expand on careless’s comment:

Second hit:

http://www.triplepundit.com/2012/06/exxon-ceo-thinks-climate-change-priority/

‘While speaking on June 27 to the Council on Foreign Relations, someone asked Rex Tillerson what ExxonMobil plans to do about climate change. Tillerson questioned the climate change models, stating that “our ability to predict, with any accuracy, what the future’s going to be is really pretty limited.” He did state that he believes carbon emissions in the atmosphere are “going to have an impact…a warming impact,” but calls into question how big the impact will be. He even went as far as saying, “We believe those consequences are manageable.”’

This is in clear contradition of Bill’s statement.

24 maguro October 9, 2012 at 8:01 pm

Seems like the safe bet to me, profit or non-profit. The climate is very much the same as it has been for the past 150 years or so, and is overwhelmingly likely to remain that way for the foreseeable future.

25 Eva October 9, 2012 at 4:46 pm

I doubt they truly flip a coin. If the estimates were precise in the first place, they should come up with something like 48% or 51%, and then round anything above 50 up to 60 and anything below 50 down to 40. Why should their calculations result in 10% increments?

26 Matt October 10, 2012 at 10:09 am

My understanding was that the computer spit out a 50% chance, and then they flipped the coin to put that to either 40% or 60%.

27 Urso October 10, 2012 at 10:00 am

Anyway the greater scam is that “60% chance of rain” really means that “it will rain in about 60% of the viewing area of this television station (or newspaper, or whatever).”

28 rjh October 10, 2012 at 12:39 pm

As an ex-forecaster, some corrections. The story is amusing, but doesn’t capture reality.

There are two kinds of professional forecasters:
1) Professional entertainers, e.g., TV weathermen. They maximize ratings. Accuracy is a part of that equation. It’s also full of non-scientific guesses about what will lead to long term satisfied viewers.
2) Commercial forecasters. These are never asked to give a probability of rain. Typical questions are: “what flight route will minimize fuel consumption subject to these schedule constraints?” or “When should we send out the snow plows? we need X hours advance notice.” You keep the customers based on how well your forecasts perform vs your competitors.

Actual forecast skill estimation is typically a “simple” RMS error estimate. You forecast X% probability of rain during a period. The result is either no rain, 0%, or rain, 100%. Long term skill is the RMS error result from lots of forecasts. There are usually situational corrections to deal with easy vs hard and other aspects of skill. I’ve seen plenty of 50% estimates in that kind of skill test. People usually pick a round percentage whereas computer competitors give some precise estimate.

29 Brad Hutchings October 11, 2012 at 12:14 am

I was telling a friend about this as the local SoCal ABC7 weather guy, “Dallas Raines” (yes, and he replaced “Johnny Mountain”), came on. Sure enough, after I mentioned this, he posted a 50% chance of rain for Thursday.

Comments on this entry are closed.

Previous post:

Next post: