The weather wisdom of crowds?

by on October 26, 2007 at 10:36 am in Science | Permalink

Jason Kottke reports:

Ben Tesch is about to launch a collaborative weather site called cumul.us.
It’ll aggregate weather information and harness the wisdom of crowds to
see if they can make better weather predictions than the experts.

Will this all work? Who knows, but it only took me two months to make, and I wanted to find out.

Unlike so many other types of information, the web has had little
impact on how weather reporting is done (the Weather Channel stuff is
still rudimentary), so it’ll be interesting to see if this works.

I predict this will fail — how many government agencies already work at predicting the weather?, or in other words the crowd is already in place.  The alternative hypothesis is that weather forecasting awaits its Orley Ashenfelter, and that a mechanism like this will bring the best nerdy, quantitative "amateur" forecast into public prominence.

sa October 26, 2007 at 10:52 am

I don’t get you Prof Cowen. Isn’t this a question of incentives (or the lack therof)?

Kennon Ballou October 26, 2007 at 11:15 am

As with all “wisdom of crowds”-type things, this hinges on the Condorcet Jury Theorem – is there a better than 50% chance that an individual making the prediction will be correct? If so, this should succeed if enough people are involved.

The main reason that prediction markets work well is because the individuals trading more often than not DO have some sort of knowledge that will affect the outcome (and have the incentive to put money on their prediction).

As far as the weather is concerned, I am skeptical that anyone knows much about what the weather will do more than a few days in advance.

rjh October 26, 2007 at 11:48 am

This will both work and fail. There is an extensive science already established regarding how forecast accuracy is measured, and one of the important things to understand is the very wide variety of forecast goals. Typical forecast requirements for the fairly large private forecasting industry include “Forecast the time that snow plows should be started in town X (perhaps to the individual street level) indicating uncertainties because 30min early costs X and 30min late costs Y”. There are internal forecasting efforts to “Minimize the cost of a specific airline flight, including fuel, maintenance, crew time, schedule impact, and ATC costs. (Flying through Canada costs much more than flying through the US for example.) This must take into account enroute and terminal weather, likely congestion areas, and individual flight characteristics (aircraft and load).” I do not expect amateur enthusiasts to approach the skill shown by professionals here.

A second kind of goal is “perform forecast simulations for the next X hours”. This will remain purely the territory of the supercomputer centers.

The third goal of “give a forecast that generates minimum complaints from the public about errors” is one where the crowds might do better. This is a low priority goal for the weather bureaus, and it is extremely hard to define scientifically. It also calls for highly precise localization, which is very labor intensive. The federal funding for this is primarily public safety driven. The local news organizations have argued effectively that there is no federal or state purpose to using tax money to compete with local news organizations for non-safety public forecasts. Similarly, the private forecasters have argued that for the highly tailored local forecasts like aviation, snow plows, etc., the funds should come from the forecast user and not from the federal or state funds. It shouldn’t be too hard for skilled amateurs to match the minimal effort invested by the federal level forecasters.

anomdebus October 26, 2007 at 12:34 pm

I currently use Weather Underground (WU) and am fortunate that there is a privately operated weather station in my neighborhood. If this new site teamed up with WU, they would have no shortage of data.
Anybody who wonders if there are enough data sources just needs to visit http://www.wunderground.com, put in your zip code and at the bottom are a list of the closest privately owned weather stations. Its pretty cool.

disclaimer – I have no connection with Weather Underground, except being a user of their data.

sa October 26, 2007 at 6:37 pm

Interesting points by rjh and Bob Knaus.

Anonymous October 27, 2007 at 6:03 pm

“but we do know enough to predict the global climate over the next 100 years, right?”

Predicting the climate is different from forecasting the weather: http://www.skepticalscience.com/weather-forecasts-vs-climate-models-predictions.htm.

F.

Bob March 14, 2008 at 1:09 am

hi,I University majoring in the legal profession.After graduation,I 徵信 the work of the strong interest.Has worked in several徵信社.Has a wealth of experience. Now I immigrants France,Hope to continue to engage in the work of徵信 credit.
now,is to wake up every day to drink å’–å•¡, shopping. I hope that early awareness of Boles.
thanks,thank very much.

Comments on this entry are closed.

Previous post:

Next post: