Who’s the Fairest Forecaster of Them All?

Accuweather, The Weather Channel, and the PopSci Dartboard battle it out.

sci0502weather_A.jpg

CLOUDED JUDGEMENT Forecasting the weather, our experiment reveals, is much tougher than it looks.

“Even I could predict the weather better than those guys.” That statement, uttered by Managing Editor Jill Shomer on a colder-than-predicted January afternoon, started it all. The mounting conventional wisdom, it seemed to us, was that today’s meteorologists couldn’t forecast clouds in a rainstorm. Fair? Certainly not, but poor recent forecasting had us questioning the accuracy of 5- and even 3-day forecasts. Could we get the same precision from, say, a dartboard?

To find out, we created a twofold approach: First, learn how forecasts are made-the science, the techniques, the obstacles. For that, we sent colleague Michael Dolan to the National Weather Service in Upton, New York. His report is below. Next, we wanted to determine the accuracy of today’s forecasts. We focused on two big forecasters: The Weather Channel, which reaches 80 million households; and Accuweather, which works with over 2,500 media outlets. We also included a $15 Smithsonian at-home kit for kids (which could only do next-day forecasts), and the PopSci Dartboard. We took Weather Channel and Accuweather forecasts off the Web; our managing editor was our at-home forecaster; and we weighted our dartboard toward historical averages, though we threw in a few once-in-a-lifetime events (like plague of locusts) for fun.

For each of 29 days in February and early March, we recorded (or made) 5-day, 3-day, and 24-hour forecasts-high and low temps, sky conditions, and precipitation-for our office zip code, 10016. Ten points were available in each of the four categories. We granted a degree of grace on both sides of actual temperature, but one point was subtracted for each additional degree off. Scoring skies and precipitation was more subjective, but fair for all. The results (left) surprised us all, but no one more than our humbled managing editor.

WHO’S THE MOST ACCURATE?

Accuweather
5-day: 61.13%
3-day: 69.93%
24-hour: 79.33%

The Weather Channel
5-day: 61.93%
3-day: 70.03%
24-hour: 78.33%

PopSci Dartboard
5-day: 44.71%
3-day: 43.45%
24-hour: 44.90%

At-Home Kit
5-day: N/A
3-day: N/A
24-hour: 62.56%

ME, THE METEOROLOGIST

It’s going to snow tomorrow. This much I know, because I had studied the next day’s forecasting model at the New York office of the National Weather Service.

Jeffrey Tongue, a head meteorologist here, smiles. There’s too much dry air at the upper levels, he says, and any precip making it east will turn to rain because of the urban heat island. Huh?

Weather forecasting, I learn, is as much judgment as science. I relied on technology, but Tongue pulled out a more valuable tool: experience. Every forecast starts with raw data-temperatures, winds, dewpoints, precipitation, cloud heights-from 120 stations around the country. This creates mathematical models that help meteorologists predict system movements. Sometimes the models agree; sometimes they don’t. That’s when experience comes in. Just as the concrete in Manhattan could mean rain instead of snow, a wind shift-or a mountain or lake-adds a new level of complexity.

With Tongue’s help, I make a 5-day forecast. And while I now understand the snow situation, no amount of explaining helps me see the 18-degree temperature rise that Tongue predicts for Thursday evening. I take his word for it.