Two-Week Weather Prediction Accuracy

Last month I posted an analysis of the accuracy of several weather forecasting services, focusing just on temperature predictions. By the time they got to Day 7 in their predictions, the various forecasters were from 8-13 degrees (F) off from the actual temperatures for that day. The goal of this new analysis is to test predictions after that--from day 7-16, their 'long-range' forecasts. Most sites do not offer this service. For this test, I used Accuweather & The Weather Channel. Additionally, I compared those predictions to the historical averages provided by the National Weather Service. As can be seen in the charts, their long-range forecasts ranged from 4-12 degrees (F) off, and simply going by the historical averages were about as accurate as long-range predictions.

Data & Methods
For this analysis, I collected high and low temperature predictions from Accuweather and The Weather Channel from Jan 18-Jan 27 (2018), and historical temperature averages from the National Weather Service (NWS). Temperatures were typically collected at the same time of day, around 10am. I missed one day, Jan 19, so this analysis represents 9-days of forecast data, followed by another two weeks (until Feb 11) of recording the actual high-low temperatures as documented by the NWS.

For each of the 9 days of collecting forecast data, I recorded the predicted high and low temperatures the two services provided for days 7-16 into the future. Interestingly, starting with the 16th day of predictions, The Weather Channel no longer reported an attempt to predict the temperature, but simply reverted their forecast to the historical average temperature. Accuweather continued to provide a unique forecast for 3 months into the future.

Analysis & Results
The statistical analysis is based on taking the (absolute value of the) average difference of the forecasted temperature compared to the actual temperature over Days 7-16 into the future. That difference is plotted in Graph 1. Then the average of temperature deviations for all 10-days is plotted in Graph 2.

As can be seen in Graph 1, the deviations from the predicted versus actual high & low temperatures range from 4-12 degrees (F) over the course of the 10 days of predictions. What is striking in the graph is that there is no trend in the data--in other words, the further into the future the forecast goes, the predictions do not get worse (or better). This implies that by Day 7, the accuracy of the temperature forecasts are as good (or bad) as they are going to get from that point on, and that accuracy is not very good. While there is significant deviation in how badly the forecasts are off over the course of the 10-days of predictions, for example, Accuweather's low predictions range from being 5-degrees off at Day 8, versus 12-degrees off on Day 12, their low-temperature forecast did not continue to worsen--in fact, it got better. Similarly, on Day 7, The Weather Channel's low-temperature prediction was almost 10-degrees off, but decreased to about 7-degrees of after that, reaching a 6-degree deviation by Day 16. This lack of a trend implies a high-degree of randomness in the models used to predict temperatures by Day-7 into the future.

As can be seen in Graph 2, the average deviation in the forecasted temperatures over the 10-days of predictions was about 7-degrees off for both the lows and the highs. The average deviation from the historical averages is also about 7-degrees. This implies that with the current technology and weather modelling, the long-range temperature forecast is no better than simply relying on the historical averages, at least for the low-temperatures. The high-temperature predictions were slightly better--The Weather Channel's deviation was only 6-degrees off, while the historical averages were 9-degrees off. However, my guess is that this is simply a data-collection anomaly--in other words, collecting data over a longer time-perioud would likely have erased these differences.

Conclusion
While this sample size is limited, it seems reasonable to conclude that the accuracy of long-range temperature forecasts (defined as 7-16 days into the future) is no better than simply relying on the historical temperature averages. While sites like Accuweather provide temperature forecasts as far as 3-months into the future, those forecasts likely are as useless as flipping a coin to guess the temperature, or reading your horoscope to predict how your day will go.

One question I did not pursue, is the impact of climate change on the accuracy of 'historical average' temperatures. For example, assuming that climate has been changing over the last century, what if, instead of using the entire historical average highs and lows as provided by the NWS (I do not know how far back they take their averages), we simply used the highs and lows for the last 10 years? Would that data be closer to the actual temperature than the current historical average?