How accurate is the Atlas Forecast
We publish the track record of our own forecast, including the days it gets it wrong. Everything is measured with data the system had not seen while learning, against the median of the six models it combines.
In short
Loading the track record…
Week by week
The average error for each week: the Atlas Forecast's and that of the median of the models. The lower the line, the better. Choose the variable, the lead time, the area and the test.
Area by area
The Atlas Forecast switches on in an area once it has at least 2,000 verified hours and beats the models. Until then, it learns: every hour it stores what the area's stations measure.
What we measure, in plain words
The average error
For each hour we compare what we said with what the station measured and keep the difference, without its sign: if we said 18 °C and it was 16.5 °C, we were off by 1.5 °C. The “average error” is the mean of all those differences. Wind and gusts are measured the same way, in km/h.
Who we compare ourselves with
With the median of the six models we combine (ECMWF IFS and AIFS, GFS, ICON, Météo-France and Met Office): the middle value, which already beats almost any single model. Beating it is hard; that is why we also publish each model's accuracy.
With data it had not seen
We run two tests. In cross-validation we forecast each station with a system that learned without it, as happens at any point on the map. In the day-to-day test, every morning we check what the system in use would have said the day before, only for hours it could not see while learning.
Rain
For the chance of rain we use the Brier score: the square of the difference between the probability we gave and what happened (1 if it rained, 0 if not). Zero is perfect. We compare ourselves with the share of models that showed rain, which is the percentage other websites usually show.
Snow
Stations cannot tell whether what falls is rain or snow, so we can neither learn it from them nor verify it. The chance of snow and the snow level come from our temperature, which is verified, using a published physical relationship (Dai, 2008) between temperature and precipitation type. We label it as an estimate.
Up to 72 hours
Temperature goes out to 72 hours only in areas where, at that lead time, it also beats the models with at least 2,000 verified hours. Wind, gusts and rain stay at 48 hours: beyond that, we have not yet shown that we improve on the models.
What it does not say
A good average does not guarantee any particular day: there are days when the models beat us, and you will see it in the chart. Stations measure a single point; between them the weather can change, especially in the mountains and on the coast.
It is a guide forecast and does not replace official warnings from AEMET or Civil Protection. How the whole system works, in How it works.