Skip to content

Evaluation metrics

A forecast can be close on average and still useless for trading, or rough on average and still valuable. EPF reports both kinds of measure, and never reports average error alone.

The headline metric, in €/MWh:

MAE = average of |actual price − forecast price|

An MAE of 18 means the forecast is, on average, 18 €/MWh away from the cleared price for each quarter-hour. It is easy to read in market terms and is not dominated by a handful of extreme hours. All MAE figures on this site are computed over 15-minute quarters.

Squares errors before averaging, so large misses weigh more. The ratio RMSE / MAE says something about the shape of the errors: close to 1.25 is typical; well above 1.5 means a few large misses dominate.

Bias = average of (forecast − actual)

Positive means the forecast runs high, negative means it runs low. A small MAE with a large bias points at a fixable, systematic problem.

Percentage error is shown with care. Electricity prices can be near zero or negative, and dividing by a price near zero produces meaningless percentages, so those quarters are excluded. Comparing MAE to the average price level is often a clearer guide.

Skill = 1 − (MAE of EPF / MAE of a naive baseline)

Above zero, EPF beats the baseline; at zero it is no better; below zero it is worse. See Walk-forward and baselines.

Consider two forecasts for one day:

  • Forecast A predicts a flat 50 €/MWh all day. MAE 8.
  • Forecast B predicts 30 at night and 70 at noon. MAE 12.

A has the lower error, but B is the one a battery operator can use: it says when to charge and when to discharge. A sees no spread, so it earns nothing. These metrics capture that difference.

The share of consecutive periods where the forecast gets the direction of the price change right (up or down).

Score Reading
60% or more Meaningful directional signal
50 to 60% Marginal
50% or less No better than a coin

What fraction of the day’s best possible battery spread a strategy guided by the forecast would actually earn.

  1. Best possible: charge in the cheapest hours of the day, discharge in the most expensive, using the actual prices.
  2. Forecast-guided: pick the charge and discharge hours from the forecast, then value them at the actual prices.
  3. Spread capture = forecast-guided spread / best possible spread.
Score Reading
70% or more Captures most of the available arbitrage
50 to 70% Useful, but leaves value on the table
below 50% Little better than choosing hours at random

The correlation between forecast and actual prices after removing each day’s average. It asks whether the forecast puts the cheap and expensive hours of the day in the right places, regardless of the overall level. Plain correlation is less useful, because the daily shape alone makes almost any forecast look well correlated.

Of the most expensive tenth of actual quarter-hours, how many did the forecast also rank in its own most expensive tenth. These are the periods where selling earns most.

Whether errors are related to the price level. Errors that grow systematically on expensive hours mean the forecast undervalues exactly the hours that matter most.

Metrics are also reported by time of day, day of week, horizon and price band. Typical patterns: error is lowest in the middle of a sunny day, when solar output pins the price, and highest through the evening ramp; it rises with horizon; and very cheap and negative hours are harder than average.