How it works
EPF is a pipeline with five stages. Each one is automated and monitored.
-
Data in. Collectors pull public data on a schedule: market prices, grid demand and generation, official day-ahead wind and solar forecasts, weather forecasts, and fuel and carbon prices. Every run is logged, and a run that writes nothing counts as a failure, not a success. See Data sources.
-
Features. The raw data is turned into the signals a model can use: where prices have been recently, what kind of day is coming, how much demand and renewable output is expected, and what fuel costs. Only information that would really have been published at forecast time is allowed in. See What drives the model.
-
Models. One gradient-boosted tree model family per market and product predicts every future quarter-hour directly from the conditions at forecast time. There is no chain of one-step-ahead predictions, so errors do not compound across the week. See Forecasting approach.
-
Two products. A morning run produces tomorrow’s quarter-hours (day-ahead; 96 on a normal day). An afternoon run, after tomorrow’s real prices are published, produces the six days after that (strategic). Both run as scheduled cloud batch jobs, one per market and product.
-
App and API. Results are stored and served by an API that is separate from the forecasting jobs. The web app reads that API, and falls back to a published snapshot if the API is unavailable, so the latest forecast stays readable.
How a model change reaches production
Section titled “How a model change reaches production”Model changes go through a fixed gate. A candidate is retrained and its forecasts rebuilt month by month over the past year (a walk-forward rebuild), compared with the current production model and with three naive baselines, and checked on economic measures as well as error. An automated research pipeline proposes and tests candidates; a human approves every promotion. After promotion, the live forecast is checked to confirm the model receives the same inputs in production that it was evaluated with. That check is in place for Spain and Portugal; for France and Germany some inputs still arrive blank in production, and the check is being extended to them.