System
The System page (/system) is the operational health monitor for the EPF forecasting stack. It answers the questions: Is the data pipeline running? Are predictions being written? How accurate is the production model?
Layout
The page is organized into per-country columns — one column for each enabled market (ES, PT, FR, DE). On desktop, all four columns appear side-by-side. On mobile, the active country column is shown and you can switch between them.
Each column has five collapsible sections. A status dot beside each section header summarizes its health at a glance (green = healthy, yellow = degraded, red = failing).
A Refresh button at the top of each column re-fetches all data for that country.
Pipelines
The Pipelines section shows every automated pipeline that feeds the EPF stack, merged from cron job history and Cloud Run Job logs.
Each pipeline row displays:
| Element | Description |
|---|---|
| Name | Pipeline identifier (e.g. entsoe-ingestion, predict-dayahead, news-ingestion) |
| Cadence | How often it runs: hourly, daily, every 15 min, etc. |
| Last run | Timestamp, status badge (success / failed / running), duration, and rows written |
| Next run | Live countdown to the next scheduled execution |
| 7d / 30d success rate | Percentage of runs that completed successfully over the last 7 and 30 days |
| Failure count | Number of failures in the current window |
Expand any row to see a recent run history — a list of the last N executions with individual status, timestamp, duration, and error message (if any).
Prediction pipelines (day-ahead and strategic) run on Cloud Run Jobs triggered by Cloud Scheduler. Data ingestion pipelines run on VM cron. All are tracked in the same Pipelines section.
Data Quality
Per-source data completeness for the selected country. Each row represents one data source that feeds the feature pipeline:
| Element | Description |
|---|---|
| Source name | e.g. REE (ES only), ENTSO-E, Open-Meteo weather, commodity prices |
| Status dot | Green / yellow / red based on completeness threshold |
| Completeness % | Fraction of expected rows present in the last 7 days |
| Completeness bar | Visual progress bar color-coded by threshold (green ≥90%, yellow 70–90%, red below 70%) |
| Actual / expected rows | Raw counts for the measurement window |
| Null count | Number of rows with NULL values in key columns |
| Last timestamp | Most recent data point received |
A completeness drop here — especially in weather or price history — is the first sign of an upstream data collection failure and should be investigated before attributing poor forecast accuracy to the model.
Production Model
Current state of the deployed prediction model for this country.
| Field | Description |
|---|---|
| Tag | Model version tag (e.g. v11.0, v6.0-ablation) |
| Model name | The model_name field used in the predictions table (e.g. xgboost_hybrid15) |
| Status badge | Active / degraded |
| Predictions / 24h | Number of prediction rows written in the last 24 hours |
| Last prediction date | Timestamp of the most recent prediction |
| MAE (30d) | Mean Absolute Error over the last 30 days for this country |
| Bias (30d) | Mean bias (positive = overprediction, negative = underprediction) |
A low Predictions/24h count indicates the Cloud Run Job may not have fired. Check the Pipelines section for the relevant predict-dayahead and predict-strategic job statuses.
Backtest Coverage
The controlled backtest associated with the current production tag. Backtests are the reference measurement for how the model performed before deployment.
| Field | Description |
|---|---|
| Tag | Backtest tag (e.g. xgboost_hybrid15_v11.0_backtest) |
| Period | Start and end dates of the evaluation window |
| Day count | Number of calendar days covered |
| Day-ahead rows | Prediction rows in the backtest for the D+1 product |
| Strategic rows | Prediction rows in the backtest for the D+2–D+7 product |
Evaluation Metrics
Model accuracy metrics for the current production tag, computed against actuals over the backtest window.
Two metric blocks are shown — Day-Ahead and Strategic — each with:
| Metric | Meaning |
|---|---|
| MAE | Mean Absolute Error in EUR/MWh |
| Bias | Mean error (positive = overprediction) |
| Spike Recall | % of hours where actual price was above 200 EUR/MWh that the forecast also flagged as high |
| Spread Capture | % of theoretical BESS (battery storage) arbitrage value captured by a strategy using EPF forecasts |
| n days | Number of evaluation days |
Expand any block to see a monthly breakdown table — MAE and bias per calendar month — which makes it easy to spot whether performance degrades in specific seasons or market regimes.
Interpreting system health
A healthy system state looks like:
- All pipeline rows show green status dots with recent last-run timestamps
- Data quality completeness ≥90% across all sources
- Predictions/24h ≥ 192 (96 × 2 products for 15-min ES; fewer for hourly PT/FR/DE)
- MAE within the expected range for that country and horizon (see Model Performance for per-country benchmarks)
Common failure patterns:
- Missing predictions — check
predict-dayahead/predict-strategicjobs in Pipelines; likely a Cloud Run Job failure or GCS joblib download issue - Low data completeness — check ENTSO-E or REE ingestion pipeline; often a transient API timeout that auto-recovers
- High MAE (30d) — may indicate a market regime shift rather than a pipeline failure; cross-check with the monthly breakdown in Evaluation Metrics
Related sections
- Market Price — the forecast output served by the production model
- Methodology: Cloud Run Jobs — how predictions are generated and delivered
- API: /data/freshness — programmatic staleness check for all data tables