System architecture

ingest → store → ML → serve

Production ML on Google Cloud · Spain, Portugal, France and Germany · 15-minute prices, up to 7 days ahead

Click a logo for its stack & why · hover to highlight
Production · always on
Data Sources
Ingestion & Features
Production DB
Backend API
Edge / Delivery
Frontend & Users
Production · daily forecast
Trigger
Inference
Model Registry
Local / dev · offline
Development DB
Training
Build & Deploy

How it works

The system spans two environments and two PostgreSQL databases: a development one where models are built, and a production one where they run. Four processes operate across them at different cadences — separating them is the key to understanding the architecture:

1Data pipeline
continuous

Collectors pull prices, weather, generation, commodities and news on a schedule (hourly down to every 15 minutes) and land them in PostgreSQL. This never stops: it is the always-on supply of fresh inputs.

2Model training
on demand · offline

Runs offline, when an experiment or a retrain is due. Each market’s model trains on the full history in the development database and is promoted to Cloud Storage only after it beats the current model in a walk-forward test. Occasional, not part of the daily loop (dashed).

3Daily prediction
scheduled · ~10:00 & ~15:10 UTC

Every morning the server retrains the four day-ahead models and forecasts tomorrow; in the afternoon Cloud Scheduler fires 4 Cloud Run jobs (the strategic forecasts). Each loads its market’s model from Cloud Storage, reads the latest data from the production database, and writes the forecasts back to it. After each wave the server’s caches are refreshed on a schedule and a snapshot job publishes a static copy; once a day a report job renders the PDF report.

4Serving
per request

When a visitor opens the dashboard, Vercel serves the app from the edge, the API returns the stored forecasts from its cache, and the React app renders them. No model runs here: it serves what the daily jobs already computed.

The bridge between the two worlds is the Model Registry (Cloud Storage): training builds a model from the local database and ships it there; the daily jobs load it and write forecasts into the production database, which is all the dashboard ever reads. The model never runs when you open the page — you see what the scheduled jobs already computed.