Split Conformal Prediction

Overview

Split conformal prediction generates confidence intervals around point forecasts without assuming any particular error distribution. It works by collecting prediction residuals during cross-validation and using their empirical distribution to set interval bounds at prediction time.

The Method in Three Steps

Step 1: Collect Calibration Residuals

During 5-fold cross-validation, each validation fold produces out-of-fold predictions. The signed residuals are collected:

residual = actual_price - predicted_price

These residuals form the calibration set. Using out-of-fold (not in-sample) residuals is critical — in-sample residuals would be optimistically small because the model has already seen those examples.

Step 2: Group by Horizon Bucket

Residuals are grouped by forecast horizon because uncertainty varies with prediction distance:

Bucket	Horizons	Residual Count
DA1	14–25 hours ahead	~2,000–4,000
DA2	26–37 hours ahead	~2,000–4,000
S1	33–56 hours ahead	~4,000–8,000
S2	57–80 hours ahead	~4,000–8,000
…	…	…
S5	129–176 hours ahead	~8,000–16,000

Each bucket has its own residual distribution, capturing the specific uncertainty profile of that horizon range.

Step 3: Compute Quantile Bounds

For each confidence level, quantile bounds are computed from the bucket’s residuals:

90% confidence interval:

lower_shift = quantile(residuals, 0.05)    (5th percentile)
upper_shift = quantile(residuals, 0.95)    (95th percentile)

50% confidence interval:

lower_shift = quantile(residuals, 0.25)    (25th percentile)
upper_shift = quantile(residuals, 0.75)    (75th percentile)

At prediction time:

lower_bound = prediction + lower_shift
upper_bound = prediction + upper_shift

Why “Split” Conformal?

The term “split” refers to the data splitting strategy: the calibration residuals come from a held-out portion of the training data (the cross-validation folds), not from the same data used to train the model. This split prevents the model from being overconfident about its own training data.

Full conformal prediction would retrain the model for every new test point, which is computationally infeasible. Split conformal sacrifices a small amount of calibration data efficiency for massive computational savings.

Coverage Guarantee

Conformal prediction provides a theoretical guarantee: for a calibration set of size n, the coverage rate of a (1-α) interval satisfies:

P(actual ∈ interval) ≥ 1 - α - 1/(n+1)

With n = 2,000+ residuals per bucket, the correction term 1/(n+1) is negligible, and empirical coverage converges to the target level.

Price Floor

A practical constraint: predicted lower bounds are floored at -50 EUR/MWh:

lower_bound = max(prediction + lower_shift, PRICE_FLOOR)

While negative prices are possible in the Spanish market, prices below -50 EUR/MWh are extremely rare. The floor prevents the lower bound from extending to unrealistic values.

Two Confidence Levels

The system produces two nested intervals:

Point prediction:     52.3 EUR/MWh
50% CI:              [44.2, 60.4]      (inner band)
90% CI:              [35.1, 68.5]      (outer band)

The 50% CI represents the most likely price range; the 90% CI captures all but extreme surprises. Visualized on charts, the inner band appears darker and the outer band lighter, creating an intuitive uncertainty “funnel.”

Comparison with Alternatives

Method	Distribution Assumption	Coverage Guarantee	Computation
Split conformal	None	Yes (asymptotic)	Low (quantiles only)
Gaussian intervals	Normal distribution	Only if Normal	Low
Bootstrap	None (resampling)	Approximate	High (1000+ resamples)
Quantile regression	None	No	Medium (extra models)
Bayesian credible	Prior specification	Conditional on prior	High

Split conformal offers the best trade-off: no distributional assumptions, theoretical guarantees, and minimal computation.