Forecasting bake-off

Don't guess which forecasting method to trust. Backtest all four (naive/drift, SES, Holt, Holt-Winters) on a real series, rank by out-of-sample RMSE, then forecast forward with the winner and its 95% prediction interval. Method selection without the hand-waving.

When to use this pack

You need a forecast and you're not sure whether the series is stationary, trending, or seasonal. Instead of picking a method by gut and praying, the bake-off lets the data choose: every method runs the same holdout backtest, the lowest RMSE wins, and you forecast forward with that winner only. Pure-CPU and free over PoW — only the upstream data fetch is paid.

Tools in this pack

• Stock historical bars $0.015 GET /api/stock-history Historical OHLCV bars for a symbol. Configurable interval (1m, 5m, 15m, 30m, 60m, 1d, 1wk, 1mo, 3mo) and range (1d, 5d, 1mo, 3mo, 6mo, 1y, 2y, 5y, 10y, ytd, max). Intraday intervals are limited by Yahoo to ~60 days of data. Returns a flat array of bars (time, open, high, low, close, volume) ready for charting or backtests.
• FRED time series $0.015 GET /api/fred-series Fetch any of FRED's ~800,000 economic time series by series ID — GDP (GDPC1), CPI (CPIAUCSL), unemployment (UNRATE), fed funds (DFF), and so on. Supports date windowing and the standard FRED units transformations (lin, chg, ch1, pch, pc1, pca, cca, log). ?seriesId=GDPC1&startDate=2018-01-01&endDate=2023-12-31&units=pc1
• Forecast backtest (MAPE + RMSE) $0.001 POST /api/forecast-eval Backtest a forecasting method on the input series by holding out the last `testSize` observations, forecasting them, and computing MAPE (mean absolute percentage error) + RMSE (root mean squared error). Lets an agent pick which method (mean / naive / drift / ses / holt / holt-winters) actually fits its data before committing to a forward forecast. Always returns a `warnings` array — empty when the backtest is well-posed, populated when `testSize` exceeds n/2 (treat error as indicative not predictive).
• Forecast (naive baselines) $0.001 POST /api/forecast-naive Three textbook baseline forecasts: mean (forecast = average of history), naive (forecast = last value), drift (linear extrapolation from first to last point). Use as a sanity floor — any sophisticated method (SES, Holt, Holt-Winters) should beat the best of these on a backtest, otherwise the extra complexity isn't earning its keep. Returns point forecasts + 95% prediction intervals per Hyndman §3.1.
• Forecast (simple exponential smoothing) $0.001 POST /api/forecast-ses Simple exponential smoothing (SES) — level-only forecast for series without trend or seasonality. Higher alpha (closer to 1) tracks recent values aggressively; lower alpha (closer to 0) smooths through noise. Default alpha=0.3 is a common conservative pick; pass an explicit alpha or use forecast-eval to pick the one that minimizes backtest error. Forecast is flat (= last fitted level) for all horizons.
• Forecast (Holt linear trend) $0.001 POST /api/forecast-holt Holt's linear trend method — level + trend (no seasonality). Two smoothing parameters: alpha (level) and beta (trend). Forecast extrapolates as a straight line from the last fitted level along the last fitted trend, so it grows or shrinks linearly with horizon. Use this when your series has a persistent up/down trend but no seasonal cycle (e.g. a SaaS MRR climb, a deflating cohort retention curve).
• Forecast (Holt-Winters seasonal) $0.001 POST /api/forecast-holt-winters Holt-Winters triple exponential smoothing — level + trend + seasonal component. Use for series with a repeating cycle (weekly retail traffic, monthly utility usage, quarterly revenue). Additive seasonality (constant amplitude) or multiplicative (amplitude grows with level). `period` is optional — if omitted, the kit auto-detects via autocorrelation on first differences and surfaces what it picked (with the ACF strength) so you can audit. Needs at least two full seasonal cycles to fit reliably.

Workflow

1. Fetch the equity series with stock-history (ticker, range=horizon-scaled — e.g. "2y" if you want to forecast ~6 months out). Pull `close` in chronological order; you want at least ~50 observations for the backtest to be meaningful, more if you suspect seasonality.
2. If the user is asking about a macro indicator instead (unemployment, CPI, fed funds), fetch via fred-series with the series id. Monthly FRED data with 10+ years of history is the sweet spot for Holt-Winters with period=12.
3. Run the bake-off. Call forecast-eval four times on the same values with testSize ≈ 20% of the series (capped at half): method="naive" or "drift", "ses", "holt", "holt-winters" (the last only if you have ≥ 2·period observations and suspect seasonality). Compare RMSE; lowest wins. Watch the `warnings` field — "insufficient data" or "could not detect seasonal period" means treat that method's score as suspect, not as a clean win/loss.
4. If forecast-naive (or drift, the mean-reversion variant) won, the series is essentially random-walk and there's nothing to extrapolate — call forecast-naive with the full values + horizon. The point forecast is just the last value (or last + average drift); the interval widens with √h. This is the honest answer for noisy series; don't over-engineer.
5. If forecast-ses won, the series has no trend but local level matters more than the long-run mean. Call forecast-ses with the full values + horizon; the alpha SES picked tells you how much weight goes on recent vs. older observations (high alpha = react fast, low alpha = smooth heavy). Report alpha alongside the forecast — it's diagnostic.
6. If forecast-holt won, the series has a persistent trend worth extrapolating. Call forecast-holt with full values + horizon; it returns level + trend smoothing parameters (alpha, beta) and a forecast that walks forward at the fitted trend slope. The 95% interval grows faster than SES because trend uncertainty compounds.
7. If forecast-holt-winters won, the series has seasonality you should respect (e.g. monthly macro with annual cycle, quarterly retail with year-end peak). Call forecast-holt-winters with the full values + horizon + period (12 for monthly-annual, 4 for quarterly-annual, 7 for daily-weekly) and seasonality="additive" or "multiplicative". The forecast carries the seasonal pattern forward; never report the point forecast without the interval — seasonal forecasts look confident but compound multiple sources of error.

Run it in Claude

claude mcp add agent402 -s user -- npx -y agent402-mcp@latest

Then paste this prompt into Claude:

Run a forecasting bake-off on AAPL over the last 2y and project the next 30 trading days using Agent402. (1) Fetch the daily closes via stock-history (ticker=AAPL, range=2y). (2) Run forecast-eval four times on the closes with testSize=100: method="drift", "ses", "holt", and "holt-winters" with period=21 and seasonality="multiplicative" (try the seasonal one — equities usually don't have strong calendar seasonality but the backtest will tell you). (3) Rank by RMSE ascending; the lowest is the winner. Note any `warnings` returned. (4) Call the winning forecast tool (forecast-naive / forecast-ses / forecast-holt / forecast-holt-winters) with the full closes + horizon=30 to get the forward forecast and 95% interval. (5) Return a single JSON object: {rankings: [{method, rmse, mape, warnings}, ...], winner: "holt", forecast: {point: [...], lower95: [...], upper95: [...]}, oneLineConclusion}. All bake-off + forecast calls are free over PoW; only stock-history is paid.

← All skill packs