Time Series Forecasting

Time series data is indexed by time, and time changes everything. Observations are ordered, autocorrelated, and often trend and exhibit seasonality — weekly demand cycles, monthly traffic peaks. Standard machine learning's assumption of independent, identically distributed samples breaks down, so forecasting uses its own validation scheme, its own feature engineering, and models designed for dependence over time.

Stationarity and Decomposition

A stationary series has constant mean and variance over time; most classical models require it (or require stable differences of it). Decompose a series into trend, seasonal, and residual components to see what you actually need to model. If the series wanders, differencing usually stabilizes it; the ADF test provides a formal check.

import pandas as pd
from statsmodels.tsa.seasonal import seasonal_decompose

y = df["sales"].asfreq("D").interpolate()
result = seasonal_decompose(y, model="additive", period=7)
result.plot()

d1 = y.diff().dropna()  # first difference to remove trend

Classical Models: ARIMA Family

ARIMA models today's value as a linear function of past values (autoregressive part), past forecast errors (moving-average part), and past differences (integrated part). SARIMA adds seasonal terms. The train/test split must respect order — never shuffle time series rows.

from statsmodels.tsa.arima.model import ARIMA
from sklearn.metrics import mean_absolute_error

n_test = 30
train, test = y.iloc[:-n_test], y.iloc[-n_test:]

model = ARIMA(train, order=(2, 1, 2)).fit()
pred = model.forecast(n_test)
print("MAE:", mean_absolute_error(test, pred))
print(pred.head())

Supervised Learning on Time Series

Any regression model becomes a forecaster once you feed it lag features and calendar features. This is the workhorse approach in industry because it lets random forests and gradient boosting handle nonlinearity and extra predictors (weather, promotions) side by side with history.

def make_lags(series, n_lags):
    return pd.concat({f"lag_{i}": series.shift(i) for i in range(1, n_lags + 1)},
                     axis=1)

X = make_lags(y, 7)
X["dow"] = y.index.dayofweek
X["month"] = y.index.month
y_target = y.shift(-1).dropna()
X = X.loc[y_target.index].dropna()

split = int(0.8 * len(X))
X_tr, X_te = X.iloc[:split], X.iloc[split:]
y_tr, y_te = y_target.iloc[:split], y_target.iloc[split:]

from sklearn.ensemble import HistGradientBoostingRegressor
gbr = HistGradientBoostingRegressor().fit(X_tr, y_tr)
print("MAE:", abs(gbr.predict(X_te) - y_te).mean())

Multi-Step Forecasting

Forecasts beyond one step need either recursive strategy (predict a step, append it to the lag window, repeat) or a direct strategy (train one model per horizon). Recursive models accumulate error with each step; direct models cost more to train but stay unbiased per horizon. Evaluate with TimeSeriesSplit, which walks a sliding window forward and can never peek at the future.

from sklearn.model_selection import TimeSeriesSplit, cross_val_score

tscv = TimeSeriesSplit(n_splits=5, test_size=14)
scores = cross_val_score(HistGradientBoostingRegressor(), X, y_target,
                         cv=tscv, scoring="neg_mean_absolute_error")
print("MAE by fold:", -scores)

Deep learning options round out the toolkit: LSTM or temporal convolutional networks in Keras treat a window of history as a sequence input, and Prophet offers a decomposed, interpretable curve fit that non-technical stakeholders can read directly.

Summary

Time series forecasting respects temporal order at every step: decompose and difference for stationarity, fit ARIMA-family models for compact classical baselines, and engineer lag and calendar features to unleash gradient boosting on the same problem. Validate with time-aware splits such as TimeSeriesSplit, and choose recursive or direct strategies for multi-step horizons depending on how error compounds across the forecast.