Chapter 03 · Integration of order d

ARIMA, when the series has to be tamed first

Most real series are not stationary out of the box. ARIMA is the family that differences first, then fits an ARMA on what is left. Pick the integration order well, and the rest follows. Pick it wrong, and even AIC will mislead you.

ARIMA notation

Difference$\Delta y_t = y_t - y_{t-1}$, then $\Delta^2 y_t = \Delta(\Delta y_t)$

ARIMA(p,d,q)Apply $\Delta^d$, then fit $\text{ARMA}(p,q)$ to the result

Lag form$(1 - \phi_1 L - \cdots - \phi_p L^p)(1-L)^d\,y_t = (1 + \theta_1 L + \cdots + \theta_q L^q)\,\varepsilon_t$

Stationaritytested by ADF (Augmented Dickey-Fuller) on the differenced series

The AIC trap AIC is comparable across models fitted to the same data. Different values of d produce models fitted to different series, the original, the once-differenced, the twice-differenced. Their likelihoods are computed on different sample sizes and different scales. Comparing AIC across d is therefore not meaningful, and that is the trap. Always pick d first, by ADF or by inspecting the differenced series, then compare AIC only within that d.

Pick a series, then an order

Eight series chosen to put differencing in its place. Notice when the original is stationary, when one difference suffices, and when even two does not save you from a seasonal mismatch.

Series ARIMA(p, d, q)

ADF test on the original series

AIC and BIC, grouped by d. Within-d comparable, across-d misleading.

Order (p, d, q)	AIC	BIC	RMSE

Verdict

Pick a series and a model order to see the verdict.

Five things worth knowing

Click to reveal the practitioner-level reasoning.

How do I pick d in practice?

Run ADF on the original series. If you reject the unit root null, set d=0. Otherwise difference and run ADF again. Repeat at most twice; almost no real series needs d > 2. Differencing more than necessary inflates the variance of the differenced series and washes signal away.

Why is AIC across d misleading?

Each value of d produces a different sample. ARIMA(p,0,q) fits to all n observations of the original, ARIMA(p,1,q) fits to n−1 observations of the differenced series, ARIMA(p,2,q) fits to n−2. The likelihoods are not on the same scale because the data are not the same. The fix is to pick d first, by ADF and visual inspection, then compare AIC only within that d.

What does the ADF test actually do?

It regresses Δy_t on y_{t−1} plus a few lagged differences, and tests whether the coefficient on y_{t−1} is significantly negative. A negative coefficient means deviations from the mean revert; non-rejection of zero means the series is consistent with a unit root, that is, non-stationary. The augmented part adds the lagged differences to absorb autocorrelation in the errors, which would otherwise bias the test.

When does d=2 actually help?

When the trend itself is changing. UK monthly retail sales over fifty years have an accelerating curve, and a single difference still leaves a slow upward drift. A second difference flattens it. For most economic and demand series, d=1 is enough. d=2 is uncommon and signals you are looking at a series with a structural break or compounding behaviour, and a careful analyst would investigate before fitting.

What if the residuals show seasonality?

Then ARIMA is the wrong tool. Differencing once removes a stochastic trend but leaves a seasonal cycle alone. The verdict in that case will say so explicitly and point to SARIMA(X), where seasonal differencing of the form (1 − L^m) is the right operation. An honest residual ACF that shows a spike at lag m is your invitation to the next chapter.