11  SARIMA Model Building and Diagnostics

$$$$ $$$$ $$$$ $$$$ $$$$

11.1 Model building and selection

The richness of the class of (S)AR(I)MA models is a double-edged sword – they are flexible but also difficult to use:

• ARMA has two orders: AR order $p$, MA order $q$;
• ARIMA has one more order: order of difference $d$;
• seasonal ARMA has another two orders: seasonal AR order $P$, seasonal MA order $Q$;
• SARIMA has one more order: order of seasonal difference $D$.

Now we need sensible model-building strategies and model adequacy criteria. Thus we will honor the principle of parsimony: models should be as simple as possible but not simpler.

“All models are wrong but some are useful”

Also, recall that over-fitting and/or over-differencing leads to overly complicated models and increased variance of estimates. Thus we should first plot the data. Then we may transform it to make the variance constant if necessary and also possibly remove deterministic components (trends, seasonality) if appropriate. Now we need to model the stochastic component, which can be achieved with

• differencing appropriately (until stationary, not too much, use unit root tests if uncertain);
• starting with a simple model (white noise);
• performing diagnostics of residuals (acf, pacf, Box test);
• identifying orders and modifying the model (increase model complexity) to remedy;
• continuing until approximately white noise;
• possibly checking the model by slightly overfitting.

Last, but not least, we use the model for prediction. But a question might arise whether we should differentiate or not:

• if the series is non-stationary, differencing may make it stationary;
• if the series is stationary, differencing will preserve stationarity but complicate autocorrelation (non-invertibility).

Recall that the random walk is an AR sequence with $\mathrm{\Phi }(z)=1-z$, hence $\mathrm{\Phi }$ has a unit root ($\mathrm{\Phi }(1)=0$). The presence of a unit root among the roots of the AR polynomial implies non-stationarity and thus the need to differentiate. Methods exist for hypothesis testing about unit roots with hypotheses:

• null hypothesis $H_0$: $1$ is a root;
• alternative $H_1$: all roots our outside the unit circle.

One can use, for example, the Dickey-Fuller test or Phillips-Perron test, which use non-standard distributions of the test statistics (non-stationary data under the null hypothesis) and in R they can be used with PP.test, or adf.test and pp.test in the package tseries.

11.2 Modal diagnostics

For model diagnostics, we consider the standardized residuals $${\hat{\mathrm{e}}}_t=\frac{{\mathrm{X}}_t-{\hat{\mathrm{X}}}_t(\hat{\mathit{\beta }})}{v_{t-1}(\hat{\mathit{\beta }}{)}^{1/2}},$$ where

• $\hat{\mathit{\beta }}$ is an estimator (e.g. maximum likelihood) of all model parameters (AR, MA, coefficients, white noise variance, possibly mean and covariates);
• $v_{t-1}(\hat{\mathit{\beta }})$ is the prediction error for ${\mathrm{X}}_t$ from ${\mathrm{X}}_1,\dots ,{\mathrm{X}}_{t-1}$.

If the right model is used, then for large $n$ the residuals are approximately white noise, so we can plot ACF to check for no correlation or use portmanteau tests (Ljung–Box test) to test the significance of autocorrelations up to some lag (in R use function tsdiag). Also, check for approximate normality of the residuals using the QQ plot, histogram,… (needed for prediction intervals).

11.3 Model order selection

We need an objective, which would quantitatively measure model adequacy because surely bigger models provide a better fit – likelihood or least squares are always better in more complex models. However, there is a price to pay as a more complex model increases the variance of estimates. Also, we need to find a good compromise between fit and variance and penalize for complexity – that is to maximize $$(\text{model fit})-(\text{model complexity penalty}),$$ or minimize $$(\text{model error})+(\text{model complexity penalty}).$$ To perform this optimization, we search in a set of candidate models to optimize the selection criterion. Now it is needed to choose selection criteria – one of the most widespread is the Akaike’s Information Criterion ($\mathrm{AIC}$), which is defined as $$\mathrm{AIC}=-2\mathcal{l}(\hat{\mathit{\beta }})+2r,$$ where

• $\hat{\mathit{\beta }}$ denotes collectively all parameters (AR, MA coefficients, white noise variance, possibly mean and regression coefficients);
• $r=\dim (\mathit{\beta })$ is the number of parameters (e.g. $r=1+1+p+q$ for an $\mathrm{ARMA}(p,q)$ model with mean);
• $\mathcal{l}(\mathit{\beta })=\log f({\mathrm{X}}_1,\dots ,{\mathrm{X}}_n;\mathit{\beta })$ is the $\log$-likelihood.

Our goal is then to select the model that has the smallest value of $\mathrm{AIC}$ (within a set of candidate models, e.g., with orders up to some limits).

Caution

Note that we use selection criteria for models estimated from the same data, in particular, do not compare models with different orders of differencing!

The use of pure $\mathrm{AIC}$ is discouraged as it does not penalize large models enough. Hence the corrected $\mathrm{AIC}$, denoted as ${\mathrm{AIC}}_c$, is recommended $${\mathrm{AIC}}_c=-2\mathcal{l}(\hat{\mathit{\beta }})+2r\frac{n}{n-r-1}.$$

Also one can use $\mathrm{BIC}$ (Bayesian Information Criterion, or Schwarz’s selection rule), which is defined as $$\mathrm{BIC}=-2\mathcal{l}(\hat{\mathit{\beta }})+r\log n,$$ which again strictly penalizes large models and as such tends to select smaller models.

Tip

As a rule of thumb, $\mathrm{AIC}$ or ${\mathrm{AIC}}_c$ is recommended to use for forecasting, whereas $\mathrm{BIC}$ is better used for model estimation.