AR & ARMA Models - Statistics Graduate Level

1. A process is represented by $X_t-0.7X_{t-1}=\varepsilon_t-0.7\varepsilon_{t-1},$ where $\{\varepsilon_t\}$ is white noise. Assuming the stationary causal solution is intended, which statement is correct?
   Answer: $X_t$ is white noise because the common autoregressive and moving-average factors cancel When you encounter an ARMA model, your first instinct should be to factor the lag polynomials and check for common roots — this is the key concept being tested here. The process can be written using the backshift operator $B$ as $(1 - 0.7B)X_t = (1 - 0.7B)\varepsilon_t$. Notice that both sides share the identical factor $(1 - 0.7B)$. Because the stationary causal solution is assumed, this common factor cancels legally, leaving simply $X_t = \varepsilon_t$. This means $\{X_t\}$ is white noise — serially uncorrelated, with constant variance and zero mean. So A is correct. B is wrong because focusing only on the left-hand side ignores the right-hand side entirely. The AR polynomial does not act in isolation; the full operator equation must be considered. C is wrong for the symmetric reason — focusing only on the MA side misses the cancellation. Even though $-0.7$ appears as the MA coefficient, the process after cancellation is not an MA(1). D is particularly tempting but incorrect: it imagines that cancellation affects variance while leaving autocorrelation structure intact. In reality, cancellation eliminates all serial correlation — after simplification, $X_t = \varepsilon_t$ has zero autocorrelation at every lag $k \geq 1$, not $0.7^k$. A useful rule of thumb: common factor = parameter redundancy = simplification. Whenever an ARMA model has matching AR and MA roots, the model is over-parameterized and reduces to a simpler process. Always factor both lag polynomials before interpreting any ARMA equation.
2. Consider the stationary ARMA model $X_t=0.5X_{t-1}+\varepsilon_t-0.8\varepsilon_{t-1},$ where the innovations have variance $\sigma^2.$ Which pair gives the first two autocorrelations?
   Answer: $\rho_1=-\dfrac{3}{14}$ and $\rho_2=-\dfrac{3}{28}$ When working with ARMA(1,1) models, your goal is to derive autocorrelations using the Yule-Walker framework combined with the autocovariance structure of the moving average component. For $X_t = \phi X_{t-1} + \varepsilon_t - \theta\varepsilon_{t-1}$ with $\phi = 0.5$ and $\theta = 0.8$, first compute the variance $\gamma_0$ and lag-1 autocovariance $\gamma_1$ directly. Multiplying both sides by $X_t$ and $X_{t-1}$ respectively and taking expectations: $\gamma_0 = \phi\gamma_1 + \sigma^2(1 + \theta^2) - \phi\theta\sigma^2\cdot...$ Using the known closed-form results: $\gamma_0 = \frac{(1 - 2\phi\theta + \theta^2)\sigma^2}{1-\phi^2} = \frac{(1 - 2(0.5)(0.8) + 0.64)\sigma^2}{0.75} = \frac{0.84\sigma^2}{0.75} = \frac{56\sigma^2}{50}$ $\gamma_1 = \frac{(\phi - \theta)(1 - \phi\theta)\sigma^2}{1 - \phi^2} = \frac{(-0.3)(0.6)\sigma^2}{0.75} = \frac{-0.18\sigma^2}{0.75} = \frac{-12\sigma^2}{50}$ So $\rho_1 = \gamma_1/\gamma_0 = -12/56 = -3/14$. For lags $k \geq 2$, the ACF satisfies the pure AR(1) recursion: $\rho_k = \phi\rho_{k-1}$, giving $\rho_2 = 0.5 \times (-3/14) = -3/28$. This confirms answer A. Answer B gets $\rho_1$ right but incorrectly makes $\rho_2$ positive — a sign error in the recursion. Answer C applies the wrong formula, likely confusing $\phi - \theta$ terms. Answer D simply reports $\phi$ and $\phi^2$, ignoring the MA component entirely — a common trap when students forget ARMA autocorrelations differ from pure AR. Remember: for ARMA(1,1), only $\rho_1$ has a special formula; all higher lags follow $\rho_k = \phi\rho_{k-1}$.
3. Which statement correctly describes the second-order dependence of the observed process $\{Y_t\}$?
   Answer: $\rho_1=0.25$ and $\rho_2=0.125;$ these autocorrelations are consistent with an ARMA structure but not with a pure AR(1) model When a latent AR(1) signal is observed through additive noise, the resulting process $\{Y_t\}$ is neither pure AR(1) nor white noise — it becomes an ARMA(1,1) process. Recognizing this structure is the key to the question. Start by computing the marginal variance of $X_t$. For a stationary AR(1) with coefficient $\phi = 0.5$ and innovation variance $\sigma_\varepsilon^2 = 3$, you get $\text{Var}(X_t) = \frac{3}{1 - 0.25} = 4$. The marginal variance of $Y_t$ is then $\text{Var}(Y_t) = 4 + 4 = 8$. For lag-1, the autocovariance of $Y_t$ equals that of $X_t$ (measurement noise terms are independent across time): $\gamma_Y(1) = \gamma_X(1) = \phi \cdot \text{Var}(X_t) = 0.5 \times 4 = 2$, giving $\rho_1 = 2/8 = 0.25$. At lag-2, $\gamma_Y(2) = \gamma_X(2) = \phi^2 \cdot \text{Var}(X_t) = 0.25 \times 4 = 1$, giving $\rho_2 = 1/8 = 0.125$. Notice that $\rho_2 / \rho_1 = 0.5 = \phi$, which is consistent with the AR part of an ARMA(1,1) — but the ratio $\rho_1 \neq \phi$, ruling out a pure AR(1). This confirms C is correct. A is wrong because while $\rho_1 = 0.25$ is correct, $\rho_2 = 0.0625$ would require $\rho_2 = \rho_1^2$, the signature of a pure AR(1) — which this process is not. B is wrong because measurement noise does attenuate autocorrelations by inflating the denominator (marginal variance) without affecting the numerator. D is wrong because independent noise kills contemporaneous correlation but not lagged cross-signal dependence, which persists through $X_t$. Your study tip: whenever you add independent measurement noise to an AR(1), immediately check whether the resulting ACF pattern still satisfies the pure AR(1) Yule-Walker equations. If not, suspect ARMA.
4. The noninvertible moving-average process $X_t=\varepsilon_t+2\varepsilon_{t-1}$ has innovation variance $1.$ Which invertible MA representation has the same autocovariance function?
   Answer: $X_t=u_t+0.5u_{t-1},$ with innovation variance $4$ When you encounter a noninvertible MA process, the key concept to recall is invertibility duality: any MA(1) process $X_t = \varepsilon_t + \theta\varepsilon_{t-1}$ with $|\theta| > 1$ shares its entire autocovariance function with an invertible process using coefficient $1/\theta$ — but the innovation variance must be rescaled to preserve those autocovariances. Here's the mechanics. For the given process $X_t = \varepsilon_t + 2\varepsilon_{t-1}$ with $\sigma^2 = 1$, compute the autocovariances: $\gamma(0) = (1 + 4)\cdot 1 = 5$ and $\gamma(1) = 2 \cdot 1 = 2$. The invertible partner replaces $\theta = 2$ with $1/\theta = 0.5$. For the new process $X_t = u_t + 0.5u_{t-1}$ with variance $\sigma_u^2$, you need $\gamma(0) = (1 + 0.25)\sigma_u^2 = 5$, giving $\sigma_u^2 = 4$, and $\gamma(1) = 0.5\sigma_u^2 = 2$ ✓. This confirms answer A is correct. Answer B uses the right coefficient $0.5$ but keeps the variance at $1$, producing $\gamma(0) = 1.25 \neq 5$ — the autocovariance structure is completely wrong. Answer C introduces a sign flip to $-0.5$, which changes $\gamma(1)$ to a negative value, contradicting the original process. Answer D keeps $\theta = 2$ (still noninvertible) and tries to fix things with $\sigma_u^2 = 0.25$, which produces $\gamma(0) = 1.25 \neq 5$ — wrong scaling entirely. Study tip: Always verify both autocovariances $\gamma(0)$ and $\gamma(1)$ match after the duality substitution. The rescaling rule is $\sigma_u^2 = \theta^2 \sigma_\varepsilon^2$, so swapping $\theta \to 1/\theta$ requires multiplying the variance by $\theta^2$.
5. Suppose $X_t=0.6X_{t-1}+\varepsilon_t,$ where $\{\varepsilon_t\}$ is weak white noise with finite variance: distinct innovations are uncorrelated, but they are not assumed independent or Gaussian. For the causal stationary solution, which claim is justified by these assumptions alone?
   Answer: $X_t$ is weakly stationary with autocorrelation $\rho_k=0.6^{|k|},$ but it need not be Gaussian or Markov When analyzing an AR(1) process under weak white noise assumptions, you need to carefully track exactly what the assumptions give you — and what they don't. For the causal stationary solution $X_t = \sum_{j=0}^{\infty} 0.6^j \varepsilon_{t-j}$, weak stationarity follows directly: the mean is zero, variance is $\sigma^2/(1-0.36)$, and the autocovariance depends only on lag. The autocorrelation function is $\rho_k = 0.6^{|k|}$, derived purely from the uncorrelatedness of $\{\varepsilon_t\}$ and the geometric decay of coefficients. This makes B correct — weak stationarity and $\rho_k = 0.6^{|k|}$ are fully justified, but nothing forces Gaussianity or the Markov property. A fails on multiple grounds. Strict stationarity requires the joint distributions to be shift-invariant, which demands independence of innovations, not merely uncorrelatedness. Gaussianity requires either Gaussian inputs or a CLT argument with additional conditions. The Markov property requires that the conditional distribution of $X_t$ given the entire past depends only on $X_{t-1}$ — true for Gaussian AR(1), but under weak white noise, higher-order conditional moments may still depend on further lags. C is wrong because the autocorrelation is not zero at nonzero lags. Zero autocorrelation would mean the process itself is white noise, which it isn't — the geometric decay $0.6^{|k|}$ reflects the persistence introduced by the autoregressive structure. D confuses a necessary condition with a sufficient one. Having $\rho_k = 0.6^{|k|}$ characterizes the second-order structure, but the Markov property is a statement about conditional distributions — second-order properties alone cannot guarantee it without Gaussianity. Strategy tip: On time series questions, always ask: "Are my assumptions about distributions or just second moments?" Weak white noise only controls correlations, so properties requiring independence or distributional form (Markov, Gaussian, strict stationarity) need stronger assumptions.
6. A stationary process has population autocorrelations $\rho_1=0.40,$ $\rho_2=0.28,$ and, for every $k\geq 2,$ satisfies $\rho_{k+1}=0.70\rho_k.$ Which conclusion best distinguishes an AR model from an ARMA model?
   Answer: The process cannot be AR of order one because its lag-two partial autocorrelation is nonzero, but the pattern is compatible with ARMA of order one-one When identifying time series models, your key diagnostic tools are the autocorrelation function (ACF) and the partial autocorrelation function (PACF). A pure AR(1) process produces geometric decay in the ACF starting from lag one, and its PACF cuts off sharply after lag one — meaning the lag-two partial autocorrelation equals exactly zero. Here, the ACF does decay geometrically, but only after lag one: $\rho_2 = 0.28 = 0.70 \times 0.40 = 0.70\rho_1$, and $\rho_{k+1} = 0.70\rho_k$ for $k \geq 2$. Notice that if this were a pure AR(1) with coefficient $0.70$, you'd expect $\rho_1 = 0.70$ and $\rho_2 = 0.70^2 = 0.49$. The observed $\rho_1 = 0.40$ doesn't fit that mold. This "delayed" geometric start is the signature of an ARMA(1,1) process, where the moving-average component distorts the first autocorrelation before the AR component takes over. Crucially, an AR(1) would force the lag-two partial autocorrelation to zero, but because the lag-two ACF isn't consistent with a pure AR(1), the PACF will not cut off cleanly — confirming the process cannot be AR(1). Answer C captures this precisely. Answer A is wrong because the MA(1) ACF cuts off after lag one entirely — it cannot produce the persistent geometric decay seen here. Answer B incorrectly assumes the geometric decay parameter equals the AR(1) coefficient; it ignores that $\rho_1 = 0.40 \neq 0.70$. Answer D is a red herring — nothing about these autocorrelations violates stationarity conditions. Study tip: When ACF decays geometrically but lag one is "off," immediately think ARMA(1,1). The MA component shifts the initial value without destroying the long-run AR pattern.
7. An analyst fits the model $X_t=1.2+0.8X_{t-1}+\varepsilon_t.$ Assuming stationarity, which pair gives the unconditional mean and the cumulative effect on the expected future path of a one-unit innovation occurring at time $t$?
   Answer: Unconditional mean $6$; cumulative expected effect $5$ When you see an AR(1) model with a constant, two separate mechanics are being tested: the long-run equilibrium level and how a shock propagates through time. For an AR(1) model $X_t = c + \phi X_{t-1} + \varepsilon_t$, the unconditional mean is found by setting $E[X_t] = E[X_{t-1}] = \mu$ and solving: $\mu = c + \phi\mu \Rightarrow \mu = \frac{c}{1-\phi}.$ Here, $\mu = \frac{1.2}{1-0.8} = \frac{1.2}{0.2} = 6.$ So the unconditional mean is 6. For the cumulative effect of a one-unit innovation, recognize that a shock at time $t$ decays geometrically. The impulse response at horizon $h$ is $\phi^h$, so the total cumulative effect across all future periods is the sum of a geometric series: $\sum_{h=0}^{\infty} \phi^h = \frac{1}{1-\phi} = \frac{1}{1-0.8} = 5.$ This confirms answer B: unconditional mean 6, cumulative effect 5. Answer A gets the cumulative effect right (5) but incorrectly uses the raw constant 1.2 as the mean — confusing $c$ with $\mu$. Answer C correctly identifies the mean as 6 but reports a cumulative effect of 4, which would correspond to $\phi/(1-\phi)$, mistakenly starting the sum at $h=1$ instead of $h=0$ (omitting the contemporaneous impact). Answer D swaps the two values entirely, reversing mean and cumulative effect. A useful memory anchor: both the unconditional mean formula and the cumulative impulse response share the same denominator, $1-\phi$. If you internalize $\frac{1}{1-\phi}$ as the "AR(1) multiplier," you can quickly compute both quantities without mixing them up.
8. Suppose $X_t=0.6X_{t-1}+\varepsilon_t+0.4\varepsilon_{t-1},$ where the innovations are independent with variance $1.$ At time $t,$ the observed values needed for forecasting are $X_t=1$ and $\varepsilon_t=0.5.$ Which triple gives the one-step forecast, the two-step forecast, and the two-step forecast-error variance?
   Answer: $0.80,\ 0.48,\ 2.00$ When you see an ARMA(1,1) model, the key is remembering that forecasts beyond one step ahead lose the moving-average correction term, since future innovations have conditional expectation zero. The model is $X_t = 0.6X_{t-1} + \varepsilon_t + 0.4\varepsilon_{t-1}$. The one-step forecast from time $t$ is: $\hat{X}_{t+1} = 0.6X_t + 0.4\varepsilon_t = 0.6(1) + 0.4(0.5) = 0.60 + 0.20 = 0.80.$ For the two-step forecast, you take the conditional expectation of $X_{t+2} = 0.6X_{t+1} + \varepsilon_{t+2} + 0.4\varepsilon_{t+1}$. Since $E[\varepsilon_{t+1}] = E[\varepsilon_{t+2}] = 0$, only the AR part survives: $\hat{X}_{t+2} = 0.6\hat{X}_{t+1} = 0.6(0.80) = 0.48.$ For the two-step forecast-error variance, write $X_{t+2} - \hat{X}_{t+2} = \varepsilon_{t+2} + (0.6 + 0.4)\varepsilon_{t+1} = \varepsilon_{t+2} + \varepsilon_{t+1}$. Each innovation has variance 1, so the total is $1 + 1^2 = 2.00$. This gives the triple $(0.80,\ 0.48,\ 2.00)$, confirming D. Choice A uses $1.16$, which incorrectly blends MA coefficients into a one-step error variance formula. Choice B gets $1.36$, a common error from squaring only the MA(1) coefficient $(1 + 0.4^2 = 1.16)$ or using a misremembered recursion. Choice C uses $0.60$ as the one-step forecast, forgetting to include the $0.4\varepsilon_t$ correction available at time $t$. Study tip: For ARMA forecast-error variance, build it step by step using the MA($\infty$) representation — the $h$-step variance is $\sigma^2\sum_{j=0}^{h-1}\psi_j^2$, where $\psi_j$ are the impulse-response coefficients.
9. A stationary process satisfies $X_t=2+0.6X_{t-1}+\varepsilon_t,$ where the innovations are independent with mean zero and variance $4.$ If $X_t=8,$ which pair gives the optimal linear forecast of $X_{t+2}$ and its forecast-error variance?
   Answer: Forecast $6.08$; forecast-error variance $5.44$ When forecasting from an AR(1) model, you apply the model recursively and track how forecast error variance accumulates with each additional step ahead. Start by finding the process mean. Setting $E[X_t] = \mu$ gives $\mu = 2 + 0.6\mu$, so $\mu = 5$. The one-step-ahead forecast is straightforward: $\hat{X}_{t+1} = 2 + 0.6(8) = 6.8$, with error variance equal to the innovation variance, $\sigma^2 = 4$. For the two-step forecast, substitute the one-step forecast into the model: $\hat{X}_{t+2} = 2 + 0.6\hat{X}_{t+1} = 2 + 0.6(6.8) = 2 + 4.08 = 6.08$. The two-step forecast error variance compounds across both steps: $\text{Var}_{t+2} = \sigma^2(1 + \phi^2) = 4(1 + 0.36) = 4(1.36) = 5.44$. This confirms answer A is correct. Choice B reverses the forecasts — it reports the one-step forecast (6.80) but pairs it with the two-step error variance (5.44), mixing horizons incorrectly. Choice C gets the two-step point forecast right (6.08) but uses only $\sigma^2 = 4.00$, forgetting that a second period of uncertainty from the first propagated error must be added. Choice D uses the one-step forecast again (6.80) and an incorrect variance of 6.40, which doesn't correspond to any standard formula. A useful strategy: always track the forecast horizon carefully. The $h$-step forecast error variance for an AR(1) is $\sigma^2 \sum_{j=0}^{h-1} \phi^{2j}$, not simply $h \cdot \sigma^2$. Confusing these two is the most common trap on multi-step forecasting questions.
10. Let $X_t=\phi X_{t-1}+\varepsilon_t$ with $|\phi|<1,$ and define the differenced series $Y_t=X_t-X_{t-1}.$ Which representation of $Y_t$ in terms of lag polynomials and the qualification of its moving-average component are correct?
    Answer: $(1-\phi B)Y_t=(1-B)\varepsilon_t;$ the MA factor $(1-B)$ has a unit root, so the representation is noninvertible When you apply the difference operator to an AR(1) process, you need to track what happens to both the autoregressive and moving-average structure simultaneously — this is a classic lag-polynomial manipulation problem. Start from $X_t = \phi X_{t-1} + \varepsilon_t$, which in lag-polynomial form is $(1 - \phi B)X_t = \varepsilon_t$. Now define $Y_t = (1-B)X_t$. You want to express $Y_t$ purely in terms of $\varepsilon_t$. Since $(1-\phi B)X_t = \varepsilon_t$, multiply both sides by $(1-B)$: $(1-\phi B)(1-B)X_t = (1-B)\varepsilon_t$, which gives $(1-\phi B)Y_t = (1-B)\varepsilon_t$. The MA factor here is $(1-B)$, which has a root exactly on the unit circle (root at $z=1$). A moving-average polynomial is invertible only when all its roots lie strictly outside the unit circle, so $(1-B)$ is noninvertible — making D correct. Choice A simply rearranges the AR(1) equation without applying the difference operator properly; it doesn't yield a valid ARMA representation for $Y_t$. Choice B applies the difference operator to $Y_t$ a second time, producing $(1-B)Y_t$ on the left — that's over-differencing — and incorrectly claims the MA factor $(1-\phi B)$ is what appears. Choice C incorrectly asserts $Y_t$ satisfies the same AR(1) equation as $X_t$; differencing changes the model structure entirely and introduces a moving-average component. Your study tip: whenever you difference a stationary AR process, expect the result to have a noninvertible MA component with a unit root — this is the canonical signal that the series was already stationary and should not have been differenced.

Start this quiz