Statistics Graduate Level Quiz: Gibbs And Metropolis Hastings Sampling

10 questions · exam conditions

0:00

Gibbs And Metropolis Hastings SamplingQuestion 1 of 10

A deterministic-scan Gibbs sampler targets a standard bivariate normal distribution with correlation $\rho=0.8$ . Each sweep updates $X$ conditional on the previous $Y$ and then updates $Y$ conditional on the new $X$ . At sweep boundaries, what is the integrated autocorrelation time for estimating $E[Y]$ , ignoring initialization effects?

It is approximately

9.00

, obtained by treating the lag-one coefficient as

0.8

It is approximately

2.78

, obtained from the reciprocal of

1-0.8^2

It is approximately

1.64

, obtained by adding one to the squared correlation.

It is approximately

4.56

, obtained using an autoregressive coefficient of

0.8^2

← Back to quizzes

Statistics Graduate Level Quiz

Statistics Graduate Level Quiz: Gibbs And Metropolis Hastings Sampling

Practice Gibbs And Metropolis Hastings Sampling in Statistics Graduate Level with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Gibbs And Metropolis Hastings Sampling, giving you a quick way to practice the rules, question types, and explanations that matter most for Statistics Graduate Level.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

It is approximately $9.00$ , obtained by treating the lag-one coefficient as $0.8$ .
It is approximately $2.78$ , obtained from the reciprocal of $1-0.8^2$ .
It is approximately $1.64$ , obtained by adding one to the squared correlation.
It is approximately $4.56$ , obtained using an autoregressive coefficient of $0.8^2$ . (correct answer)

Explanation: When analyzing autocorrelation in a deterministic-scan Gibbs sampler, you need to identify the effective autoregressive structure at sweep boundaries — not just read off the raw correlation

\rho

. In a bivariate normal with correlation

\rho = 0.8

, each full sweep first draws

X \mid Y

, then

Y \mid X

. After one complete sweep, the chain for

Y

behaves like an AR(1) process. The key insight is that the autocorrelation between

Y

values at successive sweep boundaries equals

\rho^2

, not

\rho

itself, because the update passes through an intermediate

X

step. So the effective autoregressive coefficient is

\rho^2 = 0.64

. For an AR(1) process with coefficient

\phi

, the integrated autocorrelation time is

\tau = \frac{1 + \phi}{1 - \phi}

. Plugging in

\phi = \rho^2 = 0.64

\tau = \frac{1 + 0.64}{1 - 0.64} = \frac{1.64}{0.36} \approx 4.56

This confirms D is correct. Now for the distractors: A incorrectly uses

\phi = \rho = 0.8

directly as the lag-one coefficient, ignoring that each sweep squares the autocorrelation. B computes

\frac{1}{1 - \rho^2} = \frac{1}{0.36} \approx 2.78

, which is the marginal variance inflation factor, not the IACT formula. C adds one to

\rho^2

to get

1.64

, which is merely the numerator of the correct IACT expression — a partial calculation error. As a study habit, always trace the sweep-level Markov chain before applying AR(1) IACT formulas. The squaring of

\rho

under deterministic scan is a frequently tested subtlety that trips up students who conflate the raw correlation with the transition kernel's spectral gap.

Question 2

For a latent-variable model, the likelihood $L(\theta)$ is unavailable, but a nonnegative estimator $\widehat L(\theta,U)$ satisfies $E[\widehat L(\theta,U)\mid\theta]=L(\theta)$ . A pseudo-marginal Metropolis–Hastings algorithm stores both $\theta$ and the auxiliary random variable $U$ as its state.

Which implementation preserves the exact posterior marginal for $\theta$ ?

Refresh the likelihood estimates at both the current and proposed parameter values on every attempt, then use their ratio.
Retain the current estimator when a proposal is rejected, and generate a new estimator only for each proposed parameter value. (correct answer)
Replace the random estimator by its logarithm before computing the acceptance ratio, since unbiasedness is preserved.
Accept using only the expected estimator variance, because unbiasedness makes the realized estimator values unnecessary.

Explanation: Whenever you encounter pseudo-marginal MCMC questions, anchor your thinking to one central principle: the joint chain over

(\theta, U)

must leave the joint target

\pi(\theta, U) \propto \widehat{L}(\theta, U)\,\pi(\theta)

invariant, so that the marginal over

\theta

recovers the true posterior. The key insight of the pseudo-marginal (Andrieu & Roberts, 2009) construction is that you treat

(\theta^{\text{current}}, U^{\text{current}})

as a single state. When a proposal

\theta^*

is generated, you draw a fresh

U^*

and form the acceptance ratio

\widehat{L}(\theta^*, U^*)/\widehat{L}(\theta^{\text{current}}, U^{\text{current}})

. Crucially, upon rejection, you carry forward the same

U^{\text{current}}

—not a newly sampled one. This is exactly what B describes, and it is the only scheme that satisfies detailed balance for the joint target. A is wrong because refreshing the current estimator on every attempt breaks detailed balance: you would be comparing two fresh draws, effectively running a different chain whose stationary distribution is not the correct posterior marginal. C is wrong because taking the logarithm of an unbiased estimator destroys unbiasedness. By Jensen's inequality,

E[\log \widehat{L}] \leq \log E[\widehat{L}] = \log L

, so log-unbiasedness does not hold, and the acceptance ratio becomes invalid. D is wrong because the realized value of

\widehat{L}

is essential to the acceptance ratio. Replacing it with a summary statistic like variance discards the stochastic information that makes the marginal chain exact. A useful study habit: for any MCMC variant, always ask what is the full state space and what joint distribution must be preserved? That framing instantly reveals whether a proposed modification is valid.

Question 3

Suppose a proposed joint posterior kernel on $\mathbb R^2$ is $\widetilde\pi(x,y)=\exp\{-(x-y)^2/2\}$ . Its formal full conditionals are $X\mid Y=y\sim N(y,1)$ and $Y\mid X=x\sim N(x,1)$ , so both can be sampled exactly. What does this imply about the resulting Gibbs chain?

It has a proper stationary distribution because proper full conditionals are sufficient to define a proper joint posterior.
It is stationary after one sweep because the two normal full conditionals have equal conditional variances.
It behaves like a random walk in the common-location direction and has no stationary probability distribution. (correct answer)
It converges to a bivariate normal distribution with unit marginal variances and zero correlation.

Explanation: When you encounter a proposed joint kernel in Gibbs sampling, your first instinct should be to check whether the kernel actually defines a proper joint distribution — not just whether the full conditionals look well-behaved individually. Here,

\widetilde\pi(x,y) = \exp\{-(x-y)^2/2\}

depends only on the difference

x - y

. If you try to integrate this over

\mathbb{R}^2

, you can reparametrize with

u = x-y

and

v = x+y

: the integral over

u

is finite (it's a Gaussian), but the integral over

v

is infinite. The "joint distribution" is improper — it has no normalizing constant. Consequently, the Gibbs chain drifts without bound in the direction

x + y

, behaving like a random walk with no stationary distribution. That confirms C is correct. A is wrong because it reflects a fundamental misconception: valid full conditionals are necessary but not sufficient for a proper joint. The Hammersley–Clifford theorem requires the joint to be proper in the first place. Improper joints can still yield coherent-looking conditionals while the chain wanders indefinitely. B is a red herring. Equal conditional variances are irrelevant to whether a stationary distribution exists — this conflates computational convenience with probabilistic validity. D is wrong because a bivariate normal would require a proper, normalizable joint. No such normalization exists here, so convergence to any proper bivariate normal is impossible. The key study tip: whenever a Gibbs sampler is proposed, always verify the joint kernel is integrable before trusting the full conditionals. Pretty conditionals can hide an improper joint.

Question 4

For a two-component target with available full conditionals, compare a random-scan Gibbs chain that chooses one coordinate at random per iteration with a deterministic-scan chain that updates the first coordinate and then the second during each sweep. Which statement is generally correct?

Only deterministic scan preserves the target, because random scan can repeatedly omit one coordinate.
Both chains are reversible, because every individual full-conditional update is reversible.
Both preserve the target; random scan is reversible, whereas a deterministic full sweep generally is not reversible. (correct answer)
Only random scan preserves the target, because composing conditional-update kernels destroys invariance.

Explanation: When analyzing Gibbs samplers, you need to think carefully about two separate properties: invariance (does the chain preserve the target?) and reversibility (does the chain satisfy detailed balance?). These properties are logically independent, and conflating them is the source of most errors on questions like this. Both scan types preserve the target distribution

\pi

. In random scan, at each step you select coordinate

i

with some probability

p_i > 0

and update it from its full conditional — this mixture kernel leaves

\pi

invariant. In deterministic scan, you apply the two conditional kernels in sequence; since each kernel individually leaves

\pi

invariant, their composition does as well. So C is correct: both chains are valid samplers for

\pi

. The key distinction is reversibility. A random-scan update is a mixture of transition kernels, and mixtures of reversible kernels are reversible — so random scan satisfies detailed balance. A deterministic full sweep is a composition (product) of kernels, not a mixture. Compositions of reversible kernels are generally not reversible themselves, because the joint kernel

K_1 K_2

typically violates

\pi(x)K(x,y) = \pi(y)K(y,x)

. A is wrong because random scan does preserve

\pi

— positive probability of selecting each coordinate is sufficient; it need not update both every iteration. B is wrong because it incorrectly claims the deterministic sweep is reversible; composition ≠ mixture. D is wrong because composing conditional-update kernels absolutely does not destroy invariance — only reversibility may be lost. A useful rule of thumb: mixture → reversibility preserved; composition → invariance preserved, reversibility generally lost. Keep this distinction sharp for any MCMC question involving scan strategies.

Question 5

A parameter $\theta>0$ has target density $\pi(\theta)\propto\theta e^{-\theta}$ . A random-walk Metropolis update is implemented on the transformed scale $Z=\log\theta$ using a proposal symmetric in $Z$ . Which acceptance probability is correct for a move from $\theta$ to $\theta'$ ?

$\min\left\{1,\exp[-(\theta'-\theta)]\right\}$
$\min\left\{1,(\theta'/\theta)\exp[-(\theta'-\theta)]\right\}$
$\min\left\{1,(\theta/\theta')^2\exp[-(\theta-\theta')]\right\}$
$\min\left\{1,(\theta'/\theta)^2\exp[-(\theta'-\theta)]\right\}$ (correct answer)

Explanation: Whenever you see a Metropolis algorithm applied to a transformed variable, the critical concept is the Jacobian correction. The sampler proposes moves symmetrically in

Z = \log\theta

, but the acceptance ratio must be evaluated using the original target density

\pi(\theta)

. Because the proposal is symmetric in

Z

, the proposal ratio cancels — but you still need to account for the change of variables between

Z

-space and

\theta

-space. The target in the original parameterization is

\pi(\theta) \propto \theta e^{-\theta}

. When the sampler moves symmetrically in

Z = \log\theta

, the effective density being sampled in

\theta

-space is

\pi(\theta)

divided by the Jacobian

|d\theta/dZ| = \theta

. So the acceptance ratio becomes:

\alpha = \min\left\{1, \frac{\pi(\theta')/\theta'}{\pi(\theta)/\theta}\right\} = \min\left\{1, \frac{\theta' e^{-\theta'}/\theta'}{\theta e^{-\theta}/\theta}\right\} = \min\left\{1, \frac{\theta'^2 e^{-\theta'}}{\theta^2 e^{-\theta}}\right\}

This simplifies to

\min\left\{1,(\theta'/\theta)^2\exp[-(\theta'-\theta)]\right\}

, confirming D is correct. Choice A ignores both the Jacobian and the ratio of

\theta

values — it only captures the exponential part of the target. Choice B uses a single power of

\theta'/\theta

, which would be correct if the target were

\pi(\theta) \propto e^{-\theta}

(no

\theta

factor), since the Jacobian alone contributes one power. Choice C inverts the ratio and the exponent sign simultaneously, reflecting a confusion about which direction the move goes. The key study tip: when a random-walk proposal is defined on a transformed scale

Z = g(\theta)

, always multiply the target ratio by the Jacobian ratio

|g'(\theta)/g'(\theta')|

to correct for the reparameterization.

Question 6

A target density is uniform on the union of the two sets $[0,1]\times[0,1]$ and $[2,3]\times[2,3]$ , with equal probability assigned to the two sets. A deterministic-scan Gibbs sampler alternately draws from the exact full conditionals of $X$ and $Y$ . If initialized in the first set, which conclusion is most accurate?

The chain eventually reaches the second set because every full conditional is sampled exactly.
The chain reaches both sets only if the coordinate-update order is randomized at every sweep.
The target is invariant, but the chain remains in the first set and cannot converge to the full target from that initialization. (correct answer)
The target is not invariant because a Gibbs sampler requires the support of the target to be convex.

Explanation: When analyzing Gibbs sampler convergence, you need to think carefully about the geometry of the target's support and how coordinate-wise updates actually move through that space. The target here is uniform on

[0,1]\times[0,1] \cup [2,3]\times[2,3]

. The full conditional of

X

given

Y=y

is uniform on the set of

x

-values compatible with that

y

. If

y \in [0,1]

, then the only compatible

x

-values are

[0,1]

; similarly, if

y \in [2,3]

, then

x \in [2,3]

. The two squares share no

x

- or

y

-coordinates — they are separated in both dimensions. So if you start at, say,

(0.5, 0.5)

, updating

X

gives you something in

[0,1]

, then updating

Y

given that new

X \in [0,1]

gives you something in

[0,1]

, and so on forever. The chain is trapped. Critically, the target is still invariant — Gibbs sampling with exact full conditionals always preserves the target as a stationary distribution — but invariance does not guarantee convergence. The chain fails to be irreducible, so it cannot converge to the full target from this initialization. This makes C correct. A is wrong because exact full conditionals ensure detailed balance and invariance, not irreducibility. The chain cannot jump between disconnected components through coordinate updates alone. B is wrong because randomizing the update order doesn't help either — the problem is geometric, not procedural. Randomized scans still respect the same full conditionals, which are equally trapped. D is wrong because Gibbs sampling does not require a convex support; it requires the full conditionals to be well-defined, which they are here. The key study tip: always distinguish stationarity from ergodicity. A Gibbs sampler on a disconnected support can have the correct stationary distribution yet fail to converge — a classic trap on graduate-level MCMC questions.

Question 7

In a hierarchical regression model, the joint posterior kernel for a coefficient vector $\beta$ and a variance parameter $\sigma^2$ is available as $\widetilde\pi(\beta,\sigma^2)$ . The full conditional for $\beta$ cannot be sampled directly, so a proposal with density $q(\beta'\mid\beta,\sigma^2)$ is used while $\sigma^2$ is held fixed.

Which acceptance probability produces a valid Metropolis-within-Gibbs update of $\beta$ ?

$\min\left\{1,\frac{\widetilde\pi(\beta',\sigma^2)q(\beta'\mid\beta,\sigma^2)}{\widetilde\pi(\beta,\sigma^2)q(\beta\mid\beta',\sigma^2)}\right\}$
$\min\left\{1,\frac{\widetilde\pi(\beta',\sigma^2)q(\beta\mid\beta',\sigma^2)}{\widetilde\pi(\beta,\sigma^2)q(\beta'\mid\beta,\sigma^2)}\right\}$ (correct answer)
$\min\left\{1,\frac{\widetilde\pi(\beta',\sigma^2)q(\beta\mid\beta',\sigma^2)}{\widetilde\pi(\beta,\sigma^2)q(\beta'\mid\beta,\sigma^2)}\frac{\pi(\sigma^2\mid\beta')}{\pi(\sigma^2\mid\beta)}\right\}$
$\min\left\{1,\frac{\pi(\beta')q(\beta\mid\beta',\sigma^2)}{\pi(\beta)q(\beta'\mid\beta,\sigma^2)}\right\}$

Explanation: Whenever you see a question involving Metropolis-within-Gibbs, recognize that you are essentially running a standard Metropolis-Hastings step inside a Gibbs sweep — but only for the block being updated, with all other parameters held fixed. The classic Metropolis-Hastings acceptance ratio for targeting a distribution proportional to

f(x)

with proposal

q

is:

\alpha = \min\left\{1, \frac{f(x')q(x \mid x')}{f(x)q(x' \mid x)}\right\}

Here, the target for

\beta

(with

\sigma^2

fixed) is the full conditional

\pi(\beta \mid \sigma^2, \text{data})

, which is proportional to the joint kernel

\widetilde\pi(\beta, \sigma^2)

evaluated at fixed

\sigma^2

. Plugging directly into the MH ratio gives exactly B: the numerator carries the proposed

\widetilde\pi(\beta', \sigma^2)

times the reverse proposal

q(\beta \mid \beta', \sigma^2)

, and the denominator carries the current

\widetilde\pi(\beta, \sigma^2)

times the forward proposal

q(\beta' \mid \beta, \sigma^2)

. A inverts the proposal ratio — it places the forward proposal

q(\beta' \mid \beta, \sigma^2)

in the numerator and the reverse in the denominator, which violates detailed balance and produces a biased chain. C introduces an extra factor

\pi(\sigma^2 \mid \beta')/\pi(\sigma^2 \mid \beta)

. Because

\sigma^2

is held fixed during this step, no correction for it is needed; this factor is an unnecessary (and incorrect) modification. D uses marginal densities

\pi(\beta')

and

\pi(\beta)

instead of the joint kernel evaluated at the current

\sigma^2

, discarding the conditioning information that defines the full conditional target. A reliable strategy: always identify the target of the MH step first (here, the full conditional for

\beta

at fixed

\sigma^2

), then write down the standard ratio — numerator is target at proposal times reverse proposal, denominator is target at current times forward proposal.

Question 8

An independence Metropolis–Hastings sampler uses a normal proposal density to target a standard Cauchy density on the real line. Both densities are strictly positive everywhere. Which assessment is most accurate?

The chain is valid and can converge to the Cauchy target, but it is not uniformly ergodic and may remain in extreme tails for long periods. (correct answer)
The chain is invalid because an independence proposal must have tails heavier than those of the target to satisfy detailed balance.
The chain is uniformly ergodic because positivity of the normal proposal makes every target state reachable in one step.
The chain has the normal proposal as its invariant distribution because proposed values are independent of the current state.

Explanation: When studying Metropolis–Hastings (MH) samplers, you need to distinguish between validity (correct invariant distribution) and ergodicity type (how fast/reliably the chain converges). These are separate properties, and confusing them is exactly what this question tests. The independence MH sampler accepts a proposed draw

x^*

from proposal

q(x^*)

with probability

\min\!\left(1, \frac{\pi(x^*)q(x_t)}{\pi(x_t)q(x^*)}\right)

. Because both the normal proposal and the Cauchy target are strictly positive everywhere, the acceptance ratio is always well-defined and every state is reachable, so the chain is valid with the Cauchy as its invariant distribution. That settles A as correct. However, uniform ergodicity requires a uniform bound on how quickly the chain forgets its starting point — formally, a bounded ratio

\pi(x)/q(x)

over all

x

. Here, the Cauchy has polynomial tails

\sim |x|^{-2}

while the normal decays exponentially. Their ratio

\pi(x)/q(x) \to \infty

|x| \to \infty

, meaning the proposal dramatically underrepresents the tails. When the chain wanders into extreme values, acceptance rates plummet and it can stay trapped there for very long stretches. This is precisely the failure of uniform ergodicity. B is wrong because detailed balance does not require the proposal to have heavier tails than the target — it only requires the acceptance ratio formula to be applied correctly. C is wrong because positivity alone is insufficient for uniform ergodicity; the critical requirement is a bounded weight ratio

\pi/q

, which fails here. D confuses the proposal distribution with the invariant distribution — independence of proposed values from the current state is a design feature, not a mechanism for changing the target. A useful rule of thumb: whenever a light-tailed proposal (e.g., normal) targets a heavy-tailed distribution (e.g., Cauchy or Student-t), expect poor tail exploration and loss of uniform ergodicity, even if the chain remains theoretically valid.

Question 9

A Metropolis–Hastings chain targets a density proportional to $x^2e^{-x}$ on $x>0$ . Its independence proposal has density $q(y)=e^{-y}$ on $y>0$ . If the current state is $x=1$ and the proposed state is $y=3$ , what is the acceptance probability?

The acceptance probability is $1$ . (correct answer)
The acceptance probability is $9e^{-2}$ .
The acceptance probability is $e^{-2}/9$ .
The acceptance probability is $1/9$ .

Explanation: When you see a Metropolis–Hastings question with an independence proposal, your first instinct should be to write out the acceptance ratio carefully. For an independence sampler, the acceptance probability is:

\alpha(x, y) = \min\left(1,\, \frac{\pi(y)\, q(x)}{\pi(x)\, q(y)}\right)

where

\pi

is the target density and

q

is the proposal density. Notice that both

q(x)

and

q(y)

appear — the proposal evaluated at the current state goes in the numerator. Here,

\pi(x) \propto x^2 e^{-x}

and

q(y) = e^{-y}

. Plugging in

x = 1

and

y = 3

\frac{\pi(y)\, q(x)}{\pi(x)\, q(y)} = \frac{(3^2 e^{-3})(e^{-1})}{(1^2 e^{-1})(e^{-3})} = \frac{9 e^{-4}}{e^{-4}} = 9

Since this ratio exceeds 1, you take the minimum with 1, giving an acceptance probability of exactly 1. Answer A is correct. Answer B,

9e^{-2}

, results from forgetting to include

q(x)

and

q(y)

in the ratio entirely — just computing

\pi(y)/\pi(x)

without accounting for the proposal. Answer C,

e^{-2}/9

, inverts the ratio, swapping

x

and

y

in the formula. Answer D,

1/9

, drops the exponential terms altogether, keeping only the polynomial parts of

\pi

. Your study tip: always write the full ratio

\pi(y)q(x)/[\pi(x)q(y)]

before simplifying — and remember, if it exceeds 1, the acceptance probability is simply 1. The proposal terms often cancel beautifully, as they do here.

Question 10

A target distribution is bivariate normal with zero means, unit variances, and correlation $\rho=0.8$ . An algorithm simultaneously generates $X'=\rho Y+\epsilon_X$ and $Y'=\rho X+\epsilon_Y$ , where the errors are independent $N(0,1-\rho^2)$ variables and $X,Y$ are the current values. Which statement is correct?

The update preserves the target because both new coordinates have the correct full conditional distributions.
The update does not preserve the target; drawing $Y'$ conditional on $X'$ instead of $X$ would restore a standard Gibbs sweep. (correct answer)
The update preserves the target because the marginal variances remain one and the errors are independent.
The update does not preserve the target; reversing the order of the two simultaneous assignments would restore invariance.

Explanation: When you see a question about MCMC samplers and target distribution preservation, your first instinct should be to check whether each update uses the correct conditioning set — specifically, whether new values are drawn conditional on the current state of all other variables, not on simultaneously updated values. In a standard Gibbs sampler, you update one variable at a time, each conditional on the most recent values of the others. Here, both

X'

and

Y'

are generated simultaneously:

X' = \rho Y + \epsilon_X

and

Y' = \rho X + \epsilon_Y

, where

X, Y

are the old values. Notice that

Y'

is conditioned on the old

X

, not the newly drawn

X'

. This means the joint update does not correspond to a valid Gibbs sweep. The correct Gibbs approach would first draw

X' \mid Y

, then draw

Y' \mid X'

— using the updated

X'

in the second step. That sequential structure is precisely what answer B describes, making B correct. Answer A is wrong because having correct full conditional distributions individually does not guarantee the joint update preserves the target. The joint transition kernel matters, and simultaneous updates with old conditioning values break detailed balance. Answer C is wrong because preserving marginal variances and using independent errors are necessary but not sufficient conditions. The joint distribution's correlation structure and the validity of the transition kernel are what actually determine invariance. Answer D is wrong because the problem isn't the order of the two assignments — it's the simultaneity itself. Reversing the order still leaves both updates conditioned on the old state, which doesn't fix the issue. Study tip: On MCMC questions, always ask: "What is each draw conditioned on?" Simultaneous updates that condition on stale values are a classic trap — they look like Gibbs but aren't.