Conjugate Priors - Statistics Graduate Level

1. A binomial success probability has mixture prior $\frac12\mathrm{Beta}(1,1)+\frac12\mathrm{Beta}(10,2)$. Two trials are observed, and both are failures. Which expression is the posterior distribution of $p$?
   Answer: $\frac{26}{29}\mathrm{Beta}(1,3)+\frac{3}{29}\mathrm{Beta}(10,4)$ When you see a mixture prior combined with binomial data, your key tool is Bayes' theorem applied component-wise: each Beta component updates independently via conjugacy, but the mixing weights must also be updated to reflect how likely each component made the observed data. With prior $\pi(p) = \frac{1}{2}\mathrm{Beta}(1,1) + \frac{1}{2}\mathrm{Beta}(10,2)$ and data of 2 failures (0 successes), each Beta$(\alpha, \beta)$ component updates to Beta$(\alpha + 0,\ \beta + 2)$, giving Beta$(1,3)$ and Beta$(10,4)$ respectively. So far every answer using those updated kernels is on the right track — the real differentiator is the updated weights. The new weight for component $k$ is proportional to the old weight times the marginal likelihood of the data under that component: $w_k \cdot p(\text{data} \mid \text{component } k)$. For a Beta$(\alpha,\beta)$ prior with $n=2$ trials and $s=0$ successes, the marginal likelihood is $B(\alpha, \beta+2)/B(\alpha,\beta) = \frac{\beta(\beta+1)}{(\alpha+\beta)(\alpha+\beta+1)}$. Computing: component 1 gives $\frac{1\cdot2}{2\cdot3}=\frac{1}{3}$, component 2 gives $\frac{2\cdot3}{12\cdot13}=\frac{1}{26}$. Weighting equally: unnormalized weights are $\frac{1}{2}\cdot\frac{1}{3}=\frac{1}{6}$ and $\frac{1}{2}\cdot\frac{1}{26}=\frac{1}{52}$. The total is $\frac{1}{6}+\frac{1}{52}=\frac{29}{156}$, yielding posterior weights $\frac{26}{29}$ and $\frac{3}{29}$. This confirms C. Choice A reverses the weights — a classic trap of forgetting that the uniform Beta$(1,1)$ component assigns higher probability to low-$p$ outcomes (like two failures) than the Beta$(10,2)$ component does, so it should receive the larger posterior weight. Choice B keeps the original equal weights, ignoring the data's role in reweighting altogether. Choice D uses the correct weights but fails to properly update the Beta parameters (adding only 1 to $\beta$ instead of 2). Study tip: In mixture-prior problems, always update both the kernel parameters (via conjugacy) and the mixing weights (via marginal likelihoods). Forgetting either step produces a plausible-looking but wrong answer.
2. Consider the normal linear model $y\mid\boldsymbol\beta\sim N(X\boldsymbol\beta,2I)$. The prior is $\boldsymbol\beta\sim N(\boldsymbol m_0,V_0)$, where $\boldsymbol m_0=(0,1)^{\mathsf T}$ and $V_0=\operatorname{diag}(1,2)$. Suppose $X^{\mathsf T}X=\begin{pmatrix}4&2\\2&3\end{pmatrix}$ and $X^{\mathsf T}y=(6,5)^{\mathsf T}$. Which posterior mean vector and covariance matrix pair is correct?
   Answer: $\boldsymbol m_n=\begin{pmatrix}3/5\\6/5\end{pmatrix},\quad V_n=\frac{1}{5}\begin{pmatrix}2&-1\\-1&3\end{pmatrix}$ When you encounter Bayesian linear regression questions, your first move should be to write down the posterior update formulas. For the model $y \mid \boldsymbol\beta \sim N(X\boldsymbol\beta, \sigma^2 I)$ with prior $\boldsymbol\beta \sim N(\boldsymbol m_0, V_0)$, the posterior covariance and mean are: $V_n = \left(V_0^{-1} + \frac{1}{\sigma^2}X^\top X\right)^{-1}, \qquad \boldsymbol m_n = V_n\left(V_0^{-1}\boldsymbol m_0 + \frac{1}{\sigma^2}X^\top y\right)$ Here $\sigma^2 = 2$. First compute $V_0^{-1} = \text{diag}(1, 1/2)$ and $\frac{1}{2}X^\top X = \begin{pmatrix}2&1\\1&3/2\end{pmatrix}$. Adding these gives $V_n^{-1} = \begin{pmatrix}3&1\\1&2\end{pmatrix}$. Inverting (determinant $= 5$) yields $V_n = \frac{1}{5}\begin{pmatrix}2&-1\\-1&3\end{pmatrix}$. Next, compute $V_0^{-1}\boldsymbol m_0 = (0, 1/2)^\top$ and $\frac{1}{2}X^\top y = (3, 5/2)^\top$. Their sum is $(3, 3)^\top$. Then $\boldsymbol m_n = \frac{1}{5}\begin{pmatrix}2&-1\\-1&3\end{pmatrix}\begin{pmatrix}3\\3\end{pmatrix} = \frac{1}{5}\begin{pmatrix}3\\6\end{pmatrix}$, confirming answer C is correct. Answer A has the right covariance but uses $(4/5, 7/5)^\top$ for the mean — this comes from incorrectly omitting the prior mean contribution $V_0^{-1}\boldsymbol m_0$. Answer B flips the covariance matrix (swapping the (1,1) and (2,2) entries), meaning $V_n^{-1}$ was inverted incorrectly. Answer D appears to use $\sigma^2 = 1$ instead of $\sigma^2 = 2$, a classic trap when the variance isn't 1. Always read $\sigma^2$ carefully — forgetting to divide $X^\top X$ and $X^\top y$ by $\sigma^2$ is the single most common computational error in these problems.
3. A Bernoulli success probability has prior distribution $p\sim\mathrm{Beta}(3,5)$. After observing $7$ successes in $10$ trials, two additional conditionally independent Bernoulli trials will be conducted. What is the posterior predictive probability that exactly one of the two additional trials is a success?
   Answer: $\frac{80}{171}$ When you see a Bayesian prediction problem, your goal is to update the prior with observed data, then integrate over uncertainty in the parameter to get the predictive distribution. Your prior is $\text{Beta}(3,5)$. After observing 7 successes in 10 trials, the posterior is $\text{Beta}(3+7, 5+3) = \text{Beta}(10, 8)$, with posterior mean $\frac{10}{18} = \frac{5}{9}$. Now you need $P(\text{exactly 1 success in 2 trials})$. With two conditionally independent trials given $p$, this equals $2p(1-p)$. You take the posterior expectation: $P(X=1) = 2\,E[p(1-p)] = 2\bigl(E[p] - E[p^2]\bigr)$ For $\text{Beta}(\alpha,\beta)$ with $\alpha=10, \beta=8$: $E[p] = \frac{10}{18}$ and $E[p^2] = \frac{\alpha(\alpha+1)}{(\alpha+\beta)(\alpha+\beta+1)} = \frac{10 \cdot 11}{18 \cdot 19} = \frac{110}{342}$. So: $2\!\left(\frac{10}{18} - \frac{110}{342}\right) = 2\!\left(\frac{190}{342} - \frac{110}{342}\right) = 2 \cdot \frac{80}{342} = \frac{160}{342} = \frac{80}{171}$ This confirms A is correct. Choice B, $\frac{40}{81}$, comes from mistakenly using $2 \cdot \frac{5}{9} \cdot \frac{4}{9}$, treating the posterior mean as if $p$ were known — ignoring posterior variance. Choice C, $\frac{5}{9}$, is just the posterior mean of $p$ itself, not the predictive probability for one-of-two. Choice D, $\frac{80}{153}$, likely stems from an incorrect posterior, such as $\text{Beta}(10,7)$, miscounting the prior failures. Study tip: Never plug in the posterior mean as a point estimate for predictive calculations — you must account for $E[p^2]$, where posterior variance creates a meaningful correction.
4. Counts from three unit-exposure periods are conditionally independent $\mathrm{Poisson}(\lambda)$ variables. The prior is $\lambda\sim\mathrm{Gamma}(4,2)$ under the shape-rate parameterization, and the observed counts have sum $5$. Let $T$ be the total count over two future unit-exposure periods. Which pair gives the posterior predictive mean and variance of $T$?
   Answer: $\left(\frac{18}{5},\frac{126}{25}\right)$ Bayesian updating with conjugate priors is the core concept here. When you observe Poisson data with a Gamma prior, the posterior is also Gamma — and predicting future Poisson counts requires layering the posterior uncertainty over the sampling uncertainty. Posterior first. With prior $\lambda \sim \mathrm{Gamma}(4, 2)$ (shape $\alpha=4$, rate $\beta=2$) and $n=3$ observations summing to $s=5$, the conjugate update gives $\lambda \mid \mathbf{x} \sim \mathrm{Gamma}(\alpha + s,\, \beta + n) = \mathrm{Gamma}(9, 5)$. The posterior mean is $9/5$ and variance is $9/25$. **Predictive distribution for $T$.** For $m=2$ future unit-exposure periods, $T \mid \lambda \sim \mathrm{Poisson}(2\lambda)$. Use the law of total expectation and total variance: $E[T] = E[E[T\mid\lambda]] = E[2\lambda] = 2 \cdot \frac{9}{5} = \frac{18}{5}$ $\mathrm{Var}(T) = E[\mathrm{Var}(T\mid\lambda)] + \mathrm{Var}(E[T\mid\lambda]) = E[2\lambda] + \mathrm{Var}(2\lambda) = \frac{18}{5} + 4\cdot\frac{9}{25} = \frac{90}{25}+\frac{36}{25} = \frac{126}{25}$ This confirms B. Why the distractors fail: A gives the correct mean but sets variance equal to the mean — the Poisson-only variance, ignoring posterior uncertainty about $\lambda$. C reports the posterior mean and a scaled variance but forgets to multiply by $m=2$ when computing $E[T]$. D uses incorrect update parameters, as if the prior rate weren't added to $n$. Study tip: On predictive problems, always decompose variance via $E[\text{Var}]+\text{Var}[E]$ — the two terms capture sampling noise and parameter uncertainty separately. Forgetting the second term is the most common trap.
5. For a normal sample with both parameters unknown, use the conjugate prior $\mu\mid\sigma^2\sim N(m_0,\sigma^2/\kappa_0)$ and $\sigma^2\sim\mathrm{IG}(\alpha_0,\beta_0)$, where the inverse-gamma density contains $\exp(-\beta_0/\sigma^2)$. Let $m_0=2$, $\kappa_0=1$, $\alpha_0=3$, and $\beta_0=5$. For $n=4$ observations, suppose $\bar{x}=6$ and $\sum_{i=1}^4(x_i-\bar{x})^2=12$. Which posterior parameter tuple $(\kappa_n,m_n,\alpha_n,\beta_n)$ is correct?
   Answer: $\left(5,\frac{26}{5},5,\frac{87}{5}\right)$ When you encounter a Normal-Normal-Inverse-Gamma conjugate update, you need four update formulas memorized cold. Starting from prior parameters $(\kappa_0, m_0, \alpha_0, \beta_0)$ with data summary $(n, \bar{x}, S = \sum(x_i - \bar{x})^2)$, the posterior parameters are: $\kappa_n = \kappa_0 + n, \quad m_n = \frac{\kappa_0 m_0 + n\bar{x}}{\kappa_n}$ $\alpha_n = \alpha_0 + \frac{n}{2}, \quad \beta_n = \beta_0 + \frac{S}{2} + \frac{\kappa_0 n(\bar{x}-m_0)^2}{2\kappa_n}$ Plugging in $\kappa_0=1, m_0=2, \alpha_0=3, \beta_0=5, n=4, \bar{x}=6, S=12$: $\kappa_n = 1+4 = 5, \quad m_n = \frac{(1)(2)+(4)(6)}{5} = \frac{26}{5}$ $\alpha_n = 3 + 2 = 5, \quad \beta_n = 5 + 6 + \frac{(1)(4)(16)}{10} = 11 + \frac{64}{10} = 11 + \frac{32}{5} = \frac{87}{5}$ This confirms B $(5, 26/5, 5, 87/5)$ is correct. Choice A gets $\beta_n = 11$, which omits the crucial third term in the $\beta$ update — the penalty for the prior mean differing from $\bar{x}$. This is the most common trap. Choice C computes $m_n$ incorrectly as $22/5$, likely by using $\kappa_0 \bar{x} + n m_0$ instead of the correct weighting. Choice D inflates $\alpha_n$ to 7 by adding $n$ rather than $n/2$, and consequently produces the wrong $\beta_n$. Your key study tip: the $\beta_n$ formula has three additive pieces — prior $\beta_0$, half the sample variance $S/2$, and a shrinkage term capturing prior-data conflict. Missing any one piece yields a wrong answer, and exams love to isolate exactly that mistake.
6. Suppose $X_1,\ldots,X_5\mid\mu$ are independent $N(\mu,2)$ variables and their sample mean is $3$. The prior distribution is $\mu\sim N(1,4)$. Which distribution is the posterior distribution of $\mu$, with the second parameter denoting variance?
   Answer: $N\left(\frac{31}{11},\frac{4}{11}\right)$ When you see a Bayesian inference question with a normal likelihood and normal prior, your first instinct should be to apply the normal-normal conjugate update formulas. The posterior is also normal, with updated mean and variance determined by precision-weighting. Here's the setup: you have $n = 5$ observations from $N(\mu, \sigma^2)$ with known variance $\sigma^2 = 2$, sample mean $\bar{x} = 3$, and prior $\mu \sim N(\mu_0, \tau^2) = N(1, 4)$. The posterior mean and variance formulas are: $\mu_{\text{post}} = \frac{\frac{\mu_0}{\tau^2} + \frac{n\bar{x}}{\sigma^2}}{\frac{1}{\tau^2} + \frac{n}{\sigma^2}}, \qquad \sigma^2_{\text{post}} = \frac{1}{\frac{1}{\tau^2} + \frac{n}{\sigma^2}}$ Plugging in: the posterior precision is $\frac{1}{4} + \frac{5}{2} = \frac{1}{4} + \frac{10}{4} = \frac{11}{4}$, giving posterior variance $\frac{4}{11}$. The posterior mean is $\frac{4}{11}\left(\frac{1}{4} + \frac{15}{2}\right) = \frac{4}{11} \cdot \frac{31}{4} = \frac{31}{11}$. This confirms A is correct. B is wrong because using $\bar{x} = 1$ (the prior mean) instead of 3 in the numerator yields $\frac{13}{11}$ — a classic mix-up of prior mean and data. C uses incorrect precision values, likely from mixing up $\sigma^2$ and $n$, producing a wrong variance of $\frac{2}{13}$. D simply reports the MLE result $N(3, \frac{2}{5})$, ignoring the prior entirely — this is what you'd get with no Bayesian updating. Always derive the posterior variance first (it's simpler), then use it as a factor in computing the posterior mean to avoid arithmetic errors.
7. Let $\theta=p/(1-p)$ denote the odds of success in a binomial model. A beta-prime prior on $\theta$ has density proportional to $\theta^{a-1}(1+\theta)^{-(a+b)}$. If $a=3$ and $b=4$ and the data contain $5$ successes in $8$ trials, what is the posterior distribution of $\theta$?
   Answer: $\mathrm{BetaPrime}(8,7)$ When working with conjugate priors on transformed parameters, your goal is to track how the likelihood updates the prior's exponents by expressing everything in a common kernel form. The beta-prime distribution with parameters $(a, b)$ has density proportional to $\theta^{a-1}(1+\theta)^{-(a+b)}$. Now consider the binomial likelihood in terms of the odds $\theta = p/(1-p)$. Since $p = \theta/(1+\theta)$, the likelihood for $s$ successes in $n$ trials becomes proportional to $p^s(1-p)^{n-s} = \theta^s(1+\theta)^{-n}$. Multiplying the prior by the likelihood gives the posterior kernel: $\theta^{(a-1)+s}(1+\theta)^{-(a+b)+(-n)} = \theta^{(a+s)-1}(1+\theta)^{-(a+s+b+n-s)}$ Wait — collecting exponents carefully: the combined exponent of $(1+\theta)$ is $-(a+b+n)$. So the posterior is $\mathrm{BetaPrime}(a+s,\, b+n-s)$. With $a=3$, $b=4$, $s=5$, $n=8$: posterior parameters are $a+s = 3+5 = 8$ and $b+(n-s) = 4+3 = 7$, giving $\mathrm{BetaPrime}(8,7)$, which is answer D. Choice A, $\mathrm{BetaPrime}(7,8)$, reverses the two parameters — a sign you've mixed up which count updates which exponent. Choice B, $\mathrm{BetaPrime}(8,12)$, incorrectly adds the full $n=8$ to $b$ rather than only the failures $n-s=3$. Choice C, $\mathrm{BetaPrime}(5,3)$, ignores the prior entirely, using only the data. The key study tip: always convert the likelihood into $\theta$-and-$(1+\theta)$ form before pattern-matching to the conjugate kernel — successes update $a$, and failures (not total trials) update $b$.
8. Independent event counts satisfy $Y_i\mid\lambda\sim\mathrm{Poisson}(t_i\lambda)$, where $t_i$ is exposure. The prior is $\lambda\sim\mathrm{Gamma}(2,3)$ under the shape-rate parameterization. The observed total count is $14$ and the total exposure is $5$. For a future exposure of $2$, what is the posterior predictive probability of observing no events?
   Answer: $\left(\frac{8}{10}\right)^{16}$ When you see a Bayesian predictive problem with Poisson likelihoods and a Gamma prior, your roadmap is: update the prior, then marginalize over the posterior to get a negative binomial predictive distribution. Posterior update: With $n$ observations, the conjugate update gives $\lambda \mid \mathbf{y} \sim \mathrm{Gamma}\!\left(\alpha + \sum y_i,\; \beta + \sum t_i\right)$. Here, the shape-rate prior is $\mathrm{Gamma}(2, 3)$, total count is $14$, and total exposure is $5$, so the posterior is $\mathrm{Gamma}(2+14,\; 3+5) = \mathrm{Gamma}(16, 8)$. Posterior predictive for zero events: For future exposure $t^* = 2$, the predictive probability of $Y^* = 0$ is: $P(Y^*=0) = \int_0^\infty e^{-2\lambda} \cdot \frac{8^{16}}{\Gamma(16)}\lambda^{15}e^{-8\lambda}\,d\lambda = \frac{8^{16}}{\Gamma(16)}\cdot\frac{\Gamma(16)}{(8+2)^{16}} = \left(\frac{8}{10}\right)^{16}$ This is the MGF/Laplace transform trick: integrating gives $\left(\frac{\beta}{\beta + t^*}\right)^\alpha$. That confirms answer A. Why the distractors fail: Choice B uses $\beta + t^* = 9$ instead of $10$, suggesting a prior rate of $4$ rather than $3$ — a miscalculation in the posterior update. Choice C appears to use only the MLE ($\hat{\lambda} = 14/5$) and ignores the prior entirely, substituting it directly. Choice D, $e^{-4}$, treats the posterior mean $\hat{\lambda} = 16/8 = 2$ as fixed and computes $e^{-2 \cdot 2}$, ignoring uncertainty in $\lambda$. Study tip: Always verify both parameters of the posterior Gamma — the shape collects counts, the rate collects exposures. A one-off error in either cascades through the entire predictive calculation.
9. Bernoulli trials with success probability $p$ are observed until the $r$th success occurs. The stopping time is $N=n$, so the observed sequence contains $r$ successes and $n-r$ failures. If the prior is $p\sim\mathrm{Beta}(\alpha,\beta)$, which statement correctly describes the posterior distribution conditional on the observed stopped sequence?
   Answer: It is $\mathrm{Beta}(\alpha+r,\beta+n-r)$ because the stopping rule adds no factor depending on $p$. When you see a Bayesian updating problem with a stopping rule, the key question is: does the stopping rule introduce any factor involving $p$ that wasn't already accounted for? If not, the posterior is unchanged by the rule. In the negative binomial (inverse sampling) setting, the likelihood of the observed sequence — exactly $r$ successes and $n-r$ failures in a specific order ending in a success — is proportional to $p^r(1-p)^{n-r}$. The stopping rule determines which sequences are possible, but the binomial coefficient (the counting factor) doesn't depend on $p$. So when you multiply the Beta prior $p^{\alpha-1}(1-p)^{\beta-1}$ by the likelihood, you get $p^{\alpha+r-1}(1-p)^{\beta+(n-r)-1}$, which is exactly $\mathrm{Beta}(\alpha+r,\,\beta+n-r)$. The posterior updates by adding successes to $\alpha$ and failures to $\beta$ — the same rule as in fixed-$n$ binomial sampling. Answer D is correct. Answer A is wrong because the stopping rule doesn't break conjugacy — it only changes the support of $N$, not the functional form of the likelihood in $p$. Answer B incorrectly adds the total trials $n$ to $\alpha$, confusing total observations with successes. Answer C adds $r$ (successes) to $\beta$ and $n$ to $\alpha$, reversing the roles of successes and failures entirely. A useful principle to remember: **the posterior depends only on the likelihood's functional form in $p$, not on the sampling design**. This is sometimes called the likelihood principle, and it's why fixed-$n$ and stopped-$N$ sampling yield identical Beta posteriors given the same sufficient statistics $(r, n-r)$.
10. Let $X_1,\ldots,X_8$ be independent $N(\mu,\sigma^2)$ observations, where $\mu$ is known. Suppose $\sum_{i=1}^{8}(X_i-\mu)^2=20$. The prior is inverse-gamma with density proportional to $(\sigma^2)^{-3-1}\exp(-4/\sigma^2)$. What is the posterior mean of $\sigma^2$?
    Answer: $\frac{7}{3}$ When you see a Bayesian estimation problem like this, your goal is to identify the posterior distribution by combining the likelihood with the prior, then read off the posterior mean. The prior here is inverse-gamma: $(\sigma^2)^{-\alpha_0-1}\exp(-\beta_0/\sigma^2)$, which matches $\text{Inv-Gamma}(\alpha_0, \beta_0)$ with $\alpha_0 = 3$ and $\beta_0 = 4$. The likelihood for $n = 8$ normal observations with known $\mu$ contributes a factor of $(\sigma^2)^{-n/2}\exp\!\left(-\frac{\sum(X_i-\mu)^2}{2\sigma^2}\right) = (\sigma^2)^{-4}\exp(-10/\sigma^2)$, since $\sum(X_i-\mu)^2 = 20$ gives $20/2 = 10$. Multiplying prior and likelihood, the posterior is proportional to $(\sigma^2)^{-(3+4)-1}\exp\!\left(-\frac{4+10}{\sigma^2}\right)$, which is $\text{Inv-Gamma}(\alpha_n, \beta_n)$ with $\alpha_n = 3 + 4 = 7$ and $\beta_n = 4 + 10 = 14$. The mean of an $\text{Inv-Gamma}(\alpha, \beta)$ distribution is $\frac{\beta}{\alpha - 1}$, so the posterior mean is $\frac{14}{7-1} = \frac{14}{6} = \frac{7}{3}$, confirming answer A. Choice B, $7/4$, mistakenly divides $\beta_n$ by $\alpha_n$ itself rather than $\alpha_n - 1$. Choice C, $2$, likely comes from naively dividing the sum of squares by $n$: $20/8 = 2.5$, or some similar shortcut ignoring the prior entirely. Choice D, $14/5$, uses $\alpha_n - 2 = 5$ in the denominator, confusing the mean formula with the mode formula $\beta/(\alpha+1)$. Your key memory aid: for Inv-Gamma, the posterior parameters update as $\alpha_n = \alpha_0 + n/2$ and $\beta_n = \beta_0 + \frac{1}{2}\sum(X_i-\mu)^2$, and the mean is always $\beta_n/(\alpha_n - 1)$.

Start this quiz