Bayesian Inference Basics - Statistics Graduate Level

1. What is the posterior probability that the patient has the disease?
   Answer: $\frac{16}{97}$, based on multiplying both conditional likelihood ratios Bayesian updating with multiple independent tests is the core concept here. When conditional independence holds, you can multiply likelihood ratios sequentially — each test result updates the odds independently. Start with prior odds: $\frac{P(D)}{P(\bar{D})} = \frac{0.10}{0.90} = \frac{1}{9}$. Now compute each test's likelihood ratio. For a positive result: $LR_+ = \frac{\text{sensitivity}}{1 - \text{specificity}} = \frac{0.80}{0.10} = 8$. For a negative result: $LR_- = \frac{1 - \text{sensitivity}}{\text{specificity}} = \frac{0.20}{0.90} = \frac{2}{9}$. Because the tests are conditionally independent, multiply both ratios against the prior odds: $\text{Posterior odds} = \frac{1}{9} \times 8 \times \frac{2}{9} = \frac{16}{81}$. Converting to probability: $\frac{16}{81 + 16} = \frac{16}{97}$. That confirms C. Choice A applies only the positive test's likelihood ratio, giving posterior odds $\frac{8}{9}$, hence $\frac{8}{17}$ — it ignores the negative result entirely. Choice B applies only the negative test's likelihood ratio, ignoring the positive result, yielding $\frac{2}{11}$. Both A and B commit the error of discarding information from one of the two tests. Choice D assumes the two discordant results "cancel out" and reduces back toward the prior — this is intuitive but mathematically wrong; the results don't cancel, they both independently update the odds in their respective directions. Your takeaway: whenever tests are conditionally independent, posterior odds = prior odds × product of all individual likelihood ratios. Never discard a test result, even when results conflict — discordant results still each carry real evidential weight.
2. What is the posterior probability that $\theta=0$?
   Answer: $\frac{4}{4+3\sqrt{2}}$, obtained by weighting the two component-specific predictive densities When a prior mixes a point mass with a continuous distribution, posterior inference requires Bayes' theorem applied carefully to each component's predictive density — the marginal likelihood of the data under that component. For the point-mass component ($\theta = 0$), the predictive density of $Y = 0$ is simply $f(Y=0 \mid \theta=0) = \phi(0) = \frac{1}{\sqrt{2\pi}}$, where $\phi$ is the standard normal density. For the continuous component ($\theta \sim N(0,1)$), you must marginalize: $Y \sim N(0, 1+1) = N(0,2)$, giving predictive density $\frac{1}{\sqrt{4\pi}} = \frac{1}{\sqrt{2}} \cdot \frac{1}{\sqrt{2\pi}}$. Applying Bayes' theorem with prior weights 0.4 and 0.6: $P(\theta=0 \mid Y=0) = \frac{0.4 \cdot \frac{1}{\sqrt{2\pi}}}{0.4 \cdot \frac{1}{\sqrt{2\pi}} + 0.6 \cdot \frac{1}{\sqrt{2}} \cdot \frac{1}{\sqrt{2\pi}}} = \frac{0.4}{0.4 + \frac{0.6}{\sqrt{2}}} = \frac{4}{4 + 3\sqrt{2}}$ This confirms D is correct. Choice A ignores that the continuous component's predictive variance is 2, not 1 — the observation $Y=0$ is actually more concentrated under $\theta=0$ than under the diffuse component, so the point mass gains posterior weight. Choice B inverts the likelihood ratio, swapping numerator and denominator. Choice C applies equal prior weights (0.5/0.5) instead of the given 0.4/0.6 split. Your study tip: in spike-and-slab or mixed priors, always compute the marginal likelihood for each component separately before combining with prior weights — conflating the prior and likelihood is the most common error here.
3. Which statement about their posterior distributions for $\theta$ is correct?
   Answer: They agree because the two likelihoods differ only by a factor that does not depend on $\theta$. Whenever you see a question involving different sampling designs applied to the same data, your first instinct should be to write out both likelihoods and compare them term by term — specifically, check whether they differ only by a multiplicative constant that is free of $\theta$. For the fixed-sample design (Analyst I), the likelihood is binomial: $L_I(\theta) \propto \binom{r+f}{r} \theta^r (1-\theta)^f$. For the negative-binomial design (Analyst II), which stops at the $r$th success, the likelihood is: $L_{II}(\theta) \propto \binom{r+f-1}{r-1} \theta^r (1-\theta)^f$. Notice that both likelihoods share the same kernel $\theta^r(1-\theta)^f$. The only difference is the combinatorial prefactor — $\binom{r+f}{r}$ versus $\binom{r+f-1}{r-1}$ — and neither depends on $\theta$. Because the posterior is proportional to likelihood times prior, and these constants cancel in normalization, both analysts arrive at identical posteriors for any proper prior. This is the essence of the Likelihood Principle: inference about $\theta$ depends only on the $\theta$-dependent part of the likelihood. Choice A is wrong because the powers of $\theta$ and $1-\theta$ are identical in both likelihoods — the stopping rule only affects the combinatorial coefficient. Choice B is wrong because posterior agreement holds for any proper prior, not just the uniform; the prior is irrelevant to this cancellation. Choice C is wrong because both likelihoods involve exactly $f$ failures — there is no "one fewer failure" discrepancy between the two designs. As a study tip: when comparing likelihoods across designs, factor out everything that doesn't involve the parameter. If what remains is identical, the posteriors must agree regardless of the prior chosen.
4. Which statement correctly describes the resulting Bayesian analysis?
   Answer: The posterior is $N(\bar Y,1/3)$, but an ordinary marginal likelihood retains an arbitrary prior scaling constant. When working with improper priors in Bayesian analysis, you need to track two separate questions: (1) does the posterior normalize to a proper density, and (2) does the marginal likelihood have a well-defined value? With the flat prior $\pi(\theta) \propto 1$ and likelihood $L(\theta) \propto \exp\!\left(-\frac{3}{2}(\theta - \bar{Y})^2\right)$, the posterior is proportional to a Gaussian in $\theta$. Completing the square gives a proper $N(\bar{Y}, 1/3)$ posterior — the integral over $\theta$ converges even though the prior was improper. So far so good. However, the marginal likelihood requires integrating the joint $\pi(\theta)L(\theta)$ over $\theta$. Since $\pi(\theta) \propto c$ for an unspecified constant $c$, the marginal likelihood inherits that same arbitrary scaling factor. The posterior is fine because $c$ cancels in the ratio $\pi(\theta \mid y) \propto \pi(\theta)L(\theta)$, but the marginal likelihood itself does not cancel it. This makes answer A correct. Answer B is wrong because a likelihood that is integrable in $\theta$ can absolutely normalize with an improper prior — the posterior here is a textbook example of this. Answer C incorrectly claims the posterior's normalizability "fixes" the prior into a proper distribution. Normalizing the posterior doesn't retroactively assign a finite total mass to $\pi(\theta) \propto 1$ on $\mathbb{R}$. Answer D is the subtlest trap: the likelihood does not determine the missing constant in $\pi(\theta)$, because the prior constant never appears in the likelihood function. Remember: improper priors can yield proper posteriors, but the marginal likelihood remains scale-ambiguous — a critical distinction for Bayes factor comparisons.
5. What is the posterior mode of $\lambda$?
   Answer: $\frac{3}{10}$, using the shape-minus-one formula with the posterior rate When you see a Bayesian inference problem with a Poisson likelihood and Gamma prior, your first move should be identifying the conjugate posterior and then carefully applying the mode formula. Here's the framework: if $Y_i \mid \lambda \sim \text{Poisson}(t_i \lambda)$ with a $\text{Gamma}(\alpha_0, \beta_0)$ prior (shape, rate), the posterior is $\text{Gamma}(\alpha_0 + \sum y_i,\ \beta_0 + \sum t_i)$. With prior shape $\alpha_0 = 1/2$, prior rate $\beta_0 = 1$, total count $\sum y_i = 2$, and total exposure $\sum t_i = 4$, the posterior is $\text{Gamma}(5/2,\ 5)$. The mode of a $\text{Gamma}(\alpha, \beta)$ distribution is $\frac{\alpha - 1}{\beta}$, valid when $\alpha > 1$. This gives $\frac{5/2 - 1}{5} = \frac{3/2}{5} = \frac{3}{10}$, confirming B is correct. Choice A is wrong because it uses the posterior mean ($\alpha/\beta = 1/2$) rather than the mode, conflating two distinct summary statistics. Choice C divides $\alpha - 1 = 3/2$ only by the total exposure $4$, forgetting to include the prior rate in the denominator — a classic bookkeeping error when accumulating the rate parameter. Choice D divides the full posterior shape $5/2$ by the exposure alone, making two errors simultaneously: ignoring the prior rate and forgetting the "minus one" in the mode formula. Your study tip: memorize the Gamma mode as $(\alpha - 1)/\beta$ and always verify both updated parameters — shape and rate — before plugging in. Missing either one is the most common trap in conjugate Bayesian problems.
6. What is the Bayes factor in favor of $H_1$ over $H_0$?
   Answer: $4$, the posterior odds divided by the prior odds Whenever you encounter a Bayes factor question, anchor yourself to one core relationship: the Bayes factor measures how much the data shift your beliefs, defined precisely as the ratio of posterior odds to prior odds. The prior odds in favor of $H_1$ are $\frac{0.20}{0.80} = \frac{1}{4}$. After seeing the data, the posterior odds are $\frac{0.50}{0.50} = 1$. The Bayes factor is therefore: $BF_{10} = \frac{\text{posterior odds}}{\text{prior odds}} = \frac{1}{1/4} = 4$ This confirms A is correct. The Bayes factor of 4 tells you the data made $H_1$ four times more credible relative to $H_0$ than it was before. B is wrong because dividing the posterior odds (1) by the prior probability of $H_1$ (0.20) gives $5$, which mixes odds with a raw probability — these are not interchangeable. The formula requires odds in both numerator and denominator. C divides the posterior probability of $H_1$ (0.50) by its prior probability (0.20), yielding $2.5$. This ratio describes how much the probability of $H_1$ alone changed, but ignores the corresponding shift in $H_0$'s probability — the Bayes factor must account for both hypotheses simultaneously through odds. D constructs an arbitrary quantity unrelated to any standard definition of the Bayes factor. A reliable study tip: always convert probabilities to odds before computing the Bayes factor. Writing out prior odds and posterior odds explicitly before dividing will prevent the unit-mixing errors that traps B and C represent.
7. Which is the posterior density of $\phi$ on $(0,1)$?
   Answer: $f(\phi\mid X=1)=1$, obtained after including the transformation Jacobian When transforming parameters in a Bayesian setting, you must apply the full change-of-variables procedure to the posterior density, not just the prior or just the likelihood. The posterior of $\phi = \theta^2$ requires substituting $\theta = \sqrt{\phi}$ and multiplying by the Jacobian $\left|\frac{d\theta}{d\phi}\right| = \frac{1}{2\sqrt{\phi}}$. Start with Bayes' theorem. The prior is $\pi(\theta) = 1$ on $(0,1)$, and the likelihood given $X=1$ is $L(\theta) = \theta$. So the posterior is $\pi(\theta \mid X=1) \propto \theta \cdot 1 = \theta$, which normalizes to $f(\theta \mid X=1) = 2\theta$ on $(0,1)$. Now transform: $f(\phi \mid X=1) = f(\theta \mid X=1)\big|_{\theta=\sqrt{\phi}} \cdot \frac{1}{2\sqrt{\phi}} = 2\sqrt{\phi} \cdot \frac{1}{2\sqrt{\phi}} = 1$. This confirms B — the posterior of $\phi$ is Uniform$(0,1)$. Choice A mistakenly plugs $\phi$ directly into the likelihood as if $\phi$ were itself a Bernoulli parameter, skipping both the prior transformation and the Jacobian. Choice C applies the Jacobian only to the prior, ignoring the posterior's likelihood component — it transforms the wrong density. Choice D substitutes $\theta = \sqrt{\phi}$ into the posterior correctly but forgets to include the Jacobian factor, leaving the density unnormalized and incorrect. The key strategy: whenever you change variables in a posterior, always transform the complete posterior density and never forget the Jacobian. A quick self-check is verifying your result integrates to 1.
8. Which posterior distribution for $\mu$ is correct?
   Answer: $N\left(\frac{73}{25},\frac{36}{25}\right)$, combining prior and sample-mean precisions Bayesian updating with a normal-normal conjugate model is a classic graduate topic. Whenever you see a normal likelihood and normal prior, remember the precision-weighted posterior mean formula: the posterior mean is a weighted average of the prior mean and the data mean, where weights are precisions (reciprocals of variances), and the posterior variance is the reciprocal of the total precision. Here's the key mechanics. With $n=4$ observations and individual variance $\sigma^2=9$, the sample mean $\bar{X}$ has variance $\sigma^2/n = 9/4$, giving it precision $4/9$. The prior $\mu \sim N(1,4)$ has precision $1/4$. The posterior precision is the sum: $1/4 + 4/9 = 9/36 + 16/36 = 25/36$, so the posterior variance is $36/25$. The posterior mean is $\frac{(1/4)(1)+(4/9)(4)}{25/36} = \frac{9/36+64/36}{25/36} = \frac{73/36}{25/36} = 73/25$. This confirms C is correct: $N\!\left(\tfrac{73}{25},\tfrac{36}{25}\right)$. Choice A makes the error of using variance $9$ for $\bar{X}$ rather than $9/4$, ignoring that averaging four observations reduces variance by $n$. Choice B swaps the means: it attaches the prior precision to the data mean and vice versa, a simple but fatal bookkeeping error. Choice D computes the right posterior mean but reports the precision $25/36$ as if it were the variance — a classic sign/inverse confusion. Your study tip: always track whether you're working with variance or precision. Label each quantity explicitly, because the most common errors on conjugate-normal problems come from accidentally inverting or misassigning these reciprocal quantities.
9. What is the posterior predictive probability that both of the next two items will be defective?
   Answer: $\frac{3}{8}$, obtained by integrating the two-outcome probability over the posterior When you see a question about posterior predictive probabilities, your instinct should be to integrate over uncertainty in $\theta$ rather than plug in a point estimate. The posterior predictive distribution accounts for the fact that $\theta$ is unknown even after observing data. With a $\text{Beta}(2,3)$ prior and 7 defectives in 10 trials, the posterior is $\text{Beta}(2+7, 3+3) = \text{Beta}(9, 6)$. To find the probability that the next two items are both defective, you compute $P(X_1=1, X_2=1 \mid \text{data}) = E[\theta^2 \mid \text{data}]$, since the two future draws are conditionally independent given $\theta$. For a $\text{Beta}(\alpha, \beta)$ distribution, $E[\theta^2] = \frac{\alpha(\alpha+1)}{(\alpha+\beta)(\alpha+\beta+1)} = \frac{9 \cdot 10}{15 \cdot 16} = \frac{90}{240} = \frac{3}{8}$. That confirms B. A is wrong because it squares the posterior mean: $\left(\frac{9}{15}\right)^2 = \frac{9}{25}$. This ignores posterior variance — squaring the mean underestimates $E[\theta^2]$ by Jensen's inequality (since $\theta^2$ is convex). C is a fabricated procedure with no Bayesian foundation; "updating the prior predictive with the observed proportion" is not a standard operation. D imagines conditioning on one more hypothetical success to get a $\text{Beta}(10,6)$ posterior mean of $\frac{10}{16} = \frac{5}{8}$, then squaring — an ad hoc approach with no justification. The key study tip: whenever computing the posterior predictive for multiple future observations, always compute $E[\theta^k \mid \text{data}]$ using the Beta moment formula — never just raise the posterior mean to the $k$-th power.
10. What is the posterior probability of $H_1$?
    Answer: $\frac{512}{567}$, obtained by integrating the likelihood over the uniform prior under $H_1$ When a question mixes a point hypothesis with a composite (prior-weighted) hypothesis, your core tool is Bayes' theorem on the hypothesis level. You need to compute the marginal likelihood of the data under each hypothesis, weight by the prior on each hypothesis, and normalize. Under $H_0$, the likelihood is simply $\theta = 1/2$ plugged in: $P(X \mid H_0) = \binom{10}{9}(1/2)^{10} = 10/1024$. Under $H_1$, $\theta$ is unknown with a Uniform(0,1) prior, so you must integrate out $\theta$: $P(X \mid H_1) = \int_0^1 \binom{10}{9}\theta^9(1-\theta)\,d\theta = 10\cdot B(10,2) = 10\cdot\frac{9!\,1!}{11!} = \frac{10}{110} = \frac{1}{11}.$ With equal priors on $H_0$ and $H_1$, the posterior probability of $H_1$ is: $P(H_1 \mid X) = \frac{1/11}{1/11 + 10/1024} = \frac{1/11}{1/11 + 5/512} = \frac{512}{512 + 55} = \frac{512}{567}.$ This confirms D is correct. A is wrong because it inverts the Bayes factor, putting $H_0$ in the numerator and yielding $55/567$, which is actually $P(H_0 \mid X)$, not $P(H_1 \mid X)$. B is wrong because it substitutes the posterior mean of $\theta$ under $H_1$ (which is $10/12 = 5/6$) as if it were a probability of $H_1$ itself — a category error confusing a parameter estimate with a hypothesis probability. C is wrong because it plugs in the MLE ($\hat\theta = 0.9$) rather than integrating over the prior. Skipping the integral is the classic Bayesian composite-hypothesis mistake. Study tip: Whenever one hypothesis has a free parameter with a prior, always marginalize — integrate the likelihood against the prior. Never substitute a point estimate for the integral.

Start this quiz