Simulation & Monte Carlo - Business Analytics

1. Which simulation design most appropriately represents both sources of uncertainty?
   Answer: Sample plausible treatment-effect parameters, then simulate future customer outcomes conditional on each sample When you see a Monte Carlo simulation question, ask yourself: how many distinct sources of uncertainty exist, and does the design account for all of them? Here, there are two layers — uncertainty about the true treatment effect (what did the experiment actually reveal?) and randomness in future customer behavior (even knowing the true rate, individual conversions are probabilistic). The right approach, captured in D, handles both layers sequentially. First, you sample plausible values of the treatment effect from a posterior or sampling distribution — acknowledging that your A/B test estimate is noisy. Then, conditional on each sampled parameter, you simulate individual customer conversions. This two-stage structure propagates both sources of variance into your profit distribution, giving management an honest picture of rollout risk. A collapses all uncertainty into zero on the parameter side — it treats the observed lift as ground truth, so your simulation only reflects conversion randomness, not estimation error. This systematically underestimates total uncertainty. B shortcuts the entire simulation by drawing profit directly from a fitted normal distribution. This may seem elegant, but it doesn't model the data-generating process at all — it just re-samples from a summary statistic, ignoring the structural relationship between conversion rates and profit, and conflates both uncertainty sources into one unjustified distribution. C is a classic bootstrap trap: resampling future customers while freezing both conversion rates at their sample values addresses only sampling variability in future demand, not uncertainty about the true underlying rates. It's the same flaw as A, dressed up as resampling. Study tip: Whenever a problem names two sources of uncertainty, the correct simulation design will have two explicit stages — one for each. If a choice collapses them into one step, it's almost certainly wrong.
2. What is the most defensible conclusion about the simulation's estimate of expected ticket revenue?
   Answer: It will tend to be too high because independence replaces a negative covariance with zero covariance Whenever simulation questions involve correlated inputs, your instinct should be to recall the covariance formula for the expectation of a product: $E[P \cdot Q] = E[P]\cdot E[Q] + \text{Cov}(P, Q)$. This identity is the key to unlocking this entire question. In reality, price and volume share a negative covariance — higher prices drive fewer sales. That negative covariance term reduces true expected revenue below the simple product of marginal means. When the analyst samples price and volume independently, the covariance term drops to zero. The simulation therefore computes revenue as roughly $E[P] \cdot E[Q]$, which is larger than the true expectation that includes the negative adjustment. The simulation inflates expected revenue — making A the correct answer. B is wrong because independent sampling doesn't systematically generate more low-volume outcomes; the marginal distribution of volume is preserved correctly. The bias comes from the relationship between variables, not the shape of any individual distribution. C is the most tempting trap. Preserving marginal distributions does keep $E[P]$ and $E[Q]$ correct, but it does not preserve the joint distribution or covariance. The product expectation depends on more than just the marginals — it requires the correct dependency structure. D invents a condition with no statistical basis. Equal standard deviations have no special role in determining whether the product expectation is biased. Study tip: Anytime a simulation question mentions correlated inputs being treated as independent, immediately apply $E[XY] = E[X]E[Y] + \text{Cov}(X,Y)$ and ask what sign the missing covariance term carries — that sign tells you the direction of bias.
3. Which interpretation of an approximate $95\%$ Monte Carlo confidence interval is most accurate?
   Answer: The interval is about $1.78$ to $1.90$ million dollars and quantifies simulation sampling error in the estimated mean Whenever you see a question about Monte Carlo confidence intervals, your first instinct should be to distinguish between two very different things: uncertainty about the estimated mean (simulation sampling error) versus variability in individual outcomes (the underlying distribution of results). A Monte Carlo standard error (MCSE) measures how precisely your simulation has estimated the true mean — it shrinks as you run more replications. The approximate $95\%$ confidence interval is constructed as: $\hat{\mu} \pm 1.96 \times \text{MCSE} = 1.84 \pm 1.96(0.03) \approx 1.84 \pm 0.059$ which gives roughly $1.78$ to $1.90$ million dollars. This interval tells you where the true mean likely falls, given the finite number of simulation runs you performed. That's exactly what B describes — the correct calculation paired with the correct interpretation. A is wrong on two counts: the interval $1.81$ to $1.87$ uses $\pm 1$ MCSE (not $1.96$), and claiming it contains $95\%$ of individual outcomes confuses a confidence interval for the mean with a prediction interval for raw outcomes. C is wrong because a Monte Carlo confidence interval — built from the MCSE — only captures simulation sampling error, not the parameter uncertainty embedded in the fitted input distributions. That would require a separate sensitivity or second-order analysis. D is wrong because $1.75$ to $1.93$ doesn't match the correct formula, and again misidentifies the interval as representing individual simulated profit outcomes rather than the estimated mean. Your study tip: always ask what is the interval actually measuring? MCSE-based intervals narrow uncertainty about the mean estimate, not the spread of outcomes. Mixing these up is the most common trap on simulation questions.
4. Which plan should be recommended if the simulated estimates are used as the decision criteria?
   Answer: Plan D, because it satisfies both the profit and loss-probability requirements When a question gives you multiple criteria that all must be satisfied, your first move is to eliminate any plan that fails even one requirement — not to find the "best" on any single dimension. Here, management sets two hard constraints: mean profit $\geq \$530{,}000$ and loss probability $\leq 10\%$. Run every plan through both filters before comparing anything else. Plan A ($520K mean, 14% loss probability) fails both thresholds. Plan B ($500K mean, 6% loss probability) has an excellent loss probability but falls well short on mean profit. Plan C ($550K mean, 11% loss probability) clears the profit hurdle but narrowly misses on loss probability — 11% exceeds the 10% cap. Only Plan D ($540K mean, 8% loss probability) clears both gates: $\$540{,}000 \geq \$530{,}000$ ✓ and $8\% \leq 10\%$ ✓. Plan D is the recommendation, making D correct. Choice A is a trap because it frames Plan A's near-miss on profit as acceptable while ignoring that it violates both constraints — rationalizing a failure doesn't make it a pass. Choice B highlights Plan B's superior loss probability, but optimizing one metric while ignoring the other is exactly the mistake the question is testing. Choice C is tempting because highest mean profit sounds best, but Plan C's 11% loss probability disqualifies it — a single constraint violation is enough to eliminate a plan. Strategy tip: On multi-criteria decision questions, always screen for hard constraints first. Any plan that violates a requirement is automatically out, regardless of how strong it looks on other dimensions.
5. What best explains the narrower confidence interval from the second analysis?
   Answer: Pairing creates positive covariance between policy costs, reducing variance of their difference Whenever you see a question comparing variance-reduction techniques in simulation, think about what drives the variance of a difference between two random variables. Recall that $\text{Var}(X - Y) = \text{Var}(X) + \text{Var}(Y) - 2\text{Cov}(X, Y)$. This identity is the key to the whole question. When both policies face the same simulated demand path in each trial — a technique called Common Random Numbers (CRN) — their costs move together. A high-demand draw makes both policies expensive; a low-demand draw makes both cheap. This creates positive covariance between the two cost outcomes. Plugging that positive covariance back into the formula above, you subtract a large positive term, dramatically shrinking the variance of the cost difference. A smaller variance means a narrower confidence interval. That's exactly what option B describes, making it correct. Option A is tempting but wrong. CRN doesn't remove demand uncertainty from each policy's individual cost estimate — each policy still faces random, uncertain demand. It only exploits the correlation that pairing creates. Option C is flatly incorrect: sharing demand paths does not force the two policies to have equal expected costs. They can and do respond differently to the same demand, which is precisely why we're comparing them. Option D misunderstands what sample size means here. Each paired trial still counts as one observation of the difference — you haven't doubled your sample size just because two policies experienced the same path. A good study tip: when a simulation question mentions "same random draws for both alternatives," your first thought should be variance reduction through positive covariance, not sample size or bias.
6. What conclusion should the analyst draw from this simulation?
   Answer: The evidence is insufficient because the upper bound is about $0.3\%$, above the required limit Whenever you see a simulation study used to validate a risk threshold, your instinct should be to ask: does the simulation actually have enough power to confirm safety, or does absence of evidence just reflect a small sample? This is a classic application of statistical confidence bounds for rare events. The key tool here is the rule of three (or more precisely, the $3/n$ approximation): when you observe zero events in $n$ independent trials, an approximate upper 95% confidence bound on the true event probability is $3/n$. With $n = 1{,}000$, that bound is $3/1{,}000 = 0.003 = 0.3\%$. The bank's requirement is that the true exceedance probability be below $0.1\%$. Since $0.3\% > 0.1\%$, the simulation cannot statistically confirm compliance — the upper bound exceeds the target. That makes C the correct conclusion: the evidence is insufficient. A is wrong because zero observed events never implies the true probability is exactly zero. You simply didn't see the event in this sample — that's a statement about your data, not about reality. B is a seductive trap: yes, the observed rate is 0%, which beats 0.1% by a wide margin, but this ignores sampling uncertainty entirely. Observed rates and confidence bounds are not the same thing. D inverts the logic — a lack of observed events does not mean the probability must be above 0.3%; that bound is an upper limit, not a lower one. Study tip: On business analytics exams, whenever a simulation returns zero adverse outcomes, immediately apply $3/n$ to check whether the sample size is actually large enough to validate the risk threshold. Don't let a clean simulation fool you into ignoring statistical power.
7. Based on these five trials, what are the simulated average profit and the simulated stockout frequency?
   Answer: $1{,}008$ dollars and $20\%$, respectively When you see a simulation question like this, your job is to apply the random-number-to-demand mapping trial by trial, compute profit for each, then aggregate. Mapping the five random numbers to demand: - $0.18 < 0.25$ → demand = $80$ - $0.25 \leq 0.64 \leq 0.75$ → demand = $100$ - $0.91 > 0.75$ → demand = $140$ - $0.22 < 0.25$ → demand = $80$ - $0.25 \leq 0.73 \leq 0.75$ → demand = $100$ Profit formula: You ordered $100$ units. Units sold = $\min(\text{demand}, 100)$. Unsold = $100 - \text{units sold}$. Profit = $(\text{sold} \times \$30) - (100 \times \$18) + (\text{unsold} \times \$6)$. - Demand 80: sold=80, unsold=20 → $2{,}400 - 1{,}800 + 120 = \$720$ - Demand 100: sold=100, unsold=0 → $3{,}000 - 1{,}800 + 0 = \$1{,}200$ - Demand 140: sold=100, unsold=0 → $3{,}000 - 1{,}800 = \$1{,}200$ - Demand 80: $\$720$ - Demand 100: $\$1{,}200$ Average profit: $(720 + 1{,}200 + 1{,}200 + 720 + 1{,}200) \div 5 = 5{,}040 \div 5 = \$1{,}008$ Stockout occurs when demand exceeds supply (140 > 100): only trial 3. That's $1/5 = 20\%$. This confirms A. Choice B inflates the average to $\$1{,}104$, likely by miscalculating salvage value or cost. Choice C correctly finds $\$1{,}008$ but misidentifies stockouts — trials with demand = 80 have surplus, not stockouts. Choice D compounds both errors. Study tip: Always distinguish stockout (demand > inventory) from surplus (demand < inventory) — exam distractors routinely swap these two conditions.
8. Which staffing level is optimal under management's stated decision rule?
   Answer: Nine agents, because it is the lowest-cost option satisfying the service constraint When a question gives you both a cost metric and a constraint, your first job is to filter — eliminate any option that violates the constraint, then optimize among whatever remains. This "constrained optimization" logic appears constantly in business analytics and operations management. Here, management's hard rule is that the probability of average wait exceeding five minutes must be $\leq 10\%$. Start by checking each option against that threshold. Eight agents produces a $16\%$ exceedance probability — that violates the constraint, so it's immediately eliminated regardless of its cost. Nine agents yields $9\%$, which satisfies the rule. Ten agents yields $5\%$, which also satisfies it. Now you optimize cost among the two feasible options: nine agents costs $\$3{,}770$ per day versus ten agents at $\$3{,}900$. Nine agents is cheaper, making it the optimal choice — confirming B. A is a classic trap: it ignores the service constraint entirely and ranks all three options purely on cost. Eight agents is cheapest, but it's not feasible — a constraint violation means it's off the table, full stop. C is wrong because it confuses "best on one metric" with "optimal overall." Ten agents does have the lowest exceedance probability, but once nine agents already satisfies the constraint, spending an extra $\$130$ per day to push probability from $9\%$ to $5\%$ isn't required or justified. D goes even further in the wrong direction by suggesting feasible options should be ranked only by service probability, completely discarding cost — that's not how constrained optimization works. Your study tip: whenever you see a constraint (a maximum, minimum, or threshold), apply it as a filter before you optimize. Cost minimization, profit maximization, or any other objective only applies to the feasible set.
9. Which validation approach provides the strongest evidence for using the simulation as a decision-support tool?
   Answer: Back-test on withheld periods, comparing simulated service and cost distributions with realized outcomes When evaluating a simulation model's credibility, the central question is: does the model accurately represent reality, or does it just reflect the data it was built on? This distinction between calibration and validation is what this question tests. True validation requires exposing the model to data it has never seen. Option C does exactly this — by withholding a portion of historical data, running the simulation over that period, and comparing the simulated service levels and cost distributions against what actually happened, you get genuine out-of-sample evidence. If the model reproduces real outcomes on unseen data, management has strong grounds to trust its recommendations. This is the simulation equivalent of cross-validation in machine learning. Option A describes overfitting, not validation. Forcing the model's average output to match the calibration-period average guarantees it performs well on training data but tells you nothing about generalizability — it actually makes the model less trustworthy as a decision tool. Option B reduces Monte Carlo sampling error, which is a precision concern, not a validity concern. Even a perfectly precise simulation of the wrong model will lead you astray. Reducing variance around a biased estimate doesn't fix the bias. Option D conflates code verification with model validation. Confirming that formulas are error-free ensures the model does what you programmed it to do, but not whether what you programmed correctly represents the real system. These are separate questions. Study tip: On simulation and analytics questions, always distinguish three concepts — calibration (fitting to data), verification (correct implementation), and validation (out-of-sample accuracy). Exam questions often conflate them as distractors.
10. Approximately how many total trials are required?
    Answer: $22{,}500$ total trials, applying the inverse-square-root relationship for standard error Whenever you see a Monte Carlo simulation question asking about standard error reduction, your anchor concept is the inverse-square-root relationship: standard error scales as $SE \propto \frac{1}{\sqrt{n}}$. This means to cut standard error in half, you need four times the trials — not twice. Here, the analyst wants to reduce $SE$ from $\$4{,}800$ to $\$1{,}600$, which is a reduction by a factor of 3. Using the relationship $\frac{SE_1}{SE_2} = \sqrt{\frac{n_2}{n_1}}$, you can solve for the required trials: $\frac{4{,}800}{1{,}600} = \sqrt{\frac{n_2}{2{,}500}} \implies 3 = \sqrt{\frac{n_2}{2{,}500}} \implies 9 = \frac{n_2}{2{,}500} \implies n_2 = 22{,}500$ That confirms B is correct. Choice A is wrong because it treats standard error as falling linearly with the number of trials — tripling the trials from 2,500 to 7,500. This ignores the square-root relationship entirely, a common intuition trap. Choice C miscounts by taking the additional trials needed (22,500 − 2,500 = 20,000) and treating that as the total, confusing "additional" with "total." Choice D applies a halving rule (doubling trials cuts SE by $\sqrt{2}$, not by 3), which doesn't match the target reduction specified in the problem. Your study tip: memorize $n \propto \frac{1}{SE^2}$. If you want to reduce SE by a factor of $k$, you need $k^2$ times as many trials. Writing this out quickly on scratch paper during the exam prevents the linear-thinking trap that makes A so tempting.

Start this quiz