Markov Chains: Steady State - Finite Mathematics

1. Let $W = (w_1, w_2, ..., w_n)$ be the unique steady-state vector for a regular transition matrix $P$. Which of the following statements about $W$ is NOT always true?
   Answer: The component $w_i$ represents the one-step probability of transitioning to state $i$ from any other state. The steady-state vector $W$ represents the long-run probability distribution of the states. Choice A is true by definition of a probability vector. Choice B is the mathematical definition of a steady-state vector ($WP=W$ is equivalent to $W(P-I)=0$, which is a left eigenvector problem for eigenvalue 1). Choice C is the fundamental theorem for regular Markov chains. Choice D is incorrect. The one-step probability of transitioning to state $i$ from state $j$ is given by the matrix entry $P_{ji}$, which is not generally equal to $w_i$ and depends on the starting state $j$. The component $w_i$ is the long-run average probability of being in state $i$.
2. Two companies, Apex and Bedrock, compete in a market. Each year, Apex keeps 70% of its customers while 30% switch to Bedrock. Bedrock keeps 80% of its customers while 20% switch to Apex. Assuming these trends continue and the total number of customers in the market is stable, what is the long-run market share for Apex?
   Answer: 40% Let the states be Apex (A) and Bedrock (B). The transition matrix is $P = \begin{pmatrix} 0.7 & 0.3 \\ 0.2 & 0.8 \end{pmatrix}$, where $P_{ij}$ is the probability of moving from state $i$ to state $j$. Let the steady-state vector be $W = (w_A, w_B)$. We need to solve the system of equations given by $WP = W$ and $w_A + w_B = 1$. The equation $WP = W$ gives $0.7w_A + 0.2w_B = w_A$ and $0.3w_A + 0.8w_B = w_B$. Both simplify to $0.2w_B = 0.3w_A$, or $2w_B = 3w_A$. Using the constraint $w_B = 1 - w_A$, we substitute to get $2(1 - w_A) = 3w_A$. This simplifies to $2 - 2w_A = 3w_A$, which gives $2 = 5w_A$, so $w_A = 2/5 = 0.4$. Thus, Apex's long-run market share is 40%.
3. A simplified model of customer loyalty uses three states: Loyal (L), Neutral (N), and Competitor (C). Market research shows that the long-run proportions are 40% Loyal, 35% Neutral, and 25% Competitor. If the probability of moving from Loyal to Neutral in one period is 0.1, and from Loyal to Competitor is 0.05, what is the probability of remaining Loyal given that a customer is currently Loyal?
   Answer: $0.85$ because the remaining probability after accounting for transitions to other states must be self-retention This is a Markov chain problem involving transition probabilities between states. When you encounter customer loyalty or market share questions with multiple states and transition probabilities, you're working with a system where all probabilities from any given state must sum to 1. From the Loyal state, you know two transition probabilities: 0.1 to move to Neutral and 0.05 to move to Competitor. Since a customer must either stay Loyal or move to one of the other two states, these three probabilities must sum to 1. Therefore: P(stay Loyal) = 1 - P(move to Neutral) - P(move to Competitor) = 1 - 0.1 - 0.05 = 0.85. Choice A (0.75) incorrectly focuses only on competitor transitions, ignoring the movement to Neutral customers. This represents a common error of considering only some transitions rather than all possible outcomes. Choice B (0.40) confuses the steady-state proportion (long-run percentage of loyal customers) with the transition probability (likelihood of staying loyal in one period). These are completely different concepts in Markov chain analysis. Choice C (0.90) makes the mistake of only accounting for the transition to Neutral (1 - 0.1 = 0.90) while completely ignoring the transition to Competitor. This incomplete calculation misses a key piece of given information. Choice D (0.85) correctly recognizes that after accounting for all outgoing transitions, the remaining probability represents self-retention. Study tip: In transition probability problems, always verify that probabilities from each state sum to 1. Set up the equation: P(stay) + P(transition 1) + P(transition 2) + ... = 1, then solve for the unknown.
4. Consider a Markov chain with three states and transition matrix $P = \begin{pmatrix} 0.5 & 0.3 & 0.2 \\ 0.1 & 0.8 & 0.1 \\ 0.4 & 0.2 & 0.4 \end{pmatrix}$. Which statement about the long-run behavior is correct?
   Answer: The steady-state probability for state 2 is $\frac{12}{23}$ and represents the long-run proportion of time spent in state 2 To find steady state, solve $\pi P = \pi$ with $\pi_1 + \pi_2 + \pi_3 = 1$. This yields the system: $0.5\pi_1 + 0.1\pi_2 + 0.4\pi_3 = \pi_1$, $0.3\pi_1 + 0.8\pi_2 + 0.2\pi_3 = \pi_2$, $0.2\pi_1 + 0.1\pi_2 + 0.4\pi_3 = \pi_3$. Solving gives $\pi_1 = \frac{6}{23}$, $\pi_2 = \frac{12}{23}$, $\pi_3 = \frac{5}{23}$. Choice A confuses self-transition rates with steady-state probabilities. Choice C incorrectly suggests the steady state isn't unique (it is for irreducible chains). Choice D misunderstands absorption - this isn't an absorbing chain.
5. A simple weather model transitions between Sunny (S) and Cloudy (C) days. The transition matrix is $P = \begin{pmatrix} 1-p & p \\ q & 1-q \end{pmatrix}$, where $p$ and $q$ are the probabilities of the weather changing state. In the long run, a Sunny day is observed to be twice as likely as a Cloudy day. Which of the following describes the relationship between $p$ and $q$?
   Answer: $q = 2p$ The long-run probabilities are given by the steady-state vector $W = (w_S, w_C)$. We are given that $w_S = 2w_C$. Since $w_S + w_C = 1$, we can substitute to get $2w_C + w_C = 1$, which means $3w_C = 1$, so $w_C = 1/3$ and $w_S = 2/3$. The steady-state vector is $W = (2/3, 1/3)$. This vector must satisfy $WP = W$. Using the first component of the resulting equation: $(2/3)(1-p) + (1/3)(q) = 2/3$. Multiplying the entire equation by 3 gives $2(1-p) + q = 2$. This simplifies to $2 - 2p + q = 2$, which implies $q = 2p$.
6. If the economy is currently in a Boom, what is the long-run probability that it will be in a Recession?
   Answer: 2/7 For a regular Markov chain, the long-run probabilities are given by the steady-state vector $W$, and these probabilities are independent of the initial state. We need to find $W = (w_B, w_S, w_R)$ by solving $WP=W$. Due to the symmetry of the matrix ($P_{12}=P_{32}$, $P_{21}=P_{23}$, etc.), we can deduce that $w_B = w_R$. The first component equation is $0.6w_B + 0.2w_S + 0.1w_R = w_B$. Substituting $w_R=w_B$, we get $0.7w_B + 0.2w_S = w_B$, which simplifies to $0.2w_S = 0.3w_B$, or $w_S = \frac{3}{2}w_B$. Using the condition $w_B + w_S + w_R = 1$, we have $w_B + \frac{3}{2}w_B + w_B = 1$. This becomes $3.5w_B = 1$, so $w_B = 1/3.5 = 2/7$. Then $w_R = 2/7$ and $w_S=3/7$. The long-run probability of a Recession is $w_R = 2/7$. The fact that the economy starts in a Boom is irrelevant to the long-run probability. Distractor 1/10 is the one-step probability. Distractor 9/50 is the two-step probability.
7. Let $P$ be the transition matrix for a regular Markov chain, and let $W$ be its unique steady-state vector. Which of the following statements best describes the matrix $P^n$ as the number of steps $n$ approaches infinity?
   Answer: $P^n$ approaches a matrix where each row is the steady-state vector $W$. For a regular Markov chain, the powers of the transition matrix, $P^n$, converge to a limiting matrix as $n \to \infty$. This limiting matrix, often denoted as $\bar{P}$, has the property that all of its rows are identical and equal to the unique steady-state vector $W$. This signifies that the long-run probability distribution of being in any state is independent of the initial state. The other options are incorrect: $P^n$ does not generally approach the identity or zero matrix, and the convention is for rows, not columns, to be the steady-state vector.
8. A weather model uses a Markov chain where each day is classified as Sunny (S) or Rainy (R). The transition probabilities are: P(tomorrow sunny | today sunny) = 0.8, P(tomorrow rainy | today rainy) = 0.6. After many days, what is the long-run probability that any randomly selected day will be rainy?
   Answer: $\frac{1}{3}$ because rainy days are less persistent than sunny days in this model When you encounter Markov chain problems asking for long-run probabilities, you're looking for the steady-state distribution where the probability of being in each state remains constant over time. This occurs when the rate of flow into each state equals the rate of flow out. To find the steady-state probabilities, let $\pi_S$ and $\pi_R$ represent the long-run probabilities of sunny and rainy days. Since probabilities must sum to 1, we have $\pi_S + \pi_R = 1$. At equilibrium, the flow into each state equals the flow out: For the rainy state: $\pi_R = 0.2\pi_S + 0.6\pi_R$ This equation says the long-run probability of rain equals the probability of transitioning from sunny to rainy (0.2) times the probability of being sunny, plus the probability of staying rainy (0.6) times the probability of being rainy. Solving: $\pi_R = 0.2(1-\pi_R) + 0.6\pi_R$ $\pi_R = 0.2 - 0.2\pi_R + 0.6\pi_R$ $\pi_R = 0.2 + 0.4\pi_R$ $0.6\pi_R = 0.2$ $\pi_R = \frac{1}{3}$ Answer A incorrectly assumes equal probabilities despite unequal transition rates. Answer B confuses the conditional probability of staying rainy with the unconditional long-run probability. Answer C gives $\frac{2}{5}$, which would result from incorrect arithmetic in the balance equations. Remember: steady-state problems require setting up flow equations, not just reading transition probabilities directly. The long-run distribution depends on the entire transition matrix structure.
9. A transition matrix $P$ is called regular if some power $P^k$ contains only positive entries. The existence of a regular power guarantees that the Markov chain has a unique steady-state vector that is independent of the initial state. Which of the following transition matrices is regular?
   Answer: $C = \begin{pmatrix} 0.5 & 0.5 & 0 \\ 0 & 0.5 & 0.5 \\ 0.5 & 0 & 0.5 \end{pmatrix}$ We need to check if any power of the matrices has all positive entries. A) $A^2 = I$, $A^3 = A$, so powers of $A$ will always contain zeros. B) State 1 is an absorbing state. The entry $B_{12}$ will always be 0 in any power $B^k$. C) Let's compute $C^2$. $C^2 = \begin{pmatrix} 0.25 & 0.5 & 0.25 \\ 0.25 & 0.25 & 0.5 \\ 0.5 & 0.25 & 0.25 \end{pmatrix}$. Since all entries of $C^2$ are positive, matrix $C$ is regular. D) States 1 and 2 form a sub-chain that cannot reach state 3. The entries $D_{13}$ and $D_{23}$ will always be 0 in any power $D^k$. Therefore, only matrix C is regular.
10. A Markov chain has transition matrix $P = \begin{pmatrix} 0.2 & 0.8 & 0 \\ 0.3 & 0.4 & 0.3 \\ 0 & 0.5 & 0.5 \end{pmatrix}$. Based on the structure of this matrix, which statement about the steady-state behavior is most accurate?
    Answer: The chain is irreducible and aperiodic, so a unique steady-state distribution exists and can be reached from any initial state Examining the transition matrix: from state 1 we can reach states 1 and 2; from state 2 we can reach all states; from state 3 we can reach states 2 and 3. This means all states can eventually communicate through state 2 (1→2→3 and 3→2→1), making the chain irreducible. Since there are self-loops with positive probability, the chain is aperiodic. Therefore, a unique steady state exists. Choice A is wrong - state 3 isn't absorbing ($p_{33} = 0.5 \neq 1$). Choice C misses that indirect communication through state 2 makes all states accessible. Choice D incorrectly assumes zero entries prevent equilibrium.
11. If the mouse is left to wander the maze for a very long time, what is the probability of finding it in Chamber 3?
    Answer: 1/4 First, construct the transition matrix $P$ from the description, where rows represent the current chamber and columns represent the next chamber. The states are (1, 2, 3). $P = \begin{pmatrix} 0 & 1 & 0 \\ 0.5 & 0 & 0.5 \\ 0.75 & 0 & 0.25 \end{pmatrix}$. Let the steady-state vector be $W = (w_1, w_2, w_3)$. We solve the system $WP = W$ with $w_1 + w_2 + w_3 = 1$. The equations are: $w_1 = 0.5w_2 + 0.75w_3$; $w_2 = w_1$; $w_3 = 0.5w_2 + 0.25w_3$. From the second equation, $w_1 = w_2$. The third equation simplifies to $0.75w_3 = 0.5w_2$, which means $3w_3 = 2w_2$, or $w_3 = \frac{2}{3}w_2$. Now, substitute these into the sum equation: $w_2 + w_2 + \frac{2}{3}w_2 = 1$. This gives $\frac{8}{3}w_2 = 1$, so $w_2 = 3/8$. Since $w_1=w_2$, $w_1=3/8$. Finally, $w_3 = \frac{2}{3}w_2 = \frac{2}{3} (\frac{3}{8}) = \frac{2}{8} = 1/4$. The long-run probability of finding the mouse in Chamber 3 is $w_3$, which is 1/4.
12. Two different Markov chains both have the same steady-state distribution $\left(\frac{1}{4}, \frac{3}{4}\right)$. Chain X has transition matrix $P_X = \begin{pmatrix} 0.7 & 0.3 \\ 0.1 & 0.9 \end{pmatrix}$ and Chain Y has transition matrix $P_Y = \begin{pmatrix} 0.1 & 0.9 \\ 0.3 & 0.7 \end{pmatrix}$. Which statement about these chains is correct?
    Answer: Chain Y converges to steady state faster because its second eigenvalue has a smaller absolute value Both matrices are valid transition matrices (rows sum to 1) and both have steady state $\left(\frac{1}{4}, \frac{3}{4}\right)$. For convergence rate, we examine the second eigenvalue. For $P_X$: eigenvalues are 1 and $0.7 + 0.9 - 1 = 0.6$. For $P_Y$: eigenvalues are 1 and $0.1 + 0.7 - 1 = -0.2$. Since $|{-0.2}| = 0.2 < 0.6$, Chain Y converges faster. Choice A incorrectly assumes high diagonal entries mean faster convergence. Choice B ignores that convergence rates can differ. Choice D is wrong - we can verify that $P_Y$ does have the given steady state by checking $\left(\frac{1}{4}, \frac{3}{4}\right) P_Y = \left(\frac{1}{4}, \frac{3}{4}\right)$.
13. Voters in a district are registered with either Party A or Party B. Each election cycle, 85% of Party A voters remain with their party, while 15% switch to Party B. Concurrently, 95% of Party B voters remain with their party, while 5% switch to Party A. In the long run, what proportion of the electorate is expected to be aligned with Party A?
    Answer: 25% Let the states be Party A and Party B. The transition matrix is $P = \begin{pmatrix} 0.85 & 0.15 \\ 0.05 & 0.95 \end{pmatrix}$. Let the steady-state vector be $W = (w_A, w_B)$. We need to solve $WP = W$ with $w_A + w_B = 1$. The first component of $WP=W$ is $0.85w_A + 0.05w_B = w_A$. This simplifies to $0.05w_B = 0.15w_A$, or $w_B = 3w_A$. Using the constraint $w_A + w_B = 1$, we can substitute to get $w_A + 3w_A = 1$, which gives $4w_A = 1$, so $w_A = 0.25$. Therefore, in the long run, 25% of the electorate is expected to be aligned with Party A.
14. Consider the Markov chain with the transition matrix $P = \begin{pmatrix} 0.5 & 0.2 & 0.3 \\ 0.1 & 0.6 & 0.3 \\ 0.1 & 0.1 & 0.8 \end{pmatrix}$. Let its steady-state vector be $W = (w_1, w_2, w_3)$. What is the ratio $w_1/w_3$?
    Answer: 5/18 The steady-state vector $W$ satisfies $WP = W$. This gives a system of linear equations. The first and third equations are: (1) $0.5w_1 + 0.1w_2 + 0.1w_3 = w_1 \implies -0.5w_1 + 0.1w_2 + 0.1w_3 = 0 \implies 5w_1 = w_2 + w_3$. (2) $0.3w_1 + 0.3w_2 + 0.8w_3 = w_3 \implies 0.3w_1 + 0.3w_2 - 0.2w_3 = 0 \implies 3w_1 + 3w_2 = 2w_3$. We want to find the ratio $w_1/w_3$, so we can eliminate $w_2$. From equation (1), $w_2 = 5w_1 - w_3$. Substitute this into equation (2): $3w_1 + 3(5w_1 - w_3) = 2w_3$. This simplifies to $3w_1 + 15w_1 - 3w_3 = 2w_3$, which gives $18w_1 = 5w_3$. Therefore, the ratio $w_1/w_3 = 5/18$.
15. A Markov chain has transition matrix $P = \begin{pmatrix} 0.7 & 0.3 \\ 0.4 & 0.6 \end{pmatrix}$. If the system starts with initial distribution $(0.2, 0.8)$, what will be the probability of being in state 1 after the system reaches steady state?
    Answer: $\frac{4}{7}$ To find the steady-state distribution, we solve $\pi P = \pi$ where $\pi = (\pi_1, \pi_2)$. This gives us $0.7\pi_1 + 0.4\pi_2 = \pi_1$ and $0.3\pi_1 + 0.6\pi_2 = \pi_2$, along with $\pi_1 + \pi_2 = 1$. From the first equation: $-0.3\pi_1 + 0.4\pi_2 = 0$, so $\pi_2 = \frac{3}{4}\pi_1$. Substituting into $\pi_1 + \pi_2 = 1$: $\pi_1 + \frac{3}{4}\pi_1 = 1$, giving $\pi_1 = \frac{4}{7}$. The initial distribution is irrelevant for steady state. Choice B uses the wrong ratio. Choice C incorrectly assumes initial conditions matter. Choice D confuses transition probabilities with steady-state probabilities.

Start this quiz