Linear Algebra Quiz: Markov Chains And Steady States
12 questions · exam conditions
0:00
Markov Chains And Steady StatesQuestion 1 of 12

A market study tracks customer loyalty between two competing food delivery services, 'QuickEat' and 'GoGrub'. Each month, QuickEat retains 70% of its customers, while 30% switch to GoGrub. GoGrub retains 80% of its customers, while 20% switch to QuickEat. The state vector is defined as x⃗=(customers of QuickEatcustomers of GoGrub)\vec{x} = \begin{pmatrix} \text{customers of QuickEat} \\ \text{customers of GoGrub} \end{pmatrix}.

Based on the study, which matrix PP correctly represents the monthly transition of customers, where the next month's state vector x⃗k+1\vec{x}_{k+1} is given by Px⃗kP\vec{x}_k?

← Back to quizzes

Linear Algebra Quiz

Linear Algebra Quiz: Markov Chains And Steady States

Practice Markov Chains And Steady States in Linear Algebra with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Markov Chains And Steady States, giving you a quick way to practice the rules, question types, and explanations that matter most for Linear Algebra.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

A market study tracks customer loyalty between two competing food delivery services, 'QuickEat' and 'GoGrub'. Each month, QuickEat retains 70% of its customers, while 30% switch to GoGrub. GoGrub retains 80% of its customers, while 20% switch to QuickEat. The state vector is defined as x⃗=(customers of QuickEatcustomers of GoGrub)\vec{x} = \begin{pmatrix} \text{customers of QuickEat} \\ \text{customers of GoGrub} \end{pmatrix}.

Based on the study, which matrix PP correctly represents the monthly transition of customers, where the next month's state vector x⃗k+1\vec{x}_{k+1} is given by Px⃗kP\vec{x}_k?

  2. (correct answer)
Explanation: The transition matrix PP should have columns that represent the movement 'from' a state. The first column describes customers starting with QuickEat: 70% stay with QuickEat (row 1) and 30% switch to GoGrub (row 2). So, the first column is (0.70.3)\begin{pmatrix} 0.7 \\ 0.3 \end{pmatrix}. The second column describes customers starting with GoGrub: 20% switch to QuickEat (row 1) and 80% stay with GoGrub (row 2). So, the second column is (0.20.8)\begin{pmatrix} 0.2 \\ 0.8 \end{pmatrix}. Combining these gives the correct matrix P=(0.70.20.30.8)P = \begin{pmatrix} 0.7 & 0.2 \\ 0.3 & 0.8 \end{pmatrix}.

Question 2

A city's transportation habits are studied. Each year, 10% of car commuters switch to public transit, and 5% of public transit users switch to commuting by car.

If this trend continues, what will be the long-term percentage of residents who commute by car?

  1. 33.3ˉ%33.\bar{3}\% (correct answer)
  2. 50%50\%
  3. 66.6ˉ%66.\bar{6}\%
  4. 75%75\%
Explanation: Let state 1 be car and state 2 be public transit. The transition matrix is P=(0.900.050.100.95)P = \begin{pmatrix} 0.90 & 0.05 \\ 0.10 & 0.95 \end{pmatrix}. To find the steady-state vector q⃗\vec{q}, we solve (P−I)q⃗=0⃗(P-I)\vec{q}=\vec{0}. This gives −0.10q1+0.05q2=0-0.10q_1 + 0.05q_2 = 0, or 2q1=q22q_1=q_2. The eigenvector is of the form (k2k)\begin{pmatrix} k \\ 2k \end{pmatrix}. Since q1+q2=1q_1+q_2=1, we have k+2k=1k+2k=1, so k=1/3k=1/3. The steady-state vector is q⃗=(1/32/3)\vec{q} = \begin{pmatrix} 1/3 \\ 2/3 \end{pmatrix}. The long-term percentage of car commuters is q1q_1, which is 1/31/3 or 33.3ˉ%33.\bar{3}\%.

Question 3

A Markov chain has a transition matrix PP and an initial state vector x⃗0=(0.20.8)\vec{x}_0 = \begin{pmatrix} 0.2 \\ 0.8 \end{pmatrix}. The steady-state vector is found to be q⃗=(0.60.4)\vec{q} = \begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix}. Which statement correctly describes the system's evolution?

  1. The system is already in equilibrium, so x⃗k=x⃗0\vec{x}_k = \vec{x}_0 for all k≥1k \ge 1.
  2. The state vector x⃗k\vec{x}_k will approach (0.60.4)\begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix} as kk becomes large, regardless of the initial state. (correct answer)
  3. The state vector x⃗k\vec{x}_k will oscillate and never converge to a single vector because the initial state is different from the steady state.
  4. The system will converge to the steady state only if the initial state vector was (0.50.5)\begin{pmatrix} 0.5 \\ 0.5 \end{pmatrix}.
Explanation: For a regular Markov chain, the sequence of state vectors x⃗k=Pkx⃗0\vec{x}_k = P^k\vec{x}_0 converges to the unique steady-state vector q⃗\vec{q} for any initial probability vector x⃗0\vec{x}_0. The fact that x⃗0\vec{x}_0 is different from q⃗\vec{q} simply means the system is not initially in equilibrium, but it will approach it over time.

Question 4

The matrix P=(1−aba1−b)P = \begin{pmatrix} 1-a & b \\ a & 1-b \end{pmatrix} is the transition matrix for a regular Markov chain, with 0<a<10 < a < 1 and 0<b<10 < b < 1. What is the first component of the steady-state vector q⃗\vec{q}?

  1. aa+b\frac{a}{a+b}
  2. ba+b\frac{b}{a+b} (correct answer)
  3. 1−a2−a−b\frac{1-a}{2-a-b}
  4. b1+b−a\frac{b}{1+b-a}
Explanation: To find the steady-state vector, we solve (P−I)q⃗=0⃗(P-I)\vec{q} = \vec{0}. The matrix P−IP-I is (−aba−b)\begin{pmatrix} -a & b \\ a & -b \end{pmatrix}. The first row gives the equation −aq1+bq2=0-aq_1 + bq_2 = 0, which implies aq1=bq2aq_1 = bq_2. A non-trivial solution is \vec{q} = \begin{pmatrix} b \\ a \end{pmatrix}}. To make this a probability vector, we normalize by dividing by the sum of the components, a+ba+b. The normalized steady-state vector is q⃗=(b/(a+b)a/(a+b))\vec{q} = \begin{pmatrix} b/(a+b) \\ a/(a+b) \end{pmatrix}. The first component is ba+b\frac{b}{a+b}.

Question 5

Which of the following matrices cannot be a transition matrix for a regular Markov chain?

  1. (0.10.20.30.40.50.30.50.30.4)\begin{pmatrix} 0.1 & 0.2 & 0.3 \\ 0.4 & 0.5 & 0.3 \\ 0.5 & 0.3 & 0.4 \end{pmatrix}
  2. (00.50.50.500.50.50.50)\begin{pmatrix} 0 & 0.5 & 0.5 \\ 0.5 & 0 & 0.5 \\ 0.5 & 0.5 & 0 \end{pmatrix}
  3. (0.5100.500.5000.5)\begin{pmatrix} 0.5 & 1 & 0 \\ 0.5 & 0 & 0.5 \\ 0 & 0 & 0.5 \end{pmatrix}
  4. (010100001)\begin{pmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix} (correct answer)
Explanation: A regular Markov chain is one for which some power of the transition matrix, PkP^k, has all positive entries. This guarantees convergence to a unique steady-state vector regardless of the initial state. Matrix (D) describes a system with two states that trade places and a third state that is absorbing. If the system starts in state 1 or 2, it will oscillate between them (P2P^2 will have zeros) and never reach state 3. If it starts in state 3, it stays there. The long-term behavior depends on the initial state, so the chain is not regular. Matrix (A) and (B) are regular because they are positive. Matrix (C) is also regular as P2P^2 will have all positive entries in the top-left 2×22 \times 2 block and probability will eventually leak into the third state from there.

Question 6

A system's state is described by the vector x⃗k\vec{x}_k at time kk, and it evolves according to x⃗k+1=Px⃗k\vec{x}_{k+1} = P\vec{x}_k with P=(0.50.50.50.5)P = \begin{pmatrix} 0.5 & 0.5 \\ 0.5 & 0.5 \end{pmatrix}. If the initial state is x⃗0=(0.80.2)\vec{x}_0 = \begin{pmatrix} 0.8 \\ 0.2 \end{pmatrix}, what is the state vector x⃗2\vec{x}_2?

  1. (0.50.5)\begin{pmatrix} 0.5 \\ 0.5 \end{pmatrix} (correct answer)
  2. (0.80.2)\begin{pmatrix} 0.8 \\ 0.2 \end{pmatrix}
  3. (0.40.1)\begin{pmatrix} 0.4 \\ 0.1 \end{pmatrix}
  4. (0.650.35)\begin{pmatrix} 0.65 \\ 0.35 \end{pmatrix}
Explanation: This is a discrete dynamical system problem where you apply matrix multiplication iteratively to track how a state vector evolves over time. When you see x⃗k+1=Px⃗k\vec{x}_{k+1} = P\vec{x}_k, you're looking at a linear transformation that gets applied repeatedly. To find x⃗2\vec{x}_2, you need to apply the transformation matrix PP twice. First, calculate x⃗1=Px⃗0\vec{x}_1 = P\vec{x}_0: Then calculate x⃗2=Px⃗1\vec{x}_2 = P\vec{x}_1: Choice A is correct. Notice that this matrix has identical rows, which means any vector gets transformed into $$ \begin{pmatrix} 0.5 \ 0.5 \end{pmatrix} Choice B represents the initial state $$\vec{x}_0$$, suggesting the system doesn't change—a common misconception. Choice C looks like someone multiplied the initial vector by 0.5 element-wise rather than using proper matrix multiplication. Choice D appears to be an intermediate calculation error, possibly averaging incorrectly or stopping partway through the matrix multiplication. **Study tip:** With discrete systems, always compute step-by-step rather than trying shortcuts. Also, look for patterns—matrices with identical rows quickly converge to a fixed point, which can help you check your work.

Question 7

A Markov process is described by the transition matrix P=(0.50.250.50.75)P = \begin{pmatrix} 0.5 & 0.25 \\ 0.5 & 0.75 \end{pmatrix}. Which of the following vectors is the steady-state vector q⃗\vec{q} for this process?

  4. (correct answer)
Explanation: A steady-state vector q⃗\vec{q} must satisfy Pq⃗=q⃗P\vec{q} = \vec{q}, which is equivalent to solving (P−I)q⃗=0⃗(P-I)\vec{q} = \vec{0}. Here, P−I=(−0.50.250.5−0.25)P-I = \begin{pmatrix} -0.5 & 0.25 \\ 0.5 & -0.25 \end{pmatrix}. This gives the equation −0.5q1+0.25q2=0-0.5q_1 + 0.25q_2 = 0, which simplifies to 2q1=q22q_1 = q_2. An eigenvector is thus of the form (k2k)\begin{pmatrix} k \\ 2k \end{pmatrix}. For q⃗\vec{q} to be a probability vector, its components must sum to 1. So, k+2k=1k + 2k = 1, which gives 3k=13k=1 or k=1/3k=1/3. The steady-state vector is q⃗=(1/32/3)\vec{q} = \begin{pmatrix} 1/3 \\ 2/3 \end{pmatrix}.

Question 8

A system has three states. The transition matrix is given by P=(0.500.50100.500.5)P = \begin{pmatrix} 0.5 & 0 & 0.5 \\ 0 & 1 & 0 \\ 0.5 & 0 & 0.5 \end{pmatrix}. What is the most accurate description of state 2?

  1. State 2 is an absorbing state. (correct answer)
  2. State 2 is a transient state.
  3. The system will eventually reach state 2, regardless of the starting state.
  4. State 2 is part of a periodic cycle with state 1.
Explanation: When analyzing Markov chains, you need to examine how states behave by looking at the transition probabilities in the matrix. Each row represents transitions from one state, and each column represents transitions to a state. Looking at the transition matrix $$P = \begin{pmatrix} 0.5 & 0 & 0.5 \ 0 & 1 & 0 \ 0.5 & 0 & 0.5 \end{pmatrix} Nowexaminingthecolumnforstate2revealsthatnootherstatecantransitiontostate2(bothentriesare0).Thismeansstate2canonlybereachedifthesystemstartsthere.Let′scheckeachoption:Aiscorrectbecausestate2hasprobability1oftransitioningtoitself.Biswrongbecausetransientstateseventuallyleadelsewhere–state2neverleavesitself,makingitabsorbing,nottransient.Cisincorrectbecausethesystemcannotreachstate2fromstates1or3(thetransitionprobabilitiesarezero).Diswrongbecausestates1and3onlytransitionbetweeneachotherwithprobability0.5each–state2iscompletelyisolatedfromthiscycle.∗∗Studytip:∗∗ForMarkovchainproblems,alwayscheckthediagonalentriesfirst.Aprobabilityof1onthediagonalimmediatelysignalsanabsorbingstate,whileexaminingentirerowsandcolumnsrevealsthecompletestatebehavior. Now examining the column for state 2 reveals that no other state can transition to state 2 (both entries are 0). This means state 2 can only be reached if the system starts there. Let's check each option: A is correct because state 2 has probability 1 of transitioning to itself. B is wrong because transient states eventually lead elsewhere – state 2 never leaves itself, making it absorbing, not transient. C is incorrect because the system cannot reach state 2 from states 1 or 3 (the transition probabilities are zero). D is wrong because states 1 and 3 only transition between each other with probability 0.5 each – state 2 is completely isolated from this cycle. **Study tip:** For Markov chain problems, always check the diagonal entries first. A probability of 1 on the diagonal immediately signals an absorbing state, while examining entire rows and columns reveals the complete state behavior.

Question 9

A research lab has two groups of mice, A and B. Each week, 20% of mice from group A are moved to group B, and 30% of mice from group B are moved to group A. The total number of mice is 500.

In the long-term equilibrium, approximately how many mice will be in group A?

  1. 100
  2. 200
  3. 250
  4. 300 (correct answer)
Explanation: Let state 1 be group A and state 2 be group B. From group A, 80% stay and 20% go to B. From group B, 70% stay and 30% go to A. The transition matrix is P=(0.80.30.20.7)P = \begin{pmatrix} 0.8 & 0.3 \\ 0.2 & 0.7 \end{pmatrix}. To find the steady state q⃗\vec{q}, we solve (P−I)q⃗=0⃗(P-I)\vec{q}=\vec{0}, which is −0.2q1+0.3q2=0-0.2q_1 + 0.3q_2 = 0, or 2q1=3q22q_1 = 3q_2. An eigenvector is (32)\begin{pmatrix} 3 \\ 2 \end{pmatrix}. Normalizing gives q⃗=(3/52/5)=(0.60.4)\vec{q} = \begin{pmatrix} 3/5 \\ 2/5 \end{pmatrix} = \begin{pmatrix} 0.6 \\ 0.4 \end{pmatrix}. The long-term proportion of mice in group A is 0.6. With 500 mice in total, the number in group A will be 0.6×500=3000.6 \times 500 = 300.

Question 10

For a regular Markov chain with transition matrix PP and unique steady-state vector q⃗\vec{q}, what does the matrix power PkP^k approach as k→∞k \to \infty?

  1. The identity matrix, II.
  2. The zero matrix.
  3. A matrix where every column is the steady-state vector q⃗\vec{q}. (correct answer)
  4. A diagonal matrix whose diagonal entries are the components of q⃗\vec{q}.
Explanation: A key property of regular Markov chains is that as kk grows, the matrix PkP^k converges to a matrix where every column is the unique steady-state vector q⃗\vec{q}. This means that for any initial state x⃗0\vec{x}_0, the long-term state Pkx⃗0P^k \vec{x}_0 will approach (q⃗  q⃗  …  q⃗)x⃗0=q⃗(∑xi)=q⃗(\vec{q} \; \vec{q} \; \dots \; \vec{q})\vec{x}_0 = \vec{q} (\sum x_i) = \vec{q}, since the components of x⃗0\vec{x}_0 sum to 1.

Question 11

Let PP be an n×nn \times n column-stochastic matrix (all entries non-negative, columns sum to 1). Which of the following statements about PP is not guaranteed to be true?

  1. The number 1 is an eigenvalue of PP.
  2. If λ\lambda is an eigenvalue of PP, then ∣λ∣≤1|\lambda| \leq 1.
  3. The matrix PP has nn linearly independent eigenvectors. (correct answer)
  4. For any vector x⃗\vec{x}, the sum of the components of Px⃗P\vec{x} is equal to the sum of the components of x⃗\vec{x}.
Explanation: While many stochastic matrices are diagonalizable (meaning they have nn linearly independent eigenvectors), it is not a guaranteed property. For example, a matrix could have repeated eigenvalues where the geometric multiplicity is less than the algebraic multiplicity. Statements A, B, and D are always true for a column-stochastic matrix. Statement A is true because the columns of (P−I)(P-I) sum to the zero vector, making it singular. Statement B is a known property from the Perron-Frobenius theory for non-negative matrices. Statement D is true because the sum of the components of Px⃗P\vec{x} is (∑i∑jPijxj)=∑j(∑iPij)xj=∑j(1)xj(\sum_i \sum_j P_{ij}x_j) = \sum_j (\sum_i P_{ij})x_j = \sum_j (1)x_j, which is the sum of the components of x⃗\vec{x}.

Question 12

A direct justification for why any n×nn \times n column-stochastic matrix PP must have an eigenvalue of λ=1\lambda = 1 is that:

  1. the rows of the matrix P−IP-I are linearly dependent.
  2. the Perron-Frobenius theorem for positive matrices guarantees it.
  3. the columns of the matrix P−IP-I sum to the zero vector, implying they are linearly dependent. (correct answer)
  4. the trace of a stochastic matrix is always equal to 1.
Explanation: An eigenvalue λ\lambda exists if and only if the matrix (P−λI)(P-\lambda I) is singular, meaning its determinant is zero. For λ=1\lambda=1, we consider P−IP-I. The columns of PP each sum to 1. When we subtract the identity matrix II, we are subtracting 1 from each diagonal element. This means that for each column of P−IP-I, the sum of its elements becomes 1−1=01-1=0. If the elements of each column vector sum to zero, the sum of the row vectors of (P−I)(P-I) is the zero vector, meaning the rows (and columns) are linearly dependent. A matrix with linearly dependent columns is singular, so det⁡(P−I)=0\det(P-I)=0, proving that λ=1\lambda=1 is an eigenvalue.