Markov Chains & Steady States - Linear Algebra

  1. What is the steady-state vector of the column-stochastic matrix P=[[0.7,0.4],[0.3,0.6]]P = [[0.7, 0.4], [0.3, 0.6]]?
    (47,37)(\frac{4}{7}, \frac{3}{7}). −0.3q1+0.4q2=0-0.3q_1 + 0.4q_2 = 0 gives q1:q2=4:3q_1 : q_2 = 4 : 3.
  2. What is the steady-state vector of the column-stochastic matrix P=[[0.9,0.2],[0.1,0.8]]P = [[0.9, 0.2], [0.1, 0.8]]?
    (23,13)(\frac{2}{3}, \frac{1}{3}). (P−I)q=0(P - I)\mathbf{q} = 0 gives −0.1q1+0.2q2=0-0.1q_1 + 0.2q_2 = 0, so q1=2q2q_1 = 2q_2.
  3. With P=[[0.9,0.2],[0.1,0.8]]P = [[0.9, 0.2], [0.1, 0.8]] and x0=(1,0)\mathbf{x}_0 = (1, 0), what is x1=Px0\mathbf{x}_1 = P\mathbf{x}_0?
    (0.9,0.1)(0.9, 0.1), the first column of PP.
  4. In a column-stochastic matrix with xk+1=Pxk\mathbf{x}_{k+1} = P\mathbf{x}_k, what does entry pijp_{ij} represent?
    The probability of moving from state jj to state ii in one step.
  5. What is a column-stochastic (transition) matrix?
    A square matrix with nonnegative entries in which every column sums to 11.
  6. What makes a stochastic matrix PP regular?
    Some power PkP^k has all entries strictly positive.
  7. What is a steady-state vector q\mathbf{q} of a stochastic matrix PP?
    A probability vector with Pq=qP\mathbf{q} = \mathbf{q}, that is, an eigenvector for eigenvalue 11 scaled so its entries sum to 11.
  8. What are the eigenvalues of the column-stochastic matrix [[0.9,0.2],[0.1,0.8]][[0.9, 0.2], [0.1, 0.8]]?
    11 and 0.70.7. One eigenvalue is 11, and they sum to the trace 1.71.7.
  9. For a regular stochastic matrix PP, what happens to xk=Pkx0\mathbf{x}_k = P^k\mathbf{x}_0 as kk grows?
    It converges to the unique steady-state vector, whatever probability vector x0\mathbf{x}_0 is.
  10. What is a probability vector?
    A vector with nonnegative entries that sum to 11.
  11. Why does every column-stochastic matrix PP have eigenvalue 11?
    Each column sums to 11, so (1,…,1)(1, \ldots, 1) is an eigenvector of PTP^T for 11, and PP has the same eigenvalues as PTP^T.
  12. With P=[[0,1],[1,0]]P = [[0, 1], [1, 0]] and x0=(1,0)\mathbf{x}_0 = (1, 0), what happens to xk\mathbf{x}_k?
    It alternates between (1,0)(1, 0) and (0,1)(0, 1) and never converges; PP is not regular.

Study these flashcards