Markov Chains
General Information: Markov Chains are designed to model systems that change from state to state. In particular, the current state should depend only on the previous state. For example, a city’s weather could be in one of three possible states: sunny, cloudy, or raining. Here, linear algebra is used to predict future conditions.
Instructions:
1. Find information about transition matrices and Markov chains. Recommend a quick scan of
Wikipedia. Helpful Link. An important equation to know is P xp = xp or (P − I)xp = 0.
2. Let P =
0.10 0.05 0 0.25 0.33
0.20 0.35 0 0.25 0.32
0.30 0.10 0.35 0.25 0
0.15 0.40 0.55 0.25 0
0.25 0.10 0.10 0 0.35
be the transition matrix of a Markov chain
a. What is the probability that an individual at site 2 (the intitial state vector is (0, 1, 0, 0, 0)T ) will move to site 5 in three steps?
b. Suppose that 100 individuals are initially uniformly distribute at the five sites. How will individuals be distributed after four steps?
c. Find the steady state vector of P. (You might want to use MATLAB for this.)
3. Modeling Influenza At Malady College, a college campus of 5000 students, the spread of influenza is rampant. In this problem, we will call each student either susceptible or infected with influenza, and if the student is not infected they are susceptible. During any given year, the percentage of the U.S. population that will get the flu, on average, each year is between 5% and 20%. At Malady, if a student is not infected with influenza the chance that they will
catch the flu on any given day is 16 %, and if a student has the flu the chance that they will recover and return to susceptible on any given day is 40 %.
a. Create a matrix, A, (called the transition matrix) whose columns represent the current state a student, either susceptible or infected, may be in and whose rows represent the state of a student, either susceptible or infected, tomorrow, and where Ai,j is the probability of a student going from current state j today to state i tomorrow.
b. If 100 students have the flu initially, how many students have the flu on the second day? How many students have the flu on the tenth day?
c. How many days does it take for the number of students with the flu to stabilize?
d. How many students have the flu initially if there are 1400 students with the flu on the third
day?
