Comparing transition matrices for Markov chains



I have a population, each unit of which exists in one of several states that change over time. I am using first-order Markov chains to model these state transitions.

My population can be segmented into various subpopulations of interest. I've obtained the transition matrices for each of these subpopulations, and would like to know if these subpopulations differ from the general population in some principled way.

I don't know of any principled way of comparing transition matrices in this way. Comparing the transition matrices row-wise to that of the general population seems like one approach, but I'm not sure how to go about interpreting this. Another approach might be a spectral/eigendecomposition approach, which is much more readily interpretable to me, but might be harder to squeeze insight/stylised facts from.

I've had a cursory search of the literature without much luck. Any suggestions?

R Hill

Posted 2017-04-19T13:43:35.413

Reputation: 927


Continuing your first idea, you can use any discrete-probability distance-metric over the rows. Compare the rows of sub-population to the corresponding rows of the general population using measurements like Total Variation, KL divergence, Wasserstein, Kolmogorov-Smirnoff etc. You can read more to see what method fits you the most.

– ehudk – 2017-04-19T15:13:43.423

2Do you know anything about the distribution on states (i.e., what proportion of time is spent in each state)? If so, one way is to do a weighted average of the differences between corresponding row, weighted by the fraction of time spent in the state corresponding to that row. Did this make any sense? Do you have the same set of states for all subpopulations? – D.W. – 2017-04-19T23:49:27.347

That does make sense, and is applicable to my case. Thank you. – R Hill – 2017-04-21T10:41:22.107

1What about using the eigenspectrum? It would be more focus on comparing the steady-states and not the transition but it looks to me like a sound solution. – Garini – 2019-09-30T09:56:08.367

No answers