WOLFRAM

Wolfram Language & System Documentation Center
Wolfram Language Home Page »

MarkovProcessProperties
✖
MarkovProcessProperties

✖

gives a summary of properties for the finite state Markov process mproc.

✖
MarkovProcessProperties[mproc,"property"]

gives the specified "property" for the process mproc.

Details

  • MarkovProcessProperties can be used for finite state Markov processes such as DiscreteMarkovProcess and ContinuousMarkovProcess.
  • MarkovProcessProperties[mproc,"Properties"] gives a list of available properties.
  • MarkovProcessProperties[mproc,"property","Description"] gives a description of the property as a string.
  • Basic properties include:
  • "InitialProbabilities"initial state probability vector
    "TransitionMatrix"conditional transition probabilities m
    "TransitionRateMatrix"conditional transition rates q
    "TransitionRateVector"state transition rates μ
    "HoldingTimeMean"mean holding time for a state
    "HoldingTimeVariance"variance of holding time for a state
    "SummaryTable"summary of properties
  • For a continuous-time Markov process "TransitionMatrix" gives the transition matrix of the embedded discrete-time Markov process.
  • The holding time is the time spent in each state before transitioning to a different state. This takes into account self-loops which may cause the process to transition to the same state several times.
  • Structural properties include:
  • "CommunicatingClasses"sets of states accessible from each other
    "RecurrentClasses"communicating classes that cannot be left
    "TransientClasses"communicating classes that can be left
    "AbsorbingClasses"recurrent classes with a single element
    "PeriodicClasses"communicating classes with finite period greater than 1
    "Periods"period for each of the periodic classes
    "Irreducible"whether the process has a single recurrent class
    "Aperiodic"whether all classes are aperiodic
    "Primitive"whether the process is irreducible and aperiodic
  • The states of a finite Markov process can be grouped into communicating classes where from each state in a class there is a path to every other state in the class.
  • A communicating class can be transient when there is a path from the class to another class or recurrent when there is not. A special type of recurrent class, called absorbing, consist of a single element.
  • A state is periodic is if there is a non-zero probability that you return to the state after two or more steps. All the states in a class have the same period.
  • Transient properties before the process enters a recurrent class:
  • "TransientVisitMean"mean number of visits to each transient state
    "TransientVisitVariance"variance of number of visits to each transient state
    "TransientTotalVisitMean"mean total number of transient states visited
  • A Markov process will eventually enter a recurrent class. The transient properties characterize how many times each transient state is visited or how many different transient states are visited.
  • Limiting properties include:
  • "ReachabilityProbability"probability of ever reaching a state
    "LimitTransitionMatrix"Cesaro limit of the transition matrix
    "Reversible"whether the process is reversible
  • If a property is not available, this is indicated by Missing["reason"].

Examples

open allclose all

Basic Examples  (2)Summary of the most common use cases

Summary table of properties:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-p278q0
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-h7yh56
Out[2]=2
In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-e8urqp
Out[3]=3

Find the values of a specific property:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-bn728p
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-jx883o
Out[2]=2

Description of the property:

In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-f9emjw
Out[3]=3

Scope  (5)Survey of the scope of standard use cases

Find the communicating classes, highlighted in the graph through different colors:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-c3vhvk
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-baaxz
Out[2]=2
In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-bz1as
Out[3]=3
In[4]:=4
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-bn7ei7
Out[4]=4

The process is not irreducible:

In[5]:=5
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-gdv4tu
Out[5]=5
In[6]:=6
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-fwrw3h
Out[6]=6

Find the recurrent classes, represented by square and circular vertices in the graph:

In[7]:=7
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-cd6c6
Out[7]=7
In[8]:=8
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-cavj19
Out[8]=8

Find the transient classes, represented by diamond vertices in the graph:

In[9]:=9
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-g9ogjy
Out[9]=9
In[10]:=10
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-hlna0d
Out[10]=10

Find the absorbing classes, represented by square vertices in the graph:

In[11]:=11
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-xpupe
Out[11]=11
In[12]:=12
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-ohwda9
Out[12]=12

Define a Markov process with self-loops:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-fnkucy
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-bkuq9
Out[2]=2

The self-loops make the class aperiodic:

In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-gy6et
Out[3]=3

Markov process with no self-loops:

In[4]:=4
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-onwdh
In[5]:=5
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-b4g9kx
Out[5]=5

Here both classes are periodic:

In[6]:=6
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-h6ftb2
Out[6]=6

Summary table of properties for a continuous-time Markov process:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-d4pogi
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-ji9n76
Out[2]=2
In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-lhfdep
Out[3]=3

Obtain the value for a specific property for a continuous Markov chain:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-gyhnsh
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-qx2ol
In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-rqo7g
Out[3]=3
In[4]:=4
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-z92o2
Out[4]=4

Find the conditional mean number of total transitions starting in state 1 and ending in state 4:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-d51imh
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-hyzqb5
Out[2]=2
In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-7xk7nw
Out[3]=3

Compare with the results from simulation:

In[4]:=4
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-bcoaf5
In[5]:=5
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-mgh6
Out[5]=5

Find the conditional mean number of transitions from state 2 to state 3:

In[6]:=6
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-kpurr5
In[7]:=7
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-bp7m2p
Out[7]=7
In[8]:=8
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-rffwot
Out[8]=8

Compare with the results from simulation:

In[9]:=9
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-fgz1vl
In[10]:=10
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-996qf
Out[10]=10

Applications  (2)Sample problems that can be solved with this function

A gambler starts with $3 and bets $1 at each step. He wins $1 with a probability of 0.4:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-jm8rd
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-fjsu75

Find the expected number of times the gambler has units:

In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-9bba0
Out[3]=3

Verify the answer using simulation:

In[4]:=4
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-f6pwuy
In[5]:=5
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-poxolm
Out[5]=5

Find the expected time until the gambler wins $7 or goes broke:

In[6]:=6
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-csbpiu
Out[6]=6

Total states visited before the gambler wins $7 or goes broke:

In[7]:=7
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-sanmh
Out[7]=7

Verify the answer using simulation:

In[8]:=8
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-izvqc
In[9]:=9
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-ph4m7
Out[9]=9

In a game of tennis between two players, suppose the probability of the server winning a point is . There are 17 possible states:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-6d74m

Visualize the random walk graph for :

In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-comxn4
In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-kca95k
Out[3]=3

Find the probability of the server winning the game if :

In[4]:=4
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-l2lmu
Out[4]=4
In[5]:=5
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-cxrmd
Out[5]=5

Find the mean time to absorption, that is, the number of points played:

In[6]:=6
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-ieac1q
Out[6]=6
In[7]:=7
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-j5rzf2
Out[7]=7

Find the mean number of states visited:

In[8]:=8
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-ba6ssz
Out[8]=8

Find the average number of times the score will be tied at deuce:

In[9]:=9
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-cwls51
Out[9]=9

Verify the answer using simulation:

In[10]:=10
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-4ceph
Out[10]=10

Properties & Relations  (1)Properties of the function, and connections to other functions

The transition matrix of this Markov process is not irreducible:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-dkwrbt
In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-i68fuk
Out[2]=2

Hence the stationary distribution depends on the initial state probabilities:

In[3]:=3
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-czxaeb
Out[3]=3
In[4]:=4
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-k2xypd
Out[4]=4

Possible Issues  (1)Common pitfalls and unexpected behavior

Some property values may not be available:

In[1]:=1
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-hoyily
Out[1]=1

This property is available only for continuous Markov processes:

In[2]:=2
✖
https://wolfram.com/xid/0bswb51q1xtjmobnx2-cfna6g
Out[2]=2
Wolfram Research (2012), MarkovProcessProperties, Wolfram Language function, https://reference.wolfram.com/language/ref/MarkovProcessProperties.html.
✖
Wolfram Research (2012), MarkovProcessProperties, Wolfram Language function, https://reference.wolfram.com/language/ref/MarkovProcessProperties.html.

Text

Wolfram Research (2012), MarkovProcessProperties, Wolfram Language function, https://reference.wolfram.com/language/ref/MarkovProcessProperties.html.

✖
Wolfram Research (2012), MarkovProcessProperties, Wolfram Language function, https://reference.wolfram.com/language/ref/MarkovProcessProperties.html.

CMS

Wolfram Language. 2012. "MarkovProcessProperties." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/MarkovProcessProperties.html.

✖
Wolfram Language. 2012. "MarkovProcessProperties." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/MarkovProcessProperties.html.

APA

Wolfram Language. (2012). MarkovProcessProperties. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/MarkovProcessProperties.html

✖
Wolfram Language. (2012). MarkovProcessProperties. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/MarkovProcessProperties.html

BibTeX

@misc{reference.wolfram_2025_markovprocessproperties, author="Wolfram Research", title="{MarkovProcessProperties}", year="2012", howpublished="\url{https://reference.wolfram.com/language/ref/MarkovProcessProperties.html}", note=[Accessed: 29-March-2025 ]}

✖
@misc{reference.wolfram_2025_markovprocessproperties, author="Wolfram Research", title="{MarkovProcessProperties}", year="2012", howpublished="\url{https://reference.wolfram.com/language/ref/MarkovProcessProperties.html}", note=[Accessed: 29-March-2025 ]}

BibLaTeX

@online{reference.wolfram_2025_markovprocessproperties, organization={Wolfram Research}, title={MarkovProcessProperties}, year={2012}, url={https://reference.wolfram.com/language/ref/MarkovProcessProperties.html}, note=[Accessed: 29-March-2025 ]}

✖
@online{reference.wolfram_2025_markovprocessproperties, organization={Wolfram Research}, title={MarkovProcessProperties}, year={2012}, url={https://reference.wolfram.com/language/ref/MarkovProcessProperties.html}, note=[Accessed: 29-March-2025 ]}