Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [markov-process]

A stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. Use this tag for general state space processes (both discrete and continuous times); use (markov-chains) for countable state space processes.

A Markov process is a stochastic process satisfying the Markov property: the distribution of the future states given the value of the current state does not depend on the past states. This tag is used for general state space processes both in discrete and continuous time, for countable state spaces use .

2574 questions
2
votes
2 answers

Busy times for a M/M/1 queue

I have an M/M/1 queue with people arriving Poisson with parameter $\lambda$ and a service time exponentially distributed rate $\mu$. I have been asked to find the average time between the first person arriving at the queue and the queue being empty…
SalamanderSylph
  • 83
1
vote
1 answer

A question about Markov process

Let a stochastic process $(x(t),\theta(t))$ be given by $$ \dot{x}(t)=f(x(t),\theta(t)) $$ for a well defined continuous function $f(\cdot,\cdot)$. Let $\mathcal{F}_t$ denote the natural filtration of $(x(t),\theta(t))$ on the interval $[0,t]$.…
Ron
  • 143
1
vote
1 answer

Poisson process of sending e-mails

Suppose a person receives and sends e-mails according to a Poisson Process with independent intensities mu and lambda respectively. He reads a received e-mail with probability p and deletes the mail unread with probability 1-p. Suppose that person…
user109707
  • 61
  • 3
1
vote
0 answers

Three-State Markov Process with Differential Equations

This question is from a take home quiz and I could really use the help. Thanks in advance
user138063
  • 11
1
vote
0 answers

Probabilistic event triggered on a Markov Process transition

I would like to assess a system disponibility using a Markov Process. This system has two states : a functionning state 0 and a failure state 1, with a fault rate $\lambda$ and a mean time to repair MTTR = $1/\mu$. My problem is that this system…
thomasc
  • 111
  • 1
1
vote
0 answers

Calculate probability of Discrete-time Counting Process

Consider a discrete-time counting process $X_n$ with $n = 0,1,...$ so that $X_0 = 0$ and $X_n = X_{n-1} + \xi_n$ for each $n \geq 1$, where $\xi_1,\xi_2,\ldots$ are iid Bernoulli random variables with the common probability of success $p =…
waterr
  • 67
1
vote
0 answers

Understanding the generator of an interacting particle system

Let $\mathcal{S}$ be the state space and $f$ be a function from $\mathcal{S}$ to $\mathbb{R}$. An interacting particle system is a continuous-time Markov processes with generator $$ Gf(x) = \sum_{m}r_m(f(m(x)) - f(x)), $$ where $m$ are some…
Nicolas Bourbaki
  • 59
1
vote
0 answers

Derivation formula for piecewise-deterministic Markov process.

I was reading a formulation of piecewise-deterministic Markov process $\Pi_t$, $ t \in \mathbb{R}_+$. In particular $\Pi_t$ is defined as $\Pi_t = P (X_t | \mathcal{F}_{\lfloor t/ \Delta \rfloor}) $, where $X_t$ is a continuous time homogeneous…
Smalldinosaur77
  • 13
1
vote
1 answer

Markov Property definition via conditional expectation

In several textbooks I have seen the following equivalent statement for the Markov property: Let $\{X\}_{t \geq 0}$ be a stochastic process, $\mathcal{F}_u^v = \sigma\{X_s, s \in [u,v]\}$. Then $\{X_s\}_s$ has the Markov property iff for all $0 \leq…
bs_math
  • 957
1
vote
0 answers

In Markov Processes, using trasition function to define a probability over sequence of shocks.

In the context of Markov Processes, suposse I have a measurable space $(Z,\mathcal{Z})$ and a transition function $Q: Z \times \mathcal{Z} \to [0,1]$. We know that given $z_0 \in Z$, $$Q(z_0,A) = P(\tilde{z}_{t+1} \in A\,\,|\,\,\tilde{z}_t = z_0…
Fam
  • 825
1
vote
0 answers

Value function optimization problem

How to show that V* (optimal value function), is the solution to the following optimization problem: $min_V~ \Sigma_s V(s) $ with the constraint: $ V \ge T^{*}V $ Where $T^{*}$ is the optimal Bellman operator Thanks
nolwww
  • 121
1
vote
0 answers

Unique Invariant Measure for Markov Process on Interval

Suppose $(X_n)_{n\in \mathbb N_0}$ is a discret-time Markov process on state space $(0,1]$ with transition kernel $\kappa(x,\cdot)$ possessing an absolute continuous (Lebesgue-)density for all $x\in (0,1]$ with support $(0,1]$. The transition…
maliesen
  • 582
1
vote
0 answers

is the limit of a Markov process sequence a Markov process?

Assume $\{ X_t^n, \: t \geq 0 \}$ is a sequence of Markov processes (think in terms of diffusions) defined on some probability space $(\Omega, \mathcal{F}, \mathbb{P})$. Assume that there exists $\{ X_t, \: t \geq 0 \}$ a process defined on the…
megaproba
  • 267
1
vote
0 answers

Reconciling the Markov property for different types of time

Suppose $\mathbf{X} = \{X_t\}_{t \in T}$ is a random process, with $T$ being totally ordered. Let $\{\mathcal{F}_t\}_{t \in T}$ be the natural filtration generated by $\mathbf{X}$. I have seen the following definitions of the Markov property. We…
Mike Ho
  • 50
1
vote
0 answers

markov property expressed for canonical process for changing probability measure?

I am really confused for the continuous time Markov process when they use the notation such as $E_x[Z]$ or $E_{\mu}[Z]$, and even more for $E_x[Z]|_{x=Y_t}$. The Markov property expressed for the canonical process state as the following: For any $t…
micatske
  • 19
1
2
3 4 5