This Article Full Text (PDF) e-companion Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Google Scholar Articles by Heidergott, B. Articles by Leder, N.

Series Expansions for Continuous-Time Markov Processes

Department of Econometrics, Vrije Universiteit Amsterdam, and Tinbergen Institute, 1081 HV Amsterdam, The Netherlands
Mathematical Institute, Leiden University, 2300 RA Leiden, The Netherlands
Department of Mathematics, University of Hamburg, 20146 Hamburg, Germany
bheidergott{at}feweb.vu.nl
hordijk{at}math.leidenuniv.nl
nicole.leder{at}studium.uni-hamburg.de

We present update formulas that allow us to express the stationary distribution of a continuous-time Markov process with denumerable state space having generator matrix Q^* through a continuous-time Markov process with generator matrix Q. Under suitable stability conditions, numerical approximations can be derived from the update formulas, and we show that the algorithms converge at a geometric rate. Applications to sensitivity analysis and bounds on perturbations are discussed as well. Numerical examples are presented to illustrate the efficiency of the proposed algorithm.

Subject classifications: probability; Markov processes; queues; algorithms; numerical methods; retrial queue; deviation matrix; sensitivity analysis.
History: Received August 2007; revision received November 2008; accepted March 2009.