Create an aipowered research feed to stay up to date with new papers like this posted to arxiv. On perturbation bounds for continuoustime markov chains. New perturbation bounds for denumerable markov chains new perturbation bounds for denumerable markov chains mouhoubi, zahir. Denumerable markov chains can be used to represent many real systems. It considers two main methods used to study stability and to obtain appropriate quantitative estimates of perturbations of inhomogeneous markov chains with continuous time and a finite or countable state space.
Measurevalued differentiation for stationary markov. Siam journal on numerical analysis society for industrial. Two approaches to the construction of perturbation bounds for. A critical account of perturbation analysis of markov chains. Perturbation bounds for quantum markov processes and their. Gaussian elimination, perturbation theory and markov. Perturbation theory for markov reward processes with.
Meyer 1992 has developed inequalities in terms of the nonunit eigenvalues h, j 2. Jul 17, 2006 2011 perturbation analysis of continuoustime absorbing markov chains. In the present paper we propose an approach to the construction of general estimates for the perturbation bounds of markov chains in terms of special weighted. Perturbation analysis for continuoustime markov chains. We study general statespace markov chains that depend on a parameter, say, sufficient conditions are established for the stationary performance of such a markov chain to be differentiable with respect to. We present update formulas that allow us to express the stationary distribution of a continuoustime markov process with denumerable state space having generator matrix q through a continuoustime markov process with generator matrix q.
We perform perturbation analysis in the setting of discretetime markov chains. Timedependent perturbation theory literature 1 timeindependent nondegenerate perturbation theory general formulation firstorder theory secondorder theory 2 timeindependent degenerate perturbation theory general formulation example. An example in denumerable decision processes fisher, lloyd and ross, sheldon m. Perturbation theory and finite markov chains researchgate. Solutions are compared with those of variational iteration method and numerical solutions, and a good.
The problems of stability and the corresponding estimates were considered for new classes of processes in zeifman. Introduction the purpose of this paper is to describe the special problems that emerge when gaussian elimination is used to determinine the stead. Perturbation analysis for denumerable markov chains with application to queueing models. This paper is devoted to perturbation analysis of denumerable markov chains.
Moment bounds and ergodicity of switching diffusion systems involving twotimescale markov chains. Linear algebra and its applications journal homepage. Perturbation bounds perturbation analysis of markov chains residual matrix norm ergodicity coef. Bounds on convergence of entropy rate approximations in. Strong stability and perturbation bounds for discrete markov chains strong stability and perturbation bounds for discrete markov chains rabta, boualem. Apr 15, 2008 read strong stability and perturbation bounds for discrete markov chains, linear algebra and its applications on deepdyve, the largest online rental service for scholarly research with thousands of academic publications available at your fingertips. This thesis is concerned with studying the hitting time of an absorbing state on markov chain models that have a countable state space. Bounds are provided for the deviation between the stationary distribution of the perturbed and nominal chain, where the. Sensitivity of finite markov chains under perturbation. Bounds are provided for the deviation between the stationary distribution of the perturbed and nominal chain, where the bounds are given by the weighted. Perturbationiteration method for firstorder differential. Regular perturbation of vgeometrically ergodic markov chains.
The aim of this paper is to investigate the stability of markov chains with general state space. On the existence of quasistationary distributions in denumerable rtransient markov chains authors. For many models it is challenging to study the hitting time directly. Nunezqueijaperturbation analysis for denumerable markov chains with. Error bounds for augmented truncation approximations of markov. Strong stability and perturbation bounds for discrete markov. D and d are derived in terms of a drift condition, where. Perturbation methods for markovswitching dsge models. Qbd processes, which constitute a wide class of structured markov chains. Qbd processes, which constitute a wide class of structured markov. Measurevalued differentiation for stationary markov chains. Perturbation theory comprises mathematical methods for finding an approximate solution to a problem, by starting from the exact solution of a related, simpler problem.
We use cookies to distinguish you from other users and to provide you with a better experience on our websites. We provide a unified approach to pamc for finite and denumerable markov. New perturbation bounds for denumerable markov chains, linear. On the existence of quasistationary distributions in. In the first part, we introduce a condition number that measures the sensitivity of fixed points of a quantum channel to perturbations. We establish upper and lower bounds on this condition number in terms of subdominant eigenvalues of the transition map. New perturbation bounds for denumerable markov chains, linear algebra and its applications, 432, 16271649. Gaussian elimination, perturbation theory, and markov chains g. Numerical examples are given to illustrate the performance of the algorithm. Mar 19, 20 we investigate the stability of quantum markov processes with respect to perturbations of their transition maps. Let p be the transition matrix of a positive recurrent markov chain on the integers.
This paper develops a general perturbation methodology for constructing highorder approximations to the solutions of msdsge models. Summary in this paper, our interest is in the perturbation analysis of level. Reliability modelling and data analysis of vacuum circuit breaker subject to random shocks. Strong stability and perturbation bounds for discrete markov chains. Moment bounds and ergodicity of switching diffusion. Bounds are provided for the deviation between the stationary distribution of the perturbed and nominal chain, where the bounds are given by the. Perturbation analysis of finite markov chains has received much attention in the literature over recent years see in. Taylor series expansions for stationary markov chains. Strong stability and perturbation bounds for discrete. The results are based on an extension of the standard perturbation theory formulated by keller and liverani. Series expansions for finitestate markov chains semantic. Rateoptimal perturbation bounds for singular subspaces with. Finite continuous time markov chains theory of probability.
We show how to reduce the complex markovswitching problem to solving a system of quadratic polynomial equations. Series expansions for continuoustime markov processes. Stewart computer science department institute for advanced computer studies university of maryland college park, maryland jiguang sun computing center ofthe chinese academy of sciences beijing, china academic press, inc. We study general statespace markov chains that depend on a parameter, say, sufficient conditions are established for the stationary performance of such a markov chain to be differentiable with respect to specifically, we study the case of unbounded performance functions and thereby extend the result on weak differentiability of stationary distributions of markov chains to unbounded. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite markov chain. Journal of science, engineering and technology, waset world academy of science, engineering and technology ed pp. Perturbation bounds for markov chains with general state space. Robust stability of a linear multivariable system, in the sense of robustness under multiplicative transfer function perturbation, is necessarily preserved under sufficiently small perturbations in the model parameters i. Also, we obtain perturbation bounds with respect to different quantities. This paper compares and analyzes bounds found in the literature for finite and denumerable markov chains and introduces new bounds based. Bounds on convergence of entropy rate approximations in hidden markov processes by nicholas f. Additive perturbation bounds on the eigenvectors of a hermitian matrix. For finite irreducible markov chains many perturbation bounds for the.
Introduction to stochastic processes, prenticehall, new jersey. Australia received september 1992 revised november 1992 abstract. Perturbation theory is applicable if the problem at hand cannot be solved exactly, but can be formulated by. Perturbations of countable markov chains and processes. Probabilistic model checking of perturbed mdps with.
We investigate perturbation for continuoustime markov chains ctmcs on a countable state space. The previously developed new perturbationiteration algorithm has been applied to differential equation systems for the first time. Singular perturbation analysis for countable markov chains. Dec 19, 2017 the aim of this paper is to investigate the stability of markov chains with general state space. Two approaches to the construction of perturbation bounds. The algorithm is tested for a single equation, coupled two equations, and coupled three equations. This is useful for studying how sensitive the original systems eigenvectors and eigenvalues are to changes in the system.
Bounds are provided for the deviation between the stationary distribution of the perturbed and nominal chain, where the bounds are given by the weighted supremum norm. Mar 15, 2010 new perturbation bounds for denumerable markov chains new perturbation bounds for denumerable markov chains mouhoubi, zahir. Saratov state university abstract weshowthat,forreversiblecontinuoustimemarkovchains,theclosenessofthenonzero eigenvalues of the generator to zero provides complete information about the sensitivity. Siam journal on numerical analysis volume 25, issue 3. However, there are only few references available on perturbation analysis of markov chains with an in. Abstractthis paper is devoted to perturbation analysis of denumerable markov chains. Under suitable stability conditions, numerical approximations can be derived from the update formulas, and we show that the algorithms converge at a geometric. Perturbation methods for markovswitching models andrew foerstery juan rubioramirezz dan waggonerx tao zhaaugust 2, 2012 abstract markov switching models are a way to consider discrete changes in the economic environment, such as policy changes, and allow agents in the economy to form expectations over these changes. Moreover, we obtain perturbation bounds on the stationary distributions, which extends the results by liu 2012 for. If p is a transition matrix of a markov chain, andequation is derived by perturbing. To address the effects of uncertainty in probability estimates, in previous work we have developed a variety of techniques for perturbation analysis of discrete and continuoustime markov chains dtmcs and ctmcs. By the latter we mean that transition probabilities of a markov chain, with several ergodic classes, are perturbed such that rare transitions among the different ergodic classes of the unperturbed chain are allowed. Seneta school of mathematics and statistics, university of sydney, nsu. A critical feature of the technique is a middle step that breaks the problem into solvable and perturbation parts.
Lower bounds, which show that the individual perturbation bounds are rateoptimal, are also given. In this paper, our interest is in the perturbation analysis of level. Proceedings ieee conference on decision and control, 2. Apr 30, 2015 we investigate perturbation for continuoustime markov chains ctmcs on a countable state space. Xiaoyue li a,1, rui wang a,b, george yin c,2 a school of mathematics and statistics, northeast normal university, changchun, jilin, 024, china department of economics, university of kansas, lawrence, ks 66045, usa c department of. We use information technology and tools to increase productivity and facilitate new forms of scholarship.
I present a perturbative approach that allows one to uniformly bound the difference between the hitting time moment generating functions of two markov chains in a neighbourhood of the origin. Moment bounds and ergodicity of switching diffusion systems. Comparison of perturbation bounds for the stationary distribution of a markov chain. Perturbation analysis for denumerable markov chains with. In mathematics, an eigenvalue perturbation problem is that of finding the eigenvectors and eigenvalues of a system that is perturbed from one with known eigenvectors and eigenvalues. Bounds are provided for the deviation between the stationary distribution of the perturbed and nominal chain, where.
Harcourt bruce jovanovich, publishers boston san diego new york london sydney tokyo toronto. Markov chains, deviation matrix, linear pogramming, perturbation matrix analysis 1. Introduction a perturbation in a markov chain can be referred as a slight change in the entries of the corresponding transition stochastic matrix, resulting in structural changes in the underlying process, for example, sets. Sharp entrywise perturbation bounds for markov chains. Markovswitching dsge msdsge modeling has become a growing body of literature on economic and policy issues related to structural shifts. Finally, in section 4, we explicitly obtain the quasistationary distributions of a leftcontinuous random walk to demonstrate the usefulness of our results. Perturbation bounds for markov chains with general state. For many markov chains of practical interest, the invariant distribution is extremely sensitive to perturbations of some entries of the. Semigroups of conditioned shifts and approximation of markov processes kurtz, thomas g. Perturbation theory for markov reward processes with applications to queueing systems volume 20 issue 1 nico m.
Sensitivity of finite markov chains under perturbation e. New perturbation bounds for denumerable markov chains linear algebra and its applications, vol. Denumerable markov processes with bounded generators. Regular perturbation of vgeometrically ergodic markov chains deborah ferre, loic herve, james ledoux. Other applications of our results to phasetype queues will be. In the present paper we propose an approach to the construction of general estimates for the perturbation bounds of markov chains in terms of special weighted norms related to total variation. New perturbation bounds for denumerable markov chains core.
The iteration algorithm for systems is developed first. This section may be regarded as a complement of daleys work 3. Perturbation bounds for structured robust stability. Rateoptimal perturbation bounds for singular subspaces. We consider both regular and singular perturbations. Pdf series expansions for continuoustime markov chains. Perturbation results for nearly uncoupled markov chains with applications to iterative methods. We present new conditions for the strong stability of markov chains after a small perturbation of their transition kernels. Introduction a perturbation in a markov chain can be referred as a slight change in the entries of the corresponding transition stochastic matrix, resulting in structural changes in the underlying process, for. This article provides series expansions of the stationary distribution of a finite markov chain. An approach is described to the construction of perturbation estimates for the main five classes of such chains associated with queuing models. Our bounds are sharp, we do not impose any structural assumptions on the transition matrix or on the perturbation, and computing the bounds has the same complexity as computing the invariant distribution or computing other bounds in the literature.
Twodimensional harmonic oscilator 3 timedependent perturbation theory 4 literature igor luka cevi c. New perturbation bounds for denumerable markov chains citeseerx. Bounds are provided for the deviation between the stationary distribution of the perturbed and nominal. Haverford college 2005 dissertation submitted in partial satisfaction of. We study the parametric perturbation of markov chains with denumerable. For finite irreducible markov chains many perturbation bounds for the stationary vector are available. We study the parametric perturbation of markov chains with denumerable state spaces. New perturbation bounds for denumerable markov chains. We investigate the stability of quantum markov processes with respect to perturbations of their transition maps.
1526 1259 1407 1417 1405 1521 1257 1088 305 285 887 28 688 1074 1121 638 468 246 1402 117 1311 307 953 603 122 698 333 1146 762 343 289 150 346 169 1107 423