Continuoustime Markov chains and applications : a twotimescale approach
 Author/Creator
 Yin, George, 1954
 Language
 English.
 Edition
 2nd ed., [rev. and expanded]
 Imprint
 New York : Springer, c2013.
 Physical description
 xxi, 427 p. : ill. ; 25 cm.
 Series
 Stochastic modelling and applied probability 37.
Access
Contributors
 Contributor
 Zhang, Qing, 1959
Contents/Summary
 Bibliography
 Includes bibliographical references and index.
 Contents

 Prologue and preliminaries. Introduction and overview; Introduction; A brief survey: Markov chains: Singular perturbations; Outline of the book ; Mathematical preliminaries; Introduction; Martingales; Markov chains; piecewisedeterministic processes: Construction of Markov chains; Irreducibility and Quasistationary distributions; Gaussian processes and diffusions; Switching diffusions; Notes ; Markovian models; Introduction; Birth and death processes; Finitestate space models: Queues with finite capacity: System reliability: Competing risk theory: Twotimescale Cox processes: Random evolutions: Seasonal variation models; Stochastic optimization problems: Simulated annealing: Continuoustime stochastic approximation: Systems with Markovian disturbances; Linear systems with jump Markov disturbance: Linear Quadratic control problems: Singularly perturbed LQ systems with wideband noise: Largescale systems : decomposition and aggregation; Timescale separation; Notes
 Twotimescale Markov chains. Asymptotic expansions of solutions for forward equations; Introduction; Irreducible case: Asymptotic expansions: Outer expansion: Initiallayer correction: Exponential decay of ...: Asymptotic validation: Examples: Twotimescale expansion; Markov chains with multiple weakly irreducible classes: Asymptotic expansions: Analysis of remainder: Computational procedure : user's guide: Summary of results: An example; Inclusion of absorbing states; Inclusion of transient states; Remarks on countablestatespace cases: Countablestate spaces : Part I: Countablestate spaces : Part II: A remarks on finitedimensional approximation; Remarks on singularly perturbed diffusions; Notes ; Occupation measures : Asymptotic properties and ramification; Introduction; The irreducible case: Occupation measure: Conditions and preliminary results: Exponential bounds: Asymptotic normality: Extensions; Markov chains with weak and strong interactions: Aggregation of Markov chains: Exponential bounds: Asymptotic distributions; Measurable generators; Remarks on inclusion of transient and absorbing states: Inclusion of transient states: Inclusion of absorbing states; Remarks on a stability problem: Notes ; Asymptotic expansions of solutions for backward equations; Introduction; Problem formulation: A preliminary lemma: Formulation; Construction of asymptotic expansions: Leading term ... and zeroorder terminallayer term ...: Higherorder terms; Error estimates; Asymptotic expansions including transient states; Remarks: Related problems; Notes
 Applications : MDPs, nearoptimal controls, numerical methods, and LQG with switching. Markov decision problems; Introduction; Problem formulation; Limit problem; Asymptotic optimality; Convergence rate and error bound; Longrun average cost; Computational procedures; Notes ; Stochastic control of dynamical systems; Introduction; Problem formulation; Properties of the value functions; Asymptotic optimal controls; Convergence rate; Weak convergence approach: Problem setup: Relaxed control formulation: Near optimality; Notes ; Numerical methods for control and optimization; Introduction; Numerical methods for optimal control; Optimization under threshold policy: Stochastic optimization formulation: Convergence: Examples: Error bounds; Notes ; Hybrid LQG problems; Introduction; Problem formulation; Optimal controls; Twotimeseale approximation : recurrent states: Limit riccati equations: Nearly optimal controls; Twotimescale approximation : inclusion of transient states; Twotimescale approximation : inclusion of absorbing states; A numerical example; Remarks on indefinite control weights; Notes ; Background materials; Properties of generators; Weak convergence; Relaxed control; Viscosity solutions of HJB equations; Value functions and optimal controls; Miscellany ; Bibliography ; Index.
Subjects
Bibliographic information
 Publication date
 2013
 Responsibility
 G. George Yin, Qing Zhang.
 Series
 Stochastic modelling and applied probability, 01724568 ; 37
 Note
 Subtitle of 1st ed.: a singular perturbation approach.
 ISBN
 9781461443452
 1461443458