Continuous-time Markov chains and applications : a two-time-scale approach
QA274.7 .Y56 2013
- Unknown QA274.7 .Y56 2013
- Zhang, Qing, 1959-
- Includes bibliographical references and index.
- Prologue and preliminaries. Introduction and overview; Introduction; A brief survey: Markov chains: Singular perturbations; Outline of the book ; Mathematical preliminaries; Introduction; Martingales; Markov chains; piecewise-deterministic processes: Construction of Markov chains; Irreducibility and Quasi-stationary distributions; Gaussian processes and diffusions; Switching diffusions; Notes ; Markovian models; Introduction; Birth and death processes; Finite-state space models: Queues with finite capacity: System reliability: Competing risk theory: Two-time-scale Cox processes: Random evolutions: Seasonal variation models; Stochastic optimization problems: Simulated annealing: Continuous-time stochastic approximation: Systems with Markovian disturbances; Linear systems with jump Markov disturbance: Linear Quadratic control problems: Singularly perturbed LQ systems with wide-band noise: Large-scale systems : decomposition and aggregation; Time-scale separation; Notes
- Two-time-scale Markov chains. Asymptotic expansions of solutions for forward equations; Introduction; Irreducible case: Asymptotic expansions: Outer expansion: Initial-layer correction: Exponential decay of ...: Asymptotic validation: Examples: Two-time-scale expansion; Markov chains with multiple weakly irreducible classes: Asymptotic expansions: Analysis of remainder: Computational procedure : user's guide: Summary of results: An example; Inclusion of absorbing states; Inclusion of transient states; Remarks on countable-state-space cases: Countable-state spaces : Part I: Countable-state spaces : Part II: A remarks on finite-dimensional approximation; Remarks on singularly perturbed diffusions; Notes ; Occupation measures : Asymptotic properties and ramification; Introduction; The irreducible case: Occupation measure: Conditions and preliminary results: Exponential bounds: Asymptotic normality: Extensions; Markov chains with weak and strong interactions: Aggregation of Markov chains: Exponential bounds: Asymptotic distributions; Measurable generators; Remarks on inclusion of transient and absorbing states: Inclusion of transient states: Inclusion of absorbing states; Remarks on a stability problem: Notes ; Asymptotic expansions of solutions for backward equations; Introduction; Problem formulation: A preliminary lemma: Formulation; Construction of asymptotic expansions: Leading term ... and zero-order terminal-layer term ...: Higher-order terms; Error estimates; Asymptotic expansions including transient states; Remarks: Related problems; Notes
- Applications : MDPs, near-optimal controls, numerical methods, and LQG with switching. Markov decision problems; Introduction; Problem formulation; Limit problem; Asymptotic optimality; Convergence rate and error bound; Long-run average cost; Computational procedures; Notes ; Stochastic control of dynamical systems; Introduction; Problem formulation; Properties of the value functions; Asymptotic optimal controls; Convergence rate; Weak convergence approach: Problem setup: Relaxed control formulation: Near optimality; Notes ; Numerical methods for control and optimization; Introduction; Numerical methods for optimal control; Optimization under threshold policy: Stochastic optimization formulation: Convergence: Examples: Error bounds; Notes ; Hybrid LQG problems; Introduction; Problem formulation; Optimal controls; Two-time-seale approximation : recurrent states: Limit riccati equations: Nearly optimal controls; Two-time-scale approximation : inclusion of transient states; Two-time-scale approximation : inclusion of absorbing states; A numerical example; Remarks on indefinite control weights; Notes ; Background materials; Properties of generators; Weak convergence; Relaxed control; Viscosity solutions of HJB equations; Value functions and optimal controls; Miscellany ; Bibliography ; Index.
- Publisher's Summary
- This book gives a systematic treatment of singularly perturbed systems that naturally arise in control and optimization, queueing networks, manufacturing systems, and financial engineering. It presents results on asymptotic expansions of solutions of Komogorov forward and backward equations, properties of functional occupation measures, exponential upper bounds, and functional limit results for Markov chains with weak and strong interactions. To bridge the gap between theory and applications, a large portion of the book is devoted to applications in controlled dynamic systems, production planning, and numerical methods for controlled Markovian systems with large-scale and complex structures in the real-world problems. This second edition has been updated throughout and includes two new chapters on asymptotic expansions of solutions for backward equations and hybrid LQG problems. The chapters on analytic and probabilistic properties of two-time-scale Markov chains have been almost completely rewritten and the notation has been streamlined and simplified. This book is written for applied mathematicians, engineers, operations researchers, and applied scientists. Selected material from the book can also be used for a one semester advanced graduate-level course in applied probability and stochastic processes.
(source: Nielsen Book Data)
- Publication date
- G. George Yin, Qing Zhang.
- Stochastic modelling and applied probability, 0172-4568 ; 37
- Subtitle of 1st ed.: a singular perturbation approach.