LEADER 05495nam 2200661Ia 450 001 9910465390503321 005 20200520144314.0 010 $a1-299-28108-7 010 $a1-84816-794-6 035 $a(CKB)2560000000099446 035 $a(EBL)1143323 035 $a(OCoLC)830162389 035 $a(SSID)ssj0000913760 035 $a(PQKBManifestationID)11551371 035 $a(PQKBTitleCode)TC0000913760 035 $a(PQKBWorkID)10860720 035 $a(PQKB)10499631 035 $a(MiAaPQ)EBC1143323 035 $a(WSP)00002939 035 $a(Au-PeEL)EBL1143323 035 $a(CaPaEBR)ebr10674365 035 $a(CaONFJC)MIL459358 035 $a(EXLCZ)992560000000099446 100 $a20130226d2013 uy 0 101 0 $aeng 135 $aur|n|---||||| 181 $ctxt 182 $cc 183 $acr 200 10$aExamples in Markov decision processes$b[electronic resource] /$fA. B. Piunovskiy 210 $aLondon $cImperial College Press ;$aSingapore ;$aHackensack, NJ $cDistributed by World Scientific Pub.$dc2013 215 $a1 online resource (308 p.) 225 1 $aImperial College Press optimization series ;$vv. 2 300 $aDescription based upon print version of record. 311 $a1-84816-793-8 320 $aIncludes bibliographical references (p. 285-290) and index. 327 $aContents; Preface; 1. Finite-Horizon Models; 1.1 Preliminaries; 1.2 Model Description; 1.3 Dynamic Programming Approach; 1.4 Examples; 1.4.1 Non-transitivity of the correlation; 1.4.2 The more frequently used control is not better; 1.4.3 Voting; 1.4.4 The secretary problem; 1.4.5 Constrained optimization; 1.4.6 Equivalent Markov selectors in non-atomic MDPs; 1.4.7 Strongly equivalent Markov selectors in nonatomic MDPs; 1.4.8 Stock exchange; 1.4.9 Markov or non-Markov strategy? Randomized or not? When is the Bellman principle violated?; 1.4.10 Uniformly optimal, but not optimal strategy 327 $a1.4.11 Martingales and the Bellman principle1.4.12 Conventions on expectation and infinities; 1.4.13 Nowhere-differentiable function vt(x); discontinuous function vt(x); 1.4.14 The non-measurable Bellman function; 1.4.15 No one strategy is uniformly -optimal; 1.4.16 Semi-continuous model; 2. Homogeneous Infinite-Horizon Models: Expected Total Loss; 2.1 Homogeneous Non-discounted Model; 2.2 Examples; 2.2.1 Mixed Strategies; 2.2.2 Multiple solutions to the optimality equation; 2.2.3 Finite model: multiple solutions to the optimality equation; conserving but not equalizing strategy 327 $a2.2.4 The single conserving strategy is not equalizing and not optimal2.2.5 When strategy iteration is not successful; 2.2.6 When value iteration is not successful; 2.2.7 When value iteration is not successful: positive model I; 2.2.8 When value iteration is not successful: positive model II; 2.2.9 Value iteration and stability in optimal stopping problems; 2.2.10 A non-equalizing strategy is uniformly optimal; 2.2.11 A stationary uniformly -optimal selector does not exist (positive model); 2.2.12 A stationary uniformly -optimal selector does not exist (negative model) 327 $a2.2.13 Finite-action negative model where a stationary uniformly -optimal selector does not exist2.2.14 Nearly uniformly optimal selectors in negative models; 2.2.15 Semi-continuous models and the blackmailer's dilemma; 2.2.16 Not a semi-continuous model; 2.2.17 The Bellman function is non-measurable and no one strategy is uniformly -optimal; 2.2.18 A randomized strategy is better than any selector (finite action space); 2.2.19 The fluid approximation does not work; 2.2.20 The fluid approximation: refined model; 2.2.21 Occupation measures: phantom solutions 327 $a2.2.22 Occupation measures in transient models2.2.23 Occupation measures and duality; 2.2.24 Occupation measures: compactness; 2.2.25 The bold strategy in gambling is not optimal (house limit); 2.2.26 The bold strategy in gambling is not optimal (inflation); 2.2.27 Search strategy for a moving target; 2.2.28 The three-way duel ("Truel"); 3. Homogeneous Infinite-Horizon Models: Discounted Loss; 3.1 Preliminaries; 3.2 Examples; 3.2.1 Phantom solutions of the optimality equation; 3.2.2 When value iteration is not successful: positive model 327 $a3.2.3 A non-optimal strategy for which v x solves the optimality equation 330 $aThis invaluable book provides approximately eighty examples illustrating the theory of controlled discrete-time Markov processes. Except for applications of the theory to real-life problems like stock exchange, queues, gambling, optimal search etc, the main attention is paid to counter-intuitive, unexpected properties of optimization problems. Such examples illustrate the importance of conditions imposed in the theorems on Markov Decision Processes. Many of the examples are based upon examples published earlier in journal articles or textbooks while several other examples are new. The aim was 410 0$aImperial College Press optimization series ;$vv. 2. 606 $aMarkov processes 606 $aStatistical decision 608 $aElectronic books. 615 0$aMarkov processes. 615 0$aStatistical decision. 676 $a519.233 700 $aPiunovskiy$b A. B$01036537 801 0$bMiAaPQ 801 1$bMiAaPQ 801 2$bMiAaPQ 906 $aBOOK 912 $a9910465390503321 996 $aExamples in Markov decision processes$92456942 997 $aUNINA