Approximate dynamic programming. Solving the curses of by Warren B. Powell

By Warren B. Powell

Praise for the First Edition

"Finally, a publication dedicated to dynamic programming and written utilizing the language of operations examine (OR)! this pretty publication fills a spot within the libraries of OR experts and practitioners."
Computing Reviews

This re-creation showcases a spotlight on modeling and computation for complicated periods of approximate dynamic programming problems

Understanding approximate dynamic programming (ADP) is key as a way to strengthen sensible and top quality recommendations to advanced commercial difficulties, quite whilst these difficulties contain making judgements within the presence of uncertainty. Approximate Dynamic Programming, moment variation uniquely integrates 4 precise disciplines—Markov choice techniques, mathematical programming, simulation, and statistics—to exhibit the right way to effectively procedure, version, and resolve quite a lot of real-life difficulties utilizing ADP.

The booklet maintains to bridge the distance among desktop technology, simulation, and operations learn and now adopts the notation and vocabulary of reinforcement studying in addition to stochastic seek and simulation optimization. the writer outlines the basic algorithms that function a place to begin within the layout of sensible options for genuine difficulties. the 3 curses of dimensionality that impression complicated difficulties are brought and distinctive insurance of implementation demanding situations is supplied. The Second Edition additionally features:

  • A new bankruptcy describing 4 basic periods of regulations for operating with different stochastic optimization difficulties: myopic rules, look-ahead regulations, coverage functionality approximations, and guidelines in keeping with worth functionality approximations

  • A new bankruptcy on coverage seek that brings jointly stochastic seek and simulation optimization ideas and introduces a brand new classification of optimum studying strategies

  • Updated assurance of the exploration exploitation challenge in ADP, now together with a lately built strategy for doing energetic studying within the presence of a actual country, utilizing the concept that of the information gradient

  • A new series of chapters describing statistical equipment for approximating worth services, estimating the worth of a hard and fast coverage, and cost functionality approximation whereas trying to find optimum policies

The provided assurance of ADP emphasizes types and algorithms, targeting comparable purposes and computation whereas additionally discussing the theoretical part of the subject that explores proofs of convergence and expense of convergence. A comparable site gains an ongoing dialogue of the evolving fields of approximation dynamic programming and reinforcement studying, besides extra readings, software program, and datasets.

Requiring just a simple knowing of information and chance, Approximate Dynamic Programming, moment version is a superb booklet for business engineering and operations study classes on the upper-undergraduate and graduate degrees. It additionally serves as a worthy reference for researchers and pros who make the most of dynamic programming, stochastic programming, and keep watch over concept to resolve difficulties of their daily work.

Show description

Read or Download Approximate dynamic programming. Solving the curses of dimensionality PDF

Similar probability & statistics books

Theories in Probability: An Examination of Logical and Qualitative Foundations (Advanced Series on Mathematical Psychology)

General chance thought has been an greatly winning contribution to trendy technological know-how. in spite of the fact that, from many views it's too slender as a basic idea of uncertainty, rather for matters regarding subjective uncertainty. This first-of-its-kind ebook is based mostly on qualitative ways to probabilistic-like uncertainty, and comprises qualitative theories for a standard conception in addition to numerous of its generalizations.

An Introduction to Statistical Inference and Its Applications

Emphasizing strategies instead of recipes, An creation to Statistical Inference and Its purposes with R presents a transparent exposition of the equipment of statistical inference for college kids who're pleased with mathematical notation. quite a few examples, case reviews, and routines are incorporated. R is used to simplify computation, create figures, and draw pseudorandom samples—not to accomplish whole analyses.

Probability on Discrete Structures

Such a lot chance difficulties contain random variables listed by means of area and/or time. those difficulties commonly have a model within which house and/or time are taken to be discrete. This quantity bargains with parts within which the discrete model is extra typical than the continual one, maybe even the one one than should be formulated with no complex buildings and equipment.

Introduction to Bayesian Estimation and Copula Models of Dependence

Provides an creation to Bayesian facts, offers an emphasis on Bayesian equipment (prior and posterior), Bayes estimation, prediction, MCMC,Bayesian regression, and Bayesian research of statistical modelsof dependence, and contours a spotlight on copulas for possibility administration advent to Bayesian Estimation and Copula versions of Dependence emphasizes the functions of Bayesian research to copula modeling and equips readers with the instruments had to enforce the methods of Bayesian estimation in copula versions of dependence.

Additional info for Approximate dynamic programming. Solving the curses of dimensionality

Example text

Similarly an oil company purchasing oil will be given quantity discounts (or it may face the fixed cost of purchasing a tanker-load of oil). Retail outlets get a discount if they purchase a truckload of an item. All of these are instances of acquiring assets with a concave (or, more generally, nonconvex) cost function, which means there is an incentive for purchasing larger quantities. Asset acquisition with lagged information processes. We can purchase commodity futures that allow us to purchase a product in the future at a lower cost.

Almost any dynamic program with discrete states and actions can be modeled as a decision tree. The problem is that decision trees are not practical when there are a large number of states and actions. 2 A Stochastic Shortest Path Problem We are often interested in shortest path problems where there is uncertainty in the cost of traversing a link. For our transportation example it is natural to view the travel time on a link as random, reflecting the variability in traffic conditions on each link.

For our simple model we assume that pˆ t is independent of prior prices (a more typical model would assume that the change in price is independent of prior history). With this assumption, our system has two states: St = 1 we are holding the asset, 0 we have sold the asset. Assume that we measure the state immediately after the price pˆt has been revealed but before we have made a decision. If we have sold the asset, then there is nothing we can do. We want to maximize the price we receive when we sell our asset.

Download PDF sample

Rated 4.04 of 5 – based on 4 votes