TY - JOUR
T1 - Accounting for parametric uncertainty in Markov decision processes
AU - Schapaugh, Adam W.
AU - Tyre, Andrew J.
PY - 2013/4/10
Y1 - 2013/4/10
N2 - Markov decision processes have become the standard tool for modeling sequential decision-making problems in conservation. In many real-world applications, however, it is practically infeasible to accurately parameterize the state transition function. In this study, we introduce a new way of dealing with ambiguity in the state transition function. In contrast to existing methods, we explore the effects of uncertainty at the level of the policy, rather than at the level of decisions within states. We use information-gap decision theory to ask the question of how much uncertainty in the state transition function can be tolerated while still delivering a specified expected value given by the objective function. Accordingly, the goal of the optimization problem is no longer to maximize expected value, but to maximize local robustness to uncertainty (while still meeting the desired level of performance). We analyze a simple land acquisition problem, using info-gap decision theory to propagate uncertainties and rank alternative policies. Rather than requiring information about the extent of parameter uncertainty at the outset, info-gap addresses the question of how much uncertainty is permissible in the state transition function before the optimal policy would change.
AB - Markov decision processes have become the standard tool for modeling sequential decision-making problems in conservation. In many real-world applications, however, it is practically infeasible to accurately parameterize the state transition function. In this study, we introduce a new way of dealing with ambiguity in the state transition function. In contrast to existing methods, we explore the effects of uncertainty at the level of the policy, rather than at the level of decisions within states. We use information-gap decision theory to ask the question of how much uncertainty in the state transition function can be tolerated while still delivering a specified expected value given by the objective function. Accordingly, the goal of the optimization problem is no longer to maximize expected value, but to maximize local robustness to uncertainty (while still meeting the desired level of performance). We analyze a simple land acquisition problem, using info-gap decision theory to propagate uncertainties and rank alternative policies. Rather than requiring information about the extent of parameter uncertainty at the outset, info-gap addresses the question of how much uncertainty is permissible in the state transition function before the optimal policy would change.
KW - Information-gap
KW - Markov decision process
KW - Reserve selection
KW - Stochastic dynamic programming
KW - Uncertainty
UR - http://www.scopus.com/inward/record.url?scp=84873957215&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84873957215&partnerID=8YFLogxK
U2 - 10.1016/j.ecolmodel.2013.01.003
DO - 10.1016/j.ecolmodel.2013.01.003
M3 - Article
AN - SCOPUS:84873957215
VL - 254
SP - 15
EP - 21
JO - Ecological Modelling
JF - Ecological Modelling
SN - 0304-3800
ER -