Markov games with unknown random state-actions-dependent discount factors: Empirical estimation

David González-Sánchez; Fernando Luque-Vásquez; Jesus Adolfo Minjárez-Sosa

doi:10.1002/asjc.2159

Markov games with unknown random state-actions-dependent discount factors: Empirical estimation

David González-Sánchez, Fernando Luque-Vásquez, Jesus Adolfo Minjárez-Sosa^*

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

The work deals with a class of discrete-time zero-sum Markov games under a discounted optimality criterion with random state-action-dependent discount factors of the form (Formula presented.), where x_n,a_n,b_n, and ξ_n+1 are the state, the actions of players, and a random disturbance at time n, respectively, taking values in Borel spaces. The one-stage payoff is assumed to be possibly unbounded. In addition, the process {ξ_n} is formed by observable, independent, and identically distributed random variables with common distribution θ, which is unknown to players. By using the empirical distribution to estimate θ, we introduce a procedure to approximate the value V^∗ of the game; such a procedure yields construction schemes of stationary optimal strategies and asymptotically optimal Markov strategies.

Original language	English
Pages (from-to)	166-177
Number of pages	12
Journal	Asian Journal of Control
Volume	23
Issue number	1
DOIs	https://doi.org/10.1002/asjc.2159
State	Published - 1 Jan 2019

Bibliographical note

Publisher Copyright:
© 2019 Chinese Automatic Control Society and John Wiley & Sons Australia, Ltd

Keywords

discounted optimality
empirical estimation
markov games
non-constant discount factors

Access to Document

10.1002/asjc.2159

Cite this

@article{88548c86c436469694734ee16902840c,

title = "Markov games with unknown random state-actions-dependent discount factors: Empirical estimation",

abstract = "The work deals with a class of discrete-time zero-sum Markov games under a discounted optimality criterion with random state-action-dependent discount factors of the form (Formula presented.), where xn,an,bn, and ξn+1 are the state, the actions of players, and a random disturbance at time n, respectively, taking values in Borel spaces. The one-stage payoff is assumed to be possibly unbounded. In addition, the process {ξn} is formed by observable, independent, and identically distributed random variables with common distribution θ, which is unknown to players. By using the empirical distribution to estimate θ, we introduce a procedure to approximate the value V∗ of the game; such a procedure yields construction schemes of stationary optimal strategies and asymptotically optimal Markov strategies.",

keywords = "discounted optimality, empirical estimation, markov games, non-constant discount factors",

author = "David Gonz{\'a}lez-S{\'a}nchez and Fernando Luque-V{\'a}squez and Minj{\'a}rez-Sosa, {Jesus Adolfo}",

note = "Publisher Copyright: {\textcopyright} 2019 Chinese Automatic Control Society and John Wiley & Sons Australia, Ltd",

year = "2019",

month = jan,

day = "1",

doi = "10.1002/asjc.2159",

language = "Ingl{\'e}s",

volume = "23",

pages = "166--177",

journal = "Asian Journal of Control",

issn = "1561-8625",

publisher = "Wiley-Blackwell",

number = "1",

}

TY - JOUR

T1 - Markov games with unknown random state-actions-dependent discount factors

T2 - Empirical estimation

AU - González-Sánchez, David

AU - Luque-Vásquez, Fernando

AU - Minjárez-Sosa, Jesus Adolfo

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The work deals with a class of discrete-time zero-sum Markov games under a discounted optimality criterion with random state-action-dependent discount factors of the form (Formula presented.), where xn,an,bn, and ξn+1 are the state, the actions of players, and a random disturbance at time n, respectively, taking values in Borel spaces. The one-stage payoff is assumed to be possibly unbounded. In addition, the process {ξn} is formed by observable, independent, and identically distributed random variables with common distribution θ, which is unknown to players. By using the empirical distribution to estimate θ, we introduce a procedure to approximate the value V∗ of the game; such a procedure yields construction schemes of stationary optimal strategies and asymptotically optimal Markov strategies.

AB - The work deals with a class of discrete-time zero-sum Markov games under a discounted optimality criterion with random state-action-dependent discount factors of the form (Formula presented.), where xn,an,bn, and ξn+1 are the state, the actions of players, and a random disturbance at time n, respectively, taking values in Borel spaces. The one-stage payoff is assumed to be possibly unbounded. In addition, the process {ξn} is formed by observable, independent, and identically distributed random variables with common distribution θ, which is unknown to players. By using the empirical distribution to estimate θ, we introduce a procedure to approximate the value V∗ of the game; such a procedure yields construction schemes of stationary optimal strategies and asymptotically optimal Markov strategies.

KW - discounted optimality

KW - empirical estimation

KW - markov games

KW - non-constant discount factors

UR - http://www.scopus.com/inward/record.url?scp=85070753459&partnerID=8YFLogxK

U2 - 10.1002/asjc.2159

DO - 10.1002/asjc.2159

M3 - Artículo

AN - SCOPUS:85070753459

SN - 1561-8625

VL - 23

SP - 166

EP - 177

JO - Asian Journal of Control

JF - Asian Journal of Control

IS - 1

ER -

Markov games with unknown random state-actions-dependent discount factors: Empirical estimation

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this