Empirical approximation of Nash equilibria in finite Markov games with discounted payoffs

Alan D. Robles-Aguilar, David González-Sánchez, J. Adolfo Minjárez-Sosa*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


This paper deals with finite nonzero-sum Markov games under a discounted optimality criterion and infinite horizon. The state process evolves according to a stochastic difference equation and depends on players' actions as well as a random disturbance whose distribution is unknown to the players. The actions, the states, and the values of the disturbance are observed by the players, then they use the empirical distribution of the disturbances to estimate the true distribution and make choices based on the available information. In this context, we propose an almost surely convergent procedure—possibly after passing to a subsequence—to approximate Nash equilibria of the Markov game with the true distribution of the random disturbance.

Original languageEnglish
Pages (from-to)722-734
Number of pages13
JournalAsian Journal of Control
Issue number2
StatePublished - Mar 2023

Bibliographical note

Publisher Copyright:
© 2022 Chinese Automatic Control Society and John Wiley & Sons Australia, Ltd.


  • Markov games
  • Nash equilibrium
  • discounted criterion
  • empirical estimation


Dive into the research topics of 'Empirical approximation of Nash equilibria in finite Markov games with discounted payoffs'. Together they form a unique fingerprint.

Cite this