Asymptotically optimal strategies for adaptive zero-sum discounted markov games

We consider a class of discrete-time two person zero-sum Markov games with Borel state and action spaces, and possibly unbounded payoffs. The ga me evolves according to the recursive equation x_n+1 = F(x_n,a _n,b_n,_n), n = 0, 1,..., where the disturbance process {_n} is formed by independent and identically distributed R^k-valued random vectors, which are observable but whose common density p is unknown to both players. Under certain continuity and compactness conditions, we combine a nonstationary iteration procedure and suitable den sity estimation methods to construct asymptotically discounted optimal strategies for both players.