Interesting examples of discrete stochastic games



SGs are a generalization of MDPs to multiple agents. Like this previous question on MDPs, are there any interesting examples of zero-sum, discrete SGs—preferably with small state and action spaces? I'm hoping to use such examples as benchmarks, but couldn't find much in the literature. One example I can think of is a pursuit-evasion game on a graph.


Posted 2019-12-23T20:12:29.303

Reputation: 287



Some of the domains in the International Probabilistic Planning Competition, such as the Wildlife Preserve benchmark, fit quite well the constraints you have given. Note that the problems are modeled with a high-level declarative language, RDDL. This means that you can define problems as big or as small as your heart desires with relative ease, since you can parametrize state description in terms of functions describing properties of an arbitrary number of objects.

There's also a quite useful project that allows to instance OpenAI gym environments from the declarative description of the environment, states and actions.

Miquel Ramirez

Posted 2019-12-23T20:12:29.303

Reputation: 176

Thanks for your answer. I looked at the two papers listed under Wildlife Preserve in the page you linked to, but it seems neither is a multi-state game? – user76284 – 2020-02-20T04:38:47.467

From the definition of the instance (see this gist, file wildlife-preserve_inst_mdp__01.rddl), and the definition of the state variables (see section state-fluents in the file wildlife-preserve_01_mdp.rddl posted in the gist above) the smallest instance in the suite has 2^4 x 4 x 4 x 2 = 512 states.

I recommend you too to play around with the rddlgym tool I linked in my answer, will make easier to come to grips with the syntax of RDDL.

– Miquel Ramirez – 2020-02-21T06:16:45.243