This paper develops a theory of regular Markov perfect equilibrium in discrete-time, infinite-horizon stochastic games with a finite number of states and actions.
These equilibria are essential, strongly stable and admit purification. This is shown by introducing a regularity notion and proving that it is a generic properly of the equilibria.