How are the Bellman optimality equations and minimax related?


Is the philosophy between Bellman equations and minimax the same?

Both the algorithms look at the full horizon and take into account potential gains (Bellman) and potential losses (minimax).

However, do the two differ besides the obvious on the fact that Bellman equations use discounted potential rewards, while minimax deals with potential losses without the discount? Are these enough to say they are similar in philosophy or is are they dissimilar? If so, then in what sense?


Posted 2020-04-22T15:17:19.507

Reputation: 151



They have similar philosophies, in the sense that minimax and algorithms based on the Bellman optimality equations are used to solve optimization problems, but they are also different because they solve different problems.

Minimax (at least, the minimax version that I am aware of) is typically used to solve two-player games (e.g. chess, tic-tac-toe, etc.), while Bellman optimality equations (I assume you are referring to the Bellman equations that algorithms such as policy iteration are based on) do not assume the existence of two players (unless you consider the environment a player).


Posted 2020-04-22T15:17:19.507

Reputation: 19 783

Allright so the assumption of "two-players" sets them apart? – gfdsal – 2020-04-22T16:16:34.720

@gfdsal I guess so (at least in most cases), but minimax may also be used in other context and I don't exclude the existence of "Bellman equations" for two-player games (I am just not aware of them right now). – nbro – 2020-04-22T16:18:38.297