How do I represent a multi-dimensional state using a neural network?


I have a set of 15 unique playing cards from a deck of 52 playing cards. A given state is represented by the respective card values in the set of 15 cards, where the card value is a prime number associated with that card. For example, AH is represented by 3.

How should I represent a single state for the NN? Should it be a list of the 15 prime numbers representing the list of cards? I was hoping that I could represent a single state as the sum of each of all 15 prime numbers and then throw that sum through a sigmoid function. My concern, however, is that the NN will lose information if I reduce the dimension of the state to a single attribute (even if that attribute is unique to that state - the sum of n prime numbers is unique compared to the sum of any other n prime numbers).

How important is the dimensionality of each state for Deep Q Learning? I'd really appreciate even some general direction.

Harry Stuart

Posted 2019-05-16T06:16:22.003

Reputation: 151

Are there always 15 cards under consideration in your state? For example, do the cards get "played" or split between players to make hands or sets which might vary in size from 15 etc? I can see that as a way that you lose information, unless you also track $n$ for each group of cards being represented. – Neil Slater – 2019-05-16T09:53:38.300

The size would vary, but I can keep track of $n$. – Harry Stuart – 2019-05-16T09:55:37.000

@NeilSlater Since the size of $n$ is decreasing by 1 (a card is discarded) every round, would it be better to have $n$ neural networks for each hand length or just input an empty card (say a value of 0) to the remaining input nodes? – Harry Stuart – 2019-05-16T10:41:15.533

I don't want to answer in the comments. However, the clarification that n could vary does mean that just the pure sum of numbers is not enough to guarantee uniqueness because e.g. $2 + 3 = 5$ - so you cannot represent the whole state of play in the way that you hope – Neil Slater – 2019-05-16T12:41:21.850

So, assuming I used some sort of binary or one-hot encoding, what would you suggest I do in regards to my previous question? Thanks for the help btw! I am quite new to NN – Harry Stuart – 2019-05-16T12:42:53.393



In my humble opinion, it seems like it is important to have them separated, if having a certain card can influence the result in some way that is not its prime value, instead of not only using the sum. But it depends on the game and its rules. For example:

If having 5 cards of hearts in the set of 15 cards makes you win the game, then if you only represent the state as the sum of the value of the cards, the DQN will never learn that it was not the whole 15 cards that caused you to win, but only a specific part of the set.

Miguel Saraiva

Posted 2019-05-16T06:16:22.003

Reputation: 619

So that implies that my input layer would have 15 nodes (each is fed a number that represents a card in the list). Do I feed each node the prime number? Do I feed each node the "sigmoided" prime number? – Harry Stuart – 2019-05-16T09:30:19.323

@HarryStuart I'm fairly sure that In the input you can put the prime number if you want to represent the cards that way, or just some different number for each card. What exactly do you mean by using a sigmoided prime number? – Miguel Saraiva – 2019-05-16T09:37:42.780

I mean squishing the number between 0 and 1 using the sigmoid function. – Harry Stuart – 2019-05-16T09:38:21.790

I don't think that would be necessary. By the way, for a simple representation like your case, sometimes linear function aproximators might work and converge better. – Miguel Saraiva – 2019-05-16T09:39:39.187

You can also use this kind of approach

– Miguel Saraiva – 2019-05-16T09:45:56.200