What type of neural network would be most feasible for playing a realtime game?



For implementing a neural network algorithm that can play air hockey, I had two ideas for input, and I'm trying to figure out which design would be most viable.

The output must be two analog values that dictate the best position on half of the table for the robot to be at a specific point in time, evaluated 60 times per second.

Having consulted with a professor who has experience with implementing parallel algorithms, it was recommended to me to use a convolutional neural network with a single hidden layer, and directly process the image data as the input layer, after processing it to visualize a direct view of the table and somehow emphasize the puck and mallets with pre-processing. I have already started work on this and have successfully implemented object detection for the puck and mallets using OpenCV to get the center coordinates for all 3 entities.

However, having been able to successfully and accurately pre-process these data at 60 times per second, my thought was to feed those (after normalizing the values using the error function) directly on the input layer and possibly implement a deep learning algorithm that employs more than one hidden layer. The problem is that I don't have any experience implementing neural networks, so I'm not even sure what type of layer would be best for this, or how I should seed the weights.

Another reason I want to consider my idea is that given the relatively few inputs and outputs, probably won't need a GPU to execute forward-propagation 60 times per second, whereas with the convolutional network my professor recommended, I know I'll need to implement using CUDA somehow.

Which of the two input methods would be most recommended for this, and if I were to try and implement my idea, what types of layers should I consider? Also any recommendations for existing frameworks to use for either approach would be highly appreciated.

Patrick Roberts

Posted 2017-09-28T17:11:21.753

Reputation: 133



I recommend you read up on reinforcement learning. Seeing how AirHockey is similar to the old Atari game Pong, here is a write-up (with code) about how to implement a simple neural network, that plays the game; Deep Reinforcement Learning: Pong from Pixels.

As far as the choice of layers go, again, you should read up on deep reinforcement learning, perhaps study some existing models that work, for a similar task and take their architecture as your starting point. The above link is a nice introduction into such thinking.

As a continuation, I would recomend you employ one of the well known, and developed, libraries, such as TensorFlow or pyTorch. They will allow you to run your models on GPU, like you desire.

For a toolkit, that specializes to reinforcement learning, see openAI gym.

Žiga Sajovic

Posted 2017-09-28T17:11:21.753

Reputation: 136

1The first link was immensely helpful, and based on the information, I believe I'll go with the convolutional neural network like my professor recommended, using the preprocessed image frames as input. However, I believe based on my updated understanding that my 2 outputs should be the velocity of moving up and to the right, (after subtracting 0.5 from each and normalizing to the highest velocity our robot can support), that way it's action based rather than location based. – Patrick Roberts – 2017-09-29T17:07:59.930

I'm glad. Hope your project goes well. – Žiga Sajovic – 2017-09-29T17:13:06.497


I also recommend you take a look at the following work by Uber AI Labs who used an interesting approach to computer games:


Mike AI

Posted 2017-09-28T17:11:21.753

Reputation: 127

Nice link. This topic has definitely come up on the forum. – DukeZhou – 2018-02-18T21:28:14.100