Neural network help on game of continuous snake

Question

I'm trying to implement an AI for a game of 'continuous snake'. It's very different from a normal snake game, at least as far as the AI is concerned. Basically, the snake drives a bit like a car and the first one of the 2 players to crash into his trail or the other's trail loses the game. Also the screen wraps around its borders.

You can understand it better if you look at a video of my current progress: https://www.youtube.com/watch?v=i9qU-r4COQ8

It's not too bad, but it still can't beat me (I'm yellow). A winning AI would ideally need to exhibit these behaviors:

1. Avoid walls
2. Notice occasions when it can 'cut me short' (when next to me a bit ahead).
3. Avoid getting 'cut short'.
4. Have an idea of the topology of the current 2d space to try to enclose me in a smaller space / safeguard himself a bigger space.

My current approach uses the NEAT algorithm (http://www.cs.ucf.edu/~kstanley/neat.html). It's a genetic algorithm that evolves neural networks over generations. It learned how to do 1,2 and 3 to some extent (but not great) but has no idea about 4.

For the inputs, I'm using:

• the opponent angle relative to us
• the opponent distance to us
• the opponent heading relative to us
• smart rays that probe in some directions with some amount of tree search (see video)

I'm a bit stuck now though and would like to know:

• What's the class of algorithms I should look into ? Recurrent / RealTime / Continous / Unsupervised Neural networks, ... An explanation about these and how they would apply to my problem would be great.
• Any specific algorithms I should research ?
• What other sets of inputs could I use ? A human player gets to see all the pixels in the game which is a lot more information than my simple set of inputs. But I don't think feeding the 200x200 pixels in my example to my NN would work at all. Maybe if I discretize them and make them relative to the AI position/heading...sounds tricky.

I'm happy to make my code available if someone wants to see it (C#).

Thanks!

Answer 1

The main problem here is that your learning algorithm doesn't have enough information (unless you are using the recurrency feature). Basically, each frame you're asking it to navigate a maze by a few distance sensors - impossible.

What singhV said before is true - for good results, the input for the learning algorithm has to be the image (as well as your own head position and angle). You can lower the resolution a bit and convert to monochrome for efficiency.

As for your questions: * Recurrent networks are networks that can remember previous state, and use it like "memory" basically. Its not what you need for this task (unless you really want to keep the inputs as they are, but then the snake would have to learn "remembering" all it saw - it would be pretty impressive but too hard) * Unsupervised: it means that there are no "examples" to learn from, instead learning by positive / negative feedback (losing = bad). Your network is unsupervised. * Real Time / Continuous - no idea, i didnt find anything except some 2007 microsoft research: https://www.microsoft.com/en-us/research/publication/continuous-neural-networks/

By the way, NEAT is pretty neat, I'm glad I ran into this!