Originally Posted by
Nuyashaki
Ok then, what's the current progress? Has it learned to rip ukes head off yet?
The results were not pleasing (yet), for a couple of reasons. If I explained exactly why, I risk going over the text limit and I doubt many people will read it anyway, but the short version is:
1. I haven't formulated the problem in a clever enough way
2. Theres been mad frustration trying to get my code to play nice with Toribash remotely on a network right now, so I'm just using my own personal computer as the sole number cruncher.
One point one:
Jok was exactly right when he said that the amount of inputs I was using was too much to start with; this is MOSTLY because NEAT (especially the version I had adapted, which was quite old) isn't a very good algorithm when the dimensionality of your search space increases. In simpler terms, this means that the more information you give it, and the more complicated the problem, the slower progress will be (like REALLY slow). It's also called the curse of dimensionality; there are ways to get around it, more on that later
Technical reasons for it being slow:
By default, NEAT starts out with a feedforward perceptron where ALL of the inputs are connected; this means that its not left up to evolution which sensors are important, since by default they are all connected; when your inputs are from orthogonal coordinates (like x,y,z triplets for joint positions, which I was using as inputs), this is very counterproductive to making good nets.
Another frustration I had was the version of NEAT I was using was coded by some italian guy about 5 years ago, and his code is terrible and fragile; its not even written for controller problems like many of the other implementations. Every time I changed something everything surrounding it would break, so I was rewriting so much stuff that I thought it was a waste of time to go further in that direction; I was better off starting from scratch and making it really good. I thought it would be trivial and easy to patch these existing bits of software together, but it looks like if you want it done right you have to do it yourself.
What I'm doing now/Where things go from here:
First; reducing the amount of inputs. You may have seen the threads/posts I made recently about joint extension and balance. Look at the way
biological neural nets handle walking/locomotion; for us bipedal humans, its a matter of leaning forward to put yourself off balance, and catching yourself; you repeat the cycle to walk, so long as you don't lose balance completely and fall over. We use our sense of vision, balance, and of the relative positions of our body parts. You can even walk with your eyes closed, so in other words, all your brain needs to walk is information about Equilibrioception, and Proprioception. Those are big words that mean your sense of balance, and your sense of where your bodyparts are in relation to each other (like how you can touch the back of your head in the dark). Part of our balance system is also detecting the horizontal acceleration and the vertical acceleration so a neural net that is learning to walk should have access to those as well.
Its obviously going to be a lot easier for an artifical neural net to use balance and a sense of where its body parts are to walk, than to use x y and z values for the position of a joint. Instead of giving it 20x3 inputs (60 inputs is going to be SLOW), I can just give the neural net 25 inputs; 20 inputs representing the extension level of each joint, 1 input representing forward/backward orientation of the head, 1 input representing left/right orientation of the head, one input representing magnitude of horizontal acceleration on the head, and one input representing magnitude of vertical acceleration on the head.
The last input is feedback on how good the AI is doing (if we want it to walk, we should provide feedback on how close/far it is from its goal destination). This is to simulate the visual feedback we have to determine how far/close we are from an object.
Thats all our brain needs to walk, so it should be enough for my ai. 24 inputs is also a lot more feasible than 6. Not only that but what I've been doing for the past couple of weeks is studying multidimensional optimization algorithms and evolution strategies; it seems like there is a MUCH better approach out there that can bypass all these difficulties I've had.
I'm going to be writing from scratch my own implementation of EANT2; this is a similar idea to NEAT but it guarantees much better performance and generality; for double pole balancing, CMA-ES (which is a sub-algorithm incorporated into EANT2) was over 8 times faster than NEAT. By reformulating this Toribash problem correctly I think I'll be able to make it a lot more feasible. On the super plus side, once I'm done I'll be able to take my system and plug it into any game with a "score" value, and it should be able to spit out a good AI.