Ranking
Original Post
[REL] Return of the machines: Machine learning environment using Toribash
In the past people have tried genetic algorithms multiple separate times to teach computers to play Toribash, including at least one academic publication. They had their limitations, but seemed very promising. Now several years later, shall we take another shot at it with recent advances in such fields?

Enter Toribash Learning Environment (ToriLLE), a platform designed for machine learning algorithms, meaning it is convenient to use from Python (de-facto machine learning language at the moment), it runs on Linux and runs fast enough. With the included OpenAI Gym environments people can try their existing machine learning methods on Toribash in a jiffy. We also wrote a white paper on the matter in the hopes it will lure more academy side people trying this environment out (granted, it is not the most modifiable environment).

Using the environment we trained proximal policy optimization (PPO) on smaller tasks to validate if Toribash is indeed suitable as a learning environment. Like in the past we tried destroying an Uke (damage = reward), but we also tried "run-away" scenario where player has to move away from the centre of the map.



Compared to previous attempts we positively tearing a new one for the uke! Maybe this time around we could also train and try these agents against human players?

If anybody feels like they want to contribute, support for objects and custom levels would be something very cool to have (i.e. getting the information on Python side). I do not know if Toribash supports getting information of such objects, and even after that it needs to be planned out how to share the information with Python. Changing mod is easy enough, but there is no point in that if we can't "see" the level.

Special thanks to hampa and Drax for helping out with some crucial parts, and user "box" for initial, encouraging comments.


Edit: Another video of PPO learning to attack Uke, this time with harder environment (random distance to Uke) and longer training (100k games rather than 10k). Looks like there is still room for improvement:


Last edited by Miffylin; Aug 6, 2018 at 07:54 PM.
I was overwhelmed by the amount of information. Lmao Someone explain it in dumb people language, what is this
Howdy, I'm Mod Squad & The Social Media Manager @ Nabi Studios , if you have any modding or social media questions PM me or DM me on Discord: Matarika#5297
Originally Posted by matarika View Post
I was overwhelmed by the amount of information. Lmao Someone explain it in dumb people language, what is this

ToriLLE: A way for people to play Toribash from their own code (mainly Python, but can use other programming languages too)

Video: A recent-ish reinforcement learning method trained to play Toribash. ToriLLE gives points on damaging the Uke, and algorithm learns via trial'n'error what gives most points.
You're doing gods work with this. Very cool
The 20k running away attempt is better than most newish players would manage, very interested in seeing how it performs with more training.

You should collaborate with hampa and the devs if you aren't already, maybe you can make something happen together in Next
Originally Posted by jisse View Post
You're doing gods work with this. Very cool
The 20k running away attempt is better than most newish players would manage, very interested in seeing how it performs with more training.

You should collaborate with hampa and the devs if you aren't already, maybe you can make something happen together in Next

I am currently running bit longer experiments with more sophisticated setups (agent sees Uke, engagement distance is randomized). Hopefully those will be more interesting in terms of movement. In Run-Away scenario the reward is given on where head has moved, so shouldn't detaching and throwing the head away be quite good?

hampa did already help with this a bit, and we did chat little about Next. As it will use Unity, it may not be as lightweight to run as this Toribash, but will see.
I was actually reading some of the posts you've made in the questions thread a few weeks ago and wondering if you had made any progress on it. I'm very intrigued by the results so far. Very impressed by the video.

Originally Posted by Miffylin View Post
support for objects and custom levels would be something very cool to have (i.e. getting the information on Python side). I do not know if Toribash supports getting information of such objects, and even after that it needs to be planned out how to share the information with Python. Changing mod is easy enough, but there is no point in that if we can't "see" the level.

Luckily, it does. For some reason, these functions are simply not documented (probably because the documentation hasn't been updated in a hot minute). Here are some world-object-specific functions that'll help:

get_obj_angular_vel
get_obj_bounce
get_obj_flag
get_obj_linear_vel
get_obj_mass
get_obj_pos
get_obj_rot
get_obj_sides
get_obj_vis

For the record, iterating over a table referenced by _G will give you this list of all functions, strings, and tables, including ones not listed where you'd expect them:

Credit to psycore for this



I would love to contribute to this project if I had the time, otherwise I look forward to seeing it improve. Please keep us updated!
Last edited by box; Aug 2, 2018 at 08:18 AM.
[23:23:53] <AndChat|700625> Blue eyes ultimate dragon best card
[23:24:29] <AndChat|700625> You know the one with 3 heads
[23:24:39] <~Lightningkid> just like my dick



[11:35:40] <box> Hampa suck
[11:36:21] <hampa> not the first to tell me that today
@box

Thank you for the kind words, and even more thanks for sharing the functions! I did use some old dump of functions and generated a more recent one, but by then I forgot to check for such functions. I will definitely try these out at some point. Currently I am bit busy with some deadlines. However I do have couple of experiments finishing up I will share soon enough ^^.

On an another note: Would an "AI competition" for Toribash sound interesting to people? It could include human-vs-AI trials too. A platform like CrowdAI could also lure people from machine learning side to this competition.
Amazing work! You are seriously working on something a ton of people have wanted for a very long time, as for the AI competition I think that could be a very fun event that a ton of people could enjoy if done properly, you'd have to go a bit more in detail but possibly get in touch with hampa and then our ES team and see what everyone else thinks, personally it seems like a very fun idea.
[SIGPIC][/SIGPIC]
~ raku ~ Team Girl Scouts ~ Clan League 2013 Champion ~ Duelist ~
Prince
Fucking
Ravioli
I have been looking into learning this kind of stuff recently but sadly school is about to start back for me so I will not be able to contribute... but it looks like this is going to go very well I with you guys the very best of luck!
Another clip of PPO learning to play Toribash, this time with over 100k episodes and harder environment (randomized distance to opponent):




@Natejas
Thank you! ^^ . It would be awesome to see if people come up with interesting and effective ways of doing crazy moves in Toribash with machine learning. Looking at my experiments looks like simply throwing existing methods at it does not quite cut it, yet.


@+Hotbox
Thank you for the kind words!