people playing: 53, servers online: 21, games played: 142,000,000 Top Player: Imperial

  Toribash Community » Toribash » Mods » Lua scripts
Register FAQ General Rules Community
Reply
 
Thread Tools Display Modes
Old Jul 28, 2018   #1
Miffylin
Junior Member
 
White Belt
Join Date: Sep 2012



[REL] Return of the machines: Machine learning environment using Toribash
In the past people have tried genetic algorithms multiple separate times to teach computers to play Toribash, including at least one academic publication. They had their limitations, but seemed very promising. Now several years later, shall we take another shot at it with recent advances in such fields?

Enter Toribash Learning Environment (ToriLLE), a platform designed for machine learning algorithms, meaning it is convenient to use from Python (de-facto machine learning language at the moment), it runs on Linux and runs fast enough. With the included OpenAI Gym environments people can try their existing machine learning methods on Toribash in a jiffy. We also wrote a white paper on the matter in the hopes it will lure more academy side people trying this environment out (granted, it is not the most modifiable environment).

Using the environment we trained proximal policy optimization (PPO) on smaller tasks to validate if Toribash is indeed suitable as a learning environment. Like in the past we tried destroying an Uke (damage = reward), but we also tried "run-away" scenario where player has to move away from the centre of the map.



Compared to previous attempts we positively tearing a new one for the uke! Maybe this time around we could also train and try these agents against human players?

If anybody feels like they want to contribute, support for objects and custom levels would be something very cool to have (i.e. getting the information on Python side). I do not know if Toribash supports getting information of such objects, and even after that it needs to be planned out how to share the information with Python. Changing mod is easy enough, but there is no point in that if we can't "see" the level.

Special thanks to hampa and Drax for helping out with some crucial parts, and user "box" for initial, encouraging comments.


Edit: Another video of PPO learning to attack Uke, this time with harder environment (random distance to Uke) and longer training (100k games rather than 10k). Looks like there is still room for improvement:



Last edited by Miffylin; Aug 6, 2018 at 06:54 PM..
Miffylin is offline   Reply With Quote
Old Jul 28, 2018   #2
matarika
Senior Member
 
7th Dan Black Belt
Join Date: Apr 2011
Posts: 1,310
Clan: Evolution



I was overwhelmed by the amount of information. Lmao Someone explain it in dumb people language, what is this
Join the Assassin Brotherhood HERE
matarika is offline   Reply With Quote
Old Jul 28, 2018   #3
Miffylin
Junior Member
 
White Belt
Join Date: Sep 2012



Originally Posted by matarika View Post
I was overwhelmed by the amount of information. Lmao Someone explain it in dumb people language, what is this

ToriLLE: A way for people to play Toribash from their own code (mainly Python, but can use other programming languages too)

Video: A recent-ish reinforcement learning method trained to play Toribash. ToriLLE gives points on damaging the Uke, and algorithm learns via trial'n'error what gives most points.
Miffylin is offline   Reply With Quote
Old Jul 29, 2018   #4
jisse
Toribash Legend
 
4th Dan Black Belt
Join Date: Jul 2011
Posts: 2,846
Clan: [T]



You're doing gods work with this. Very cool
The 20k running away attempt is better than most newish players would manage, very interested in seeing how it performs with more training.

You should collaborate with hampa and the devs if you aren't already, maybe you can make something happen together in Next
jisse is offline   Reply With Quote
Old Jul 29, 2018   #5
Miffylin
Junior Member
 
White Belt
Join Date: Sep 2012



Originally Posted by jisse View Post
You're doing gods work with this. Very cool
The 20k running away attempt is better than most newish players would manage, very interested in seeing how it performs with more training.

You should collaborate with hampa and the devs if you aren't already, maybe you can make something happen together in Next

I am currently running bit longer experiments with more sophisticated setups (agent sees Uke, engagement distance is randomized). Hopefully those will be more interesting in terms of movement. In Run-Away scenario the reward is given on where head has moved, so shouldn't detaching and throwing the head away be quite good?

hampa did already help with this a bit, and we did chat little about Next. As it will use Unity, it may not be as lightweight to run as this Toribash, but will see.
Miffylin is offline   Reply With Quote
Old Aug 2, 2018   #6
box
?crazykid700=flashglass
 
Cardboard Belt
Join Date: Sep 2007
Posts: 9,244
Clan: I am



I was actually reading some of the posts you've made in the questions thread a few weeks ago and wondering if you had made any progress on it. I'm very intrigued by the results so far. Very impressed by the video.

Originally Posted by Miffylin View Post
support for objects and custom levels would be something very cool to have (i.e. getting the information on Python side). I do not know if Toribash supports getting information of such objects, and even after that it needs to be planned out how to share the information with Python. Changing mod is easy enough, but there is no point in that if we can't "see" the level.

Luckily, it does. For some reason, these functions are simply not documented (probably because the documentation hasn't been updated in a hot minute). Here are some world-object-specific functions that'll help:

get_obj_angular_vel
get_obj_bounce
get_obj_flag
get_obj_linear_vel
get_obj_mass
get_obj_pos
get_obj_rot
get_obj_sides
get_obj_vis

For the record, iterating over a table referenced by _G will give you this list of all functions, strings, and tables, including ones not listed where you'd expect them:

Credit to psycore for this



I would love to contribute to this project if I had the time, otherwise I look forward to seeing it improve. Please keep us updated!
[23:23:53] <AndChat|700625> Blue eyes ultimate dragon best card
[23:24:29] <AndChat|700625> You know the one with 3 heads
[23:24:39] <~Lightningkid> just like my dick



[11:35:40] <box> Hampa suck
[11:36:21] <hampa> not the first to tell me that today

Last edited by box; Aug 2, 2018 at 07:18 AM..
box is offline   Reply With Quote
Old Aug 2, 2018   #7
Miffylin
Junior Member
 
White Belt
Join Date: Sep 2012



@box

Thank you for the kind words, and even more thanks for sharing the functions! I did use some old dump of functions and generated a more recent one, but by then I forgot to check for such functions. I will definitely try these out at some point. Currently I am bit busy with some deadlines. However I do have couple of experiments finishing up I will share soon enough ^^.

On an another note: Would an "AI competition" for Toribash sound interesting to people? It could include human-vs-AI trials too. A platform like CrowdAI could also lure people from machine learning side to this competition.
Miffylin is offline   Reply With Quote
Old Aug 5, 2018   #8
Natejas

Hip 2 Be Square
 
The Bless Up Belt
Join Date: Feb 2011
Posts: 3,430



Amazing work! You are seriously working on something a ton of people have wanted for a very long time, as for the AI competition I think that could be a very fun event that a ton of people could enjoy if done properly, you'd have to go a bit more in detail but possibly get in touch with hampa and then our ES team and see what everyone else thinks, personally it seems like a very fun idea.
[SIGPIC][/SIGPIC]
~ raku ~ Team Girl Scouts ~ Clan League 2013 Champion ~ Duelist ~
Prince
Fucking
Ravioli
Natejas is offline   Reply With Quote
Old Aug 5, 2018   #9
+Hotbox
saSUKE's bitch
 
10th Dan Black Belt
Join Date: Aug 2017
Posts: 288



I have been looking into learning this kind of stuff recently but sadly school is about to start back for me so I will not be able to contribute... but it looks like this is going to go very well I with you guys the very best of luck!
I'm not a WEEB I swear
Hotbox is offline   Reply With Quote
Old Aug 6, 2018   #10
Miffylin
Junior Member
 
White Belt
Join Date: Sep 2012



Another clip of PPO learning to play Toribash, this time with over 100k episodes and harder environment (randomized distance to opponent):




@Natejas
Thank you! ^^ . It would be awesome to see if people come up with interesting and effective ways of doing crazy moves in Toribash with machine learning. Looking at my experiments looks like simply throwing existing methods at it does not quite cut it, yet.


@+Hotbox
Thank you for the kind words!
Miffylin is offline   Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:04 PM.




Powered by vBulletin® Version 3.8.2
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
ragdoll fighting game physics fighting game ragdoll fighting physics funmotion joints martial arts karate pc mac free game turn based game