SerpentAI and Terasology

#1
One of the tasks in this year's GCI - and from my point of view so far the hardest task - was investigating the Serpent AI system and showing a bot playing Terasology. So, the first thing I would like to tell you - if you want to try that that yourself, think about it again. It might have been just for me, but there is so MUCH complications even INSTALLING Serpent AI I still have to decide whether it was worth it. Also, small disclaimer, I'm on linux, so your experience on windows might not be the same.

Dependencies
Large part of my frustration probably comes already from the first of dependencies, which is Python 3.6. On SerpentAI's wiki there is suggested to use pyenv for installation and handling of python, but I have rather decided to compile python 3.6 myself. That said about python. Next dependency is Redis server. No problem in installing this, but one caveat - once installed, it launches itself and constantly uses up about 800MB RAM, which may be (and for me, was) a problem. Tesseract was also alright - it is mainly a library, and only has a few dependencies itself. A bit more problematic dependency was Kivy, having lots of dependencies itself. In the end, after all the dependencies were installed, the final installation was just a piece of cake, being it installation through pip. I ditched the optional dependency, not needing it, and also not having the required hardware.

Setting up
Setting the Serpent AI to be capable of at least launching Terasology turned out to be very hard in the end. That's because Serpent requires an executable to launch. I tried using the files downloaded from terasology.org, I tried hooking it to generated jar files in terasology workspace, but nothing worked, so in the end, I had to mark as an executable a simple shell script, which then 'cd''d to workspace location, and performed a './gradlew jar game'. Detencting of the game's windows worked then without any problem.

Something to do in the game's window
For the final task, the bot has had to interact with the window somehow. I walked through Serpent's helloworld tutorial, which by the way gave me a lot of frustration because of how the important bits are hidden between the loads of the text. The final interaction bot performed with the game, consisted of clicking on predefined coordinates, which by the way corresponded to buttons required to click to create new world.

Final note
Because the serpent failed to detect when the game finishes loading, I had to manually modify its launch script and set it to sleep for 20 seconds after it thinks the game is loaded. Because the same script is however invoked when launching a bot, there is a 20 sec delay on that.

Video : https://drive.google.com/file/d/1rpzWpRh0denUncUAc9XfHyzuwC96BcdK/view?usp=sharing
 
#2
In reply to cervator's notes on GCI task:
  • There is no need in actually supplying my scripts. The launching script was really just cd /home/username/IdeaProjects/Terasolofy;./gradlew jar game > /dev/null. The modified serpentAI's script (for me, because of some messed up permissions /usr/local/lib/python3.6/site-packages/serpent/game.py) was then modified only in a way that in method after_launch(), I added time.sleep(60) at the beginning (then changed it to 20).
  • Sorry for the terrible video quality, it's recorder on camcorder. I don't however think that I can easily supply better one, since even with just the game and serpent running, I'm already at least 200 MB swapping. I don't even want to see how utterly lagged it would be if I tried to record it software way. Also, I had to compress that video since originally it was 200 MB in size.
  • It should be possible to get going even without the pause - the whole after_launch() method is in fact overridable in game agent, so looking with it through log and awaiting since the game is launched, but not yet closed would be possible.
  • About what is going on on screen - first command used was serpent launch Terasology - that launches the game. since stderr isn't redirected to /dev/null, some warnings get printed to console. Then I awaited line starting with curly braces, saying something about window position, since it shows that the game window wa located properly. Then, command prompt is shown. After first keypress, some error from gradlew is shown about stdin error, and then I input command serpent play Terasology SerpentTerasologyGameAgent, launching the bot itself.