Environment-based Tests and Headless Server questions

[Iaron da Costa Araújo]

I've been studying issue 2780 (https://github.com/MovingBlocks/Terasology/issues/2780) to prepare my GSoC proposal, and while doing so I had some doubts:
What exactly is the Headless Server? From what I've seen, it looks like a server meant to run the game without players and it's used to test the game. Also, what exactly do you expect the headless client to do after issue 2780 is implemented?
About Environment-based tests, right now, they work like normal Unit Tests, right? The goal of this issue is to improve them so they can run alongside the game and therefore also work as integration test?

Thank you all for your time!

 

[Cervator]

So the headless server already exists and is just a way to use the game to run a game server without actually logging in as a client as well. So that's standard multiplayer stuff, not related specifically to testing.

Headless client would be similar in the sense that it will be a sort of simulated client. You wouldn't do any rendering, instead we would need a more complete way of running and interacting with the game purely through some sort of scripted approach akin to unit tests, or maybe more aptly automated acceptance testing like Selenium, but for a game. Note that there are two approaches there:

• Automated acceptance tests with a headed environment: This is like rerunning a Selenium script right in the browser. Everything is still being rendered and you can see how the test works. Probably we would have some record + replay functionality in the regular game client that would do this. The Factorio blog entry listed from the GitHub issue is like that. This whole part could be done without a headless client.
• Automated acceptance tests with a headless environment: After the automated script is stable it should be possible to run it without a full client rendering the game. That's what requires the headless client. It would allow us to run the tests in Jenkins which doesn't have a desktop environment to run the game in. Often Selenium scripts are replayed using cloud servers with desktop environments, but there are ways to run them headless as well.

Current environment-based tests are working just like unit tests, yep. But the environment/world created as part of that is absolutely minimal, and is really just an empty shell. The headless client would aim to hit a regular game server and run its tests that way.

I'm not sure if the current headless environment tests would directly relate to the headless client - they're unit test / integration tests / component tests or something while the headless client is more for automated acceptance testing. Maybe the headless client could run against a local partial or complete environment rather than just a regular standalone server.