This will be a short tutorial about Unit testing with JUnit, Hamcrest and Mockito in our Terasology environment. Note that testing is a topic comparable to clean code and there are many opionions and options out there, so I will try to give a starting point from my own perspective and leave detailed analysis for own research.
1. Why waste time with tests?
Because tests save time
There are different opinions about it, one can test first (TDD) or after writing the production code, the code can be tested as blackbox or whitebox, tests can be done on unit level, integration level or ui level. In this tutorial, I will give one example for TDD, one example for the addition of tests.
Easy said: A unit tests will test one unit, e.g. one class of your code. It may use some related classes for testing but focuses on the unit without touching the environment.
On the other hand, an integration test checks the behavior of your unit in the system. E.g. in Terasology a unit test for a money system could test the system on it's own and that it sends correct events at the right time with the correct data. An integration test could load modules, try the code over the network, test the system with a game restart etc (little support for the last at the time of this tutorial).
Example: Test Driven Development of a simple money system.
Let's use an easy and short example. We want to implement a simple money system for Terasology.
Any entity could have a Wallet which saves a currency. We won't make assumptions about any currency, so we will simply use int as measurement. (Have a look at this stackoverflow, why it is a bad idea to use float or double for currency values).
To exchange money, entities can send events for a Transaction which moves currency from one wallet to another. We will forbid negative wallets, so a transaction should fail if the sender does not have enough money for the transaction.
One note before we start: I will not write any integration tests (yet) because I am working on a facade to do this easier. At the current state writing a clean integration test would be a bit hard because the setup of an entire testing environment takes some effort. There is a TerasologyTestingEnvironment in the engine-tests project which would do some of the things we need but sadly it is not really documented.
Now let's get started. For the ones not familar with TDD: I will write my tests first, then write enough code to make my test pass, then refactor my code if needed and continue this cycle until I am done.
We have only one requirement for our system: Dealing with transactions. We did not specify where wallets are initialized and we won't write any logic for it.
In the first step, we create a test class and create two mock entities which will return a wallet component.
WE HAVE NOT IMPLEMENTED ANYTHING YET!
The test fails because there is no WalletComponent yet, so we switch to our src/main/java folder and create an empty component. We also have to implement the Component interface, otherwise the call to getComponent would not compile.
The empty setter may look odd to you but remember the TDD rule? We won't write any code until we have a failing test! We don't need a getAmound or any real amount value yet.
Note: Setter/Getter used for the example to demonstrate that only logic required for the test is implemented. By convention, the component classes should have only public fields and no further methods.
Our next step will be the TransactionEvent handling. We have no logic for event handling in our entity system available in the tests, so we will just call the method and omit the logic for receiving events.
This would be a good spot for an integration test, where we can send an event to an actual entity and verify that the event is called. I won't cover this here and just annotate the system and the event handler as needed but for pure TDD, the next step after the unit test would be to write a failing integration test which enforces us to annotate the system and event handler correctly and extend one of the ComponentSystem classes like BaseComponentSystem, same for the event class which has to implement the Event interface!
At first, we define the API call in the test class:
Note the commented line, we can't send our event in the usual way and we won't test the event system here so we call the handler directly.
This test will not compile, so we create the required classes, the handler method and the event constructor.
as before, most of this is auto generated from the missing calls, we don't have to name the parameters yet or back them up with fields.
The test will pass again, so we will now add our first asserts.
Sidenote here: Some test conventions suggest one assert per test. We could easily split this up here and get one assert per test but I will skip this style change for simplicity in this tutorial.
Just keep in mind that refactoring won't end at your production code
In our test, entity 1 sends 5 money to entity 2. We can check that both components are updated.
If you wonder where these methods come from, these are static imports from org.junit.Assert, org.mockito.Mockito and org.hamcrest.CoreMatchers. I would highly recomment to use static imports as it makes the testing code much more readable.
Now the implementation requires our code to do something. So we start to implement the actual logic.
As before: We write only enough code to get our test green.
As before, we wrote code for methods which do not exist yet (wallet.setAmount) so we update the component too:
Running the test gives us a green bar. We use the time for a short refactoring:
becomes
with the methods:
This makes the code a bit more readable. Note that we did not change the function of the code.
To verify our change, we re-run the test (cool eh) and see everything still works as expected.
Next step and some Mockito gimmicks: when the wallet change, we want a event to be send with the old amount and the new amount.
Mockito has a nice concept to capture and verify arguments:
Using an argument captor, we can grab parameters from methods while we are inside a mockito verification.
Note that we define the captor after we called the actual logic but this works thanks to Mockito magic
We can get this code passing by changing the method like this:
Which will also need a new event:
Similar to this, we can add another test for the receiver.
At this point, our single test will be too long, so it will make sense to split it into smaller tests and extract the boilerplate and entity mocking to a @Before-Method.
The process itself won't change for the next stops, therefore I will stop the tutorial at this point but feel free to continue on your own you are read for it .
Summary
We implemented a simple transaction system based on the existing event system. As we went for unit tests and not for integration tests we made use of Mockito to fake the actual event handling.
In the first test, we focused on the core logic, which is the transfer from money from one wallet to another and sending a notification. Further tests will cover edge cases like negative amounts, empty wallets, events sent to entities without a wallet component (we can add the component filter TDD like if we are in the integration test layer). Later on we can watch out for refactoring and cleanup options in our system.
Maybe you read all this and think "okay, where is the benefit? I had the same system in my mind when you told me about the requirements, you still wasted your time with so much testing".
The real benefit from our testsuite is deeper than simply implementing something "that works":
1. Once I am done with TDD for all of the requirements, I can deliver this piece of code and be sure that it will work as defined. Do you have that confidence in every line you write? Would you deliver 250k lines of code and assure me "they run as expected"?
2. I can touch every piece of my code and refactor the hell out of it without the fear of breaking everything. I just re-run my tests, get my green bar and immediately know that it still works as before. Similar to the new methods we added earlier.
3. I can write documentation for my code relating to the tests. It is just a matter of writing down the asserts in readable form and further clients can read my docs, see that "the money system will send a notification after the wallet changes" or "that the components are saved after the transaction with the new values" and these statements are confidently backed up by the tests.
4. If someone else has to touch this piece of code in 3-4 years, he can touch any line and will have an immediate failing test if he breaks one of the original requirements, like allowing negative wallets by accident.
It went a bit longer than expected, hope it helps someone and feedback / further ideas are appreciated
1. Why waste time with tests?
Because tests save time
- Running fast automated tests is way cheaper than fixing each bug using explorative methods (aka digging in the debugger)
- Tests can be the first users of your api and therefore function as a specification
- If you can write a failing test for a bug, you are close to fixing it because you now understand the problem.
There are different opinions about it, one can test first (TDD) or after writing the production code, the code can be tested as blackbox or whitebox, tests can be done on unit level, integration level or ui level. In this tutorial, I will give one example for TDD, one example for the addition of tests.
- Test your main logic first, like the main task of your code
- Test edge-cases like invalid data, exceptions etc. later on
- Use a coverage analysis to find missing spots but don't use coverage as a quality metric
Reason for the last: Simply calling the main method will give you a huge code coverage without any assert in your testing code.
Easy said: A unit tests will test one unit, e.g. one class of your code. It may use some related classes for testing but focuses on the unit without touching the environment.
On the other hand, an integration test checks the behavior of your unit in the system. E.g. in Terasology a unit test for a money system could test the system on it's own and that it sends correct events at the right time with the correct data. An integration test could load modules, try the code over the network, test the system with a game restart etc (little support for the last at the time of this tutorial).
Example: Test Driven Development of a simple money system.
Let's use an easy and short example. We want to implement a simple money system for Terasology.
Any entity could have a Wallet which saves a currency. We won't make assumptions about any currency, so we will simply use int as measurement. (Have a look at this stackoverflow, why it is a bad idea to use float or double for currency values).
To exchange money, entities can send events for a Transaction which moves currency from one wallet to another. We will forbid negative wallets, so a transaction should fail if the sender does not have enough money for the transaction.
One note before we start: I will not write any integration tests (yet) because I am working on a facade to do this easier. At the current state writing a clean integration test would be a bit hard because the setup of an entire testing environment takes some effort. There is a TerasologyTestingEnvironment in the engine-tests project which would do some of the things we need but sadly it is not really documented.
Now let's get started. For the ones not familar with TDD: I will write my tests first, then write enough code to make my test pass, then refactor my code if needed and continue this cycle until I am done.
We have only one requirement for our system: Dealing with transactions. We did not specify where wallets are initialized and we won't write any logic for it.
In the first step, we create a test class and create two mock entities which will return a wallet component.
WE HAVE NOT IMPLEMENTED ANYTHING YET!
Code:
package org.terasology.money;
import org.junit.Test;
import org.terasology.entitySystem.entity.EntityRef;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
public class MoneySystemTest {
@Test
public void testSendMoney() {
WalletComponent wallet1 = new WalletComponent();
wallet1.setAmount(7);
EntityRef entity1 = mock(EntityRef.class);
when(entity1.getComponent(WalletComponent.class)).thenReturn(wallet1);
WalletComponent wallet2 = new WalletComponent();
wallet2.setAmount(3);
EntityRef entity2 = mock(EntityRef.class);
when(entity2.getComponent(WalletComponent.class)).thenReturn(wallet2);
//not done yet...
}
}
Code:
package org.terasology.money;
import org.terasology.entitySystem.Component;
public class WalletComponent implements Component {
public void setAmount(int amount) {
}
}Note: Setter/Getter used for the example to demonstrate that only logic required for the test is implemented. By convention, the component classes should have only public fields and no further methods.
Our next step will be the TransactionEvent handling. We have no logic for event handling in our entity system available in the tests, so we will just call the method and omit the logic for receiving events.
This would be a good spot for an integration test, where we can send an event to an actual entity and verify that the event is called. I won't cover this here and just annotate the system and the event handler as needed but for pure TDD, the next step after the unit test would be to write a failing integration test which enforces us to annotate the system and event handler correctly and extend one of the ComponentSystem classes like BaseComponentSystem, same for the event class which has to implement the Event interface!
At first, we define the API call in the test class:
Code:
...
when(entity2.getComponent(WalletComponent.class)).thenReturn(wallet2);
MoneySystem moneySystem = new MoneySystem();
SendMoneyEvent event = new SendMoneyEvent(entity1, 5);
// This is what we would do in an integration test:
// entity2.send(event);
moneySystem.onMoneySend(event, entity2);This test will not compile, so we create the required classes, the handler method and the event constructor.
Code:
package org.terasology.money;
import org.terasology.entitySystem.entity.EntityRef;
import org.terasology.entitySystem.event.ReceiveEvent;
import org.terasology.entitySystem.systems.BaseComponentSystem;
import org.terasology.entitySystem.systems.RegisterSystem;
//annotation and super class not part of TDD!
@RegisterSystem
public class MoneySystem extends BaseComponentSystem {
//annotation not part of TDD!
@ReceiveEvent
public void onMoneySend(SendMoneyEvent event, EntityRef entity2) {
// TODO Auto-generated method stub
}
}
Code:
package org.terasology.money;
import org.terasology.entitySystem.entity.EntityRef;
import org.terasology.entitySystem.event.Event;
//event class not part of TDD!
public class SendMoneyEvent implements Event {
public SendMoneyEvent(EntityRef entity1, int i) {
// TODO Auto-generated constructor stub
}
}The test will pass again, so we will now add our first asserts.
Sidenote here: Some test conventions suggest one assert per test. We could easily split this up here and get one assert per test but I will skip this style change for simplicity in this tutorial.
Just keep in mind that refactoring won't end at your production code
In our test, entity 1 sends 5 money to entity 2. We can check that both components are updated.
Code:
moneySystem.onMoneySend(event, entity2);
verify(entity1).saveComponent(isA(WalletComponent.class));
verify(entity2).saveComponent(isA(WalletComponent.class));
assertThat(wallet1.getAmount(), is(equalTo(7 - 5)));
assertThat(wallet2.getAmount(), is(equalTo(3 + 5)));Now the implementation requires our code to do something. So we start to implement the actual logic.
As before: We write only enough code to get our test green.
Code:
@ReceiveEvent
public void onMoneySend(SendMoneyEvent event, EntityRef receiver) {
EntityRef sender = event.getSender();
int amount = event.getAmount();
WalletComponent senderWallet = sender.getComponent(WalletComponent.class);
WalletComponent receiverWallet = receiver.getComponent(WalletComponent.class);
senderWallet.setAmount(senderWallet.getAmount() - amount);
receiverWallet.setAmount(receiverWallet.getAmount() + amount);
sender.saveComponent(senderWallet);
receiver.saveComponent(receiverWallet);
}
Code:
public class WalletComponent implements Component {
private int amount;
public void setAmount(int amount) {
this.amount = amount;
}
public int getAmount() {
return amount;
}
}
Code:
senderWallet.setAmount(senderWallet.getAmount() - amount);
receiverWallet.setAmount(receiverWallet.getAmount() + amount);
Code:
senderWallet.withdraw(amount);
receiverWallet.deposit(amount);
Code:
public void withdraw(int amount) {
this.amount -= amount;
}
public void deposit(int amount) {
this.amount += amount;
}To verify our change, we re-run the test (cool eh
) and see everything still works as expected.Next step and some Mockito gimmicks: when the wallet change, we want a event to be send with the old amount and the new amount.
Mockito has a nice concept to capture and verify arguments:
Code:
assertThat(wallet1.getAmount(), is(equalTo(7 - 5)));
assertThat(wallet2.getAmount(), is(equalTo(3 + 5)));
ArgumentCaptor<Event> captor = ArgumentCaptor.forClass(Event.class);
verify(entity1).send(captor.capture());
assertThat(captor.getValue(), is(instanceOf(WalletChangedEvent.class)));
WalletChangedEvent walletChangedEvent = (WalletChangedEvent) captor.getValue();
assertThat(walletChangedEvent.getNewAmount(), is(equalTo(2)));
assertThat(walletChangedEvent.getOldAmount(), is(equalTo(7)));Note that we define the captor after we called the actual logic but this works thanks to Mockito magic
We can get this code passing by changing the method like this:
Code:
@ReceiveEvent
public void onMoneySend(SendMoneyEvent event, EntityRef receiver) {
EntityRef sender = event.getSender();
int amount = event.getAmount();
WalletComponent senderWallet = sender.getComponent(WalletComponent.class);
WalletComponent receiverWallet = receiver.getComponent(WalletComponent.class);
int senderOldAmount = senderWallet.getAmount();
senderWallet.withdraw(amount);
receiverWallet.deposit(amount);
sender.saveComponent(senderWallet);
receiver.saveComponent(receiverWallet);
sender.send(new WalletChangedEvent(senderOldAmount, senderWallet.getAmount()));
}
Code:
package org.terasology.money;
import org.terasology.entitySystem.event.Event;
public class WalletChangedEvent implements Event {
private final int oldAmount;
private final int newAmount;
public WalletChangedEvent(int oldAmount, int newAmount) {
this.oldAmount = oldAmount;
this.newAmount = newAmount;
}
public int getOldAmount() {
return oldAmount;
}
public int getNewAmount() {
return newAmount;
}
}At this point, our single test will be too long, so it will make sense to split it into smaller tests and extract the boilerplate and entity mocking to a @Before-Method.
The process itself won't change for the next stops, therefore I will stop the tutorial at this point but feel free to continue on your own
you are read for it .Summary
We implemented a simple transaction system based on the existing event system. As we went for unit tests and not for integration tests we made use of Mockito to fake the actual event handling.
In the first test, we focused on the core logic, which is the transfer from money from one wallet to another and sending a notification. Further tests will cover edge cases like negative amounts, empty wallets, events sent to entities without a wallet component (we can add the component filter TDD like if we are in the integration test layer). Later on we can watch out for refactoring and cleanup options in our system.
Maybe you read all this and think "okay, where is the benefit? I had the same system in my mind when you told me about the requirements, you still wasted your time with so much testing".
The real benefit from our testsuite is deeper than simply implementing something "that works":
1. Once I am done with TDD for all of the requirements, I can deliver this piece of code and be sure that it will work as defined. Do you have that confidence in every line you write? Would you deliver 250k lines of code and assure me "they run as expected"?
2. I can touch every piece of my code and refactor the hell out of it without the fear of breaking everything. I just re-run my tests, get my green bar and immediately know that it still works as before. Similar to the new methods we added earlier.
3. I can write documentation for my code relating to the tests. It is just a matter of writing down the asserts in readable form and further clients can read my docs, see that "the money system will send a notification after the wallet changes" or "that the components are saved after the transaction with the new values" and these statements are confidently backed up by the tests.
4. If someone else has to touch this piece of code in 3-4 years, he can touch any line and will have an immediate failing test if he breaks one of the original requirements, like allowing negative wallets by accident.
It went a bit longer than expected, hope it helps someone and feedback / further ideas are appreciated
Last edited:
