Architecture: Entity System

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
And official announcement that we're tagging Immortius with lead architect on the core stuff like the entity system and networking :)

I've been tinkering with Artemis but am getting nowhere fast, and Immortius has actual experience with using an ES in his Nightblock implementation. Additionally, we want to go from there to multiplayer down the road, and those two systems would go very close together - in fact, may be the same under some scenarios.

I'll still be happy to tinker, and some of the Systems we want are very close to stuff I've already worked on, like portals and spawning mobs. I've got pending code interfacing with Artemis and am still trying to get that checked in soon - but keep finding other stuff that needs to be done too!
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Nice! Thanks for the link. Looking to put it to use? :D
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I'm thinking something along those lines Heldenhaft, though not sure about what network library to use yet.

I guess I'll give a quick overview of what I'm experimenting with at the moment.

1. Entity System.

The basic idea behind an entity system is to represent entities as a composition of components. Components only contain data (no logic), and entities themselves are just identifiers used to group them. For example a GelatinousCube may have a Location component for where it is in the world, a Collision component for how it collides, a Mesh component to display it, a MovementController to hold movement directions, and an AIBehaviour to hold behavioural state.

Then you have a number of systems, each of which work across entities with a desired set of components. A Renderer system would render entities with both a Location and Mesh component for instance.

One of the major benefits of an entity system is the use of composition to produce different behaviours. It becomes possible to add new items, monsters, and so on simply though different compositions of components and different settings for those components. Take a gelatinous cube, add a particle effect component producing fire particles and increase the scale on the mesh component you can get the dreaded Giant Fire Cube for instance. More advance modding can introduce new systems and/or components as well.

I've rolled my own implementation for the moment, while I experiment with things. It probably can be improved with some of the Artemis performance trickery at some point. A version built on some sort of database (perhaps OrientDB) may provide some interesting features too.

2. Event System

Working hand in hand with the Entity System, I've started putting together an event system that allows you to "send" an event to an entity, which is propagated to event handler systems that have subscribed to events where particular combinations of components are present.

For instance, you could have a UseEvent that is sent when a player interacts with an item in the world:
Code:
public class UseEvent implements Event {
    public EntityRef user;
}
And register an EventHandler that reacts to UseEvents on entities with an ItemDispenser component:
Code:
public class ItemDispenserSystem implements EventHandler {

    @ReceiveEvent
    public void onUse(UseEvent event, EntityRef entity, ItemDispenserComponent component) {
        // ... Functionality goes here
    }
}
Then you just send the events against the entities and the correct systems get propagated the events.

I'm still messing with how exactly this should work, and there are some considerations I still need to explore (how do you handle the case where you have a lock component + event handler that should intercept the UseEvent and stop it reaching other systems? Priority + boolean return value to end the event?).

One thing I have done is added events around Component lifecycle events - when they are added to an entity, removed from an entity or updated.

3. Prefabs

This is the other core element for the entity system - a way to define recipes for entities that can be instantiated. Basically these are very similar to entities - groups of components - except they are never processed, only copied to create new entities. One thing is that components should be allowed to have references to prefabs so they create them - for instance a projectile weapon could have a reference to the prefab for the projectiles it shoots.

These can be set up to be defined in groovy config files.

Multiplayer

I've been musing about how this may work in a multiplayer game. Systems are probably fairly simple - you'ld have some systems that would only run on the server or client, and others that run on both but may behave a bit different depending on where they are.

Each component may also either be marked up or have some simple methods to determine whether they need to be replicated from server to client, and which properties need to be replicated, perhaps something like:

Code:
@ReplicatedComponent
public class Player implements Component {
    @Replicate(onCreateOnly=true) public String name;
    @Replicate public int score;
    public String password;
}
A somewhat artificial example, onCreateOnly means it only needs to be sent when the Component is first added or the Entity becomes relevent for a player, and is otherwise constant. There may also be a specific component to provide additonal replication information or to say a given entity should be replicated at all. Then a central system would replicate new or changed entities to players, if the entites are relevent and at a rate that won't flood the network.
 

Heldenhaft

New Member
Hi,
a few notes on my googlecode project....

Java
I use no singletons, because i can start the client and the server on the same vm....for testing and so on it is very nice...
For the event system i use a very simple esb and a basic fsm...

Code:
world.getEventBusManager ().subscribeOnSimpleEventBus (EntityCreateMessage.class,
						new EventHandler<EntityCreateMessage> ()
						{
							@Override
							public void handleEvent (EntityCreateMessage event)
							{
								int lifeTime = (int) (event.getTick () - world.getGameTimeManager ().getNetworkTime () + event.getLifeTime ());
						
								createByTemplateId (event.getTemplateId (), lifeTime, -1, event.getUniqueEntityId ());
							}
						});
Raw networklayer
In the raw/first network layer i use netty with simple Opcode based messages:
Code:
// Without any data your length is 2
	CLOCK((byte) 0x01, 10),
	SEQUENCE((byte)(Options.DYNAMIC_LENGTH_MESSAGE | (byte)0x02), 15),
	RELIABLE((byte)(Options.DYNAMIC_LENGTH_MESSAGE | (byte)0x03), 11),
	LOGIN ((byte)(Options.DYNAMIC_LENGTH_MESSAGE | (byte)0x04), 6),
	RELIABLE_ACK ((byte) 0x05, 6),
	PINGPONG ((byte) 0x06, 10), 
	LOGIN_RESPONSE ((byte) 0x07, 4), 
	// Entity default length is 10
	ENTITY_CREATE ((byte) 0x08, 15), 
	ENTITY_POSITION ((byte) 0x9, 18), 
	ENTITY_MOVEWAY((byte) 0x10, 30),
	ENTITY_DESTROY((byte) 0x11, 11),
	USER_ENTITY_CREATE ((byte)0x12, 11),
	USER_INPUT ((byte) 0x13, 27), 
	SEQUENCE_ACK((byte) 0x14, 10),
	CHAT_MESSAGE((byte)(Options.DYNAMIC_LENGTH_MESSAGE | 0x15), 6), 
	NEW_USER((byte)(Options.DYNAMIC_LENGTH_MESSAGE | 0x16), 6);
The messages use a simple serialized method:

Code:
public class EntityCreateMessage extends EntityNetworkMessage
{
	private byte templateId;
	private int lifeTime;

	public EntityCreateMessage (byte templateId)
	{
		super (Opcode.ENTITY_CREATE);
		this.templateId = templateId;
	}

	@Override
	public void transfer ()
	{
		super.transfer ();
		channelBuffer.writeByte (templateId);
		channelBuffer.writeInt (lifeTime);
	}
Network-Stack

The udp networkstack implements differents methods from Doom and HL2 (Valve)... I have sequence id for sorting and
a reliable flag for important messages. Reliable message will send unitl i received a ack-packet. For the movement i send
only delta packets (the last pos).

Movement
In my game all players plays in "history"-> 100ms+ping in the past. The local player movement on the client is directly.
On the server i read the current position and check all other components:

Code:
commonWorld.getEventBusManager ().subscribeOnSimpleEventBus (UserInputMessage.class,
						new EventHandler<UserInputMessage> ()
						{
							@Override
							public void handleEvent (UserInputMessage event)
							{
								String networkId = event.getNetworkId ();
								User user = commonWorld.getUserManager ().findByNetworkId (networkId);

								Entity userEntity = commonWorld.getEntityByUniqueId (user.getEntityId ());

								Position pos = userEntity.getComponent (Position.class);

								pos.setLocation (event.getPosX (), event.getPosY ());
(On the client i interpolate the positions to get a smooth movement)

If the player fire i do more things...

Code:
								if (event.getMouseClicked () > 0)
								{
									Position userPos = userEntity.getComponent (Position.class);

									EntityManager entityManager = commonWorld.getSkillFullEntityManager ();
									Entity entity = entityManager.createByTemplateId ((byte) 2, 0, (int) user.getEntityId (), -1);
									entity.getComponent (Position.class).setLocation (userPos.getX (), userPos.getY ());
									entity.getComponent (PositionHistory.class).setExpandableTimeRange (false);
									entity.refresh ();
									EntityCreateMessage entityCreateMessage = new EntityCreateMessage ((byte) 2);
									entityCreateMessage.setUniqueEntityId ((int) entity.getUniqueId ());
									entityCreateMessage.setTick (event.getTick ());

									EntitySegmentMoveMessage entitySegmentMoveMessage = new EntitySegmentMoveMessage ();
.
.
.									
..									
									// Default shot length...
									Expires entityExpires = entity.getComponent (Expires.class);
									entityExpires.setLifeTime ((int) t + 1000);
									entityCreateMessage.setLifeTime ((int) t);

									SegmentMove moveWayComp = entity.getComponent (SegmentMove.class);
.
.
.
									SegmentMoveSystem segmentMoveSystem = commonWorld.getSystemManager ().getSystem (
													SegmentMoveSystem.class);
The shoot can fire in the history of the server time, so i simulate the movement on time...
Code:
									for (long simulatedTime = event.getTick (); simulatedTime < moveWayComp
													.getDestinationTick (); simulatedTime += commonWorld.getDelta ())
									{
										segmentMoveSystem.calcMovement (entity, simulatedTime, true);
									}

									for (User dstUser : getUsers ())
									{
										commonWorld.getNetwork ().sendMessage (dstUser, entityCreateMessage);
										commonWorld.getNetwork ().sendMessage (dstUser, entitySegmentMoveMessage);
									}
									// Check Collision
To check all the collisions on different Player time i can rewind the complete game state to a given time.
No i check all collisions on the history game state...

Code:
									CollisionSystem collisionSystem = commonWorld.getSystemManager ().getSystem (
													CollisionSystem.class);
									PositionHistorySystem posHistorySystem = commonWorld.getSystemManager ().getSystem (PositionHistorySystem.class);

									posHistorySystem.reward ();
									collisionSystem.setRewindMode ();
									long delta = commonWorld.getDelta ();
									for (long backInTime = event.getTick (); backInTime < commonWorld.getGameTimeManager ().getNetworkTime (); backInTime += delta)
									{
										// Check Collision
										collisionSystem.setTime (backInTime);
										collisionSystem.process ();
									}
									collisionSystem.setForwardMode ();
									posHistorySystem.forward ();
									posHistorySystem.process ();

								}


[Artemis]
The use of artemis with his systems make it very easy to use the components on different situations...

Sorry for my bad english...

questions :)?
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
It is rare for me to be wall-of-texted into silence! Usually I'm on the sending side. I shall recuperate and return to WOT posting post-haste! :D

Although, hard coded bytes with super low level networking? Aiieee! But my knowledge there is near-nil, so I don't even truly know what I'm looking at.
 

Heldenhaft

New Member
:)Yes...basic messages with "hard coded bytes" (I use enums :)) are very small... and very fast for an "action" game :)...
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
The entity system branch is now in a reasonably decent state - I've replaced the gelatinous cubes, players and inventory system with entities, along with a few other things. Everything is working pretty much the same as previously, with some improvements along the way. At this point it would be good if people can review and provide some feedback, and then we can fold into develop or discard as desirable.

The major bits of code and their state are:

org.terasology.entitySystem - the entity system and entity eveny system. The implementation is intended to be replaced with something more sophisticated (and probably nosql database backed), so is contained behind a set of interfaces. t3hk0d3s prefab support is also in here

org.terasology.components - the various components used at this stage. Some of them are very rough (MeshComponent) or temporary (AABBCollisionComponent), others are much more solid (LocationComponent, CharacterMovementComponent)

org.terasology.componentSystems - the various component systems that process and update the entities. These are merely a first pass, and generally need firming up.

org.terasology.entityFactory - some helper classes for generating entities with the correct components.

org.terasology.events - the events used. These are basically broadcast onto an entity, and sent on to the relevent system based on what components the entity has.

org.terasology.logic.LocalPlayer - wrapper arpund the entity for the local player (as opposed to remote players as will later be the case)

org.terasology.game.CoreRegistry - the core registry is an alternative to singletons for storing major service classes like the EntityManager or AudioManager. The advantage is that the service classes may be registered against an interface, so you can request whatever implementation of EntityManager has been registered. This is a very basic implementation

org.terasology.game.ComponentSystemManager - a second registry specific for Component Systems. This registry sorts the systems into separate lists for iterating over for different sorts of usage (update, rendering) and also registers EventHandlingSystems with the event system.

Future work
  • Improve handling of user input, possibly have available for processing from a centralised Input class, or send as events to systems (most likely a combination of the two).
  • Improve item entities to support multiple use options (at the moment that have a single left-click use, and can be used to attack blocks)
  • Reimplement the Blueprint tool
  • Start making use of prefabs, so that entities can be defined in files.
  • Create a camera component, so that entities can be used for cameras as well (would want to attach the player camera to the player entity, for instance)
  • Add one or more physics related components and component systems.
  • Update the entity system to work on a nosql db
  • Persist and restore entity state.
  • Rework core Game/Renderer/World logic to be cleaner in preparation for multiplayer work later - mostly changes to the core structure (so that WorldProvider is not produced by the Renderer for instance)
  • Work out how UI elements fit into this system
  • Generally grow and improve components and component systems
  • UI for browsing entities for debugging
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Sounds great Immortius – will look into it and give you feedback by next week. :)
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Yep, looking forward to playing with this some more :)

I might try to push a little further ahead with Artemis first just to learn more, since I've gotten a little of the mindset from that one. Then see if everything useful from there has parallels in this system, and if not see what can be enhanced.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps

Immortius

Lead Software Architect
Contributor
Architecture
GUI
That is a nice set of articles.

I guess in my view the Nodes and Families are an implementation detail regarding how entities get related to different systems. I've been a bit more fluid in the work I've done, with systems able to just ask for all entities with a desired set of components from the central manager. There are benefits and costs to either technique.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Spent some time messing with the possibility of using OrientDB to sit under the entity system.

Pros:
- Was able to integrate it in without needing to change the existing entity system interface much.
- Allows for more interesting queries
- Very simple to persist
- Navigating from an entity to its components was quicker

Cons
- Performance was not adequate on finding an entity or saving changes.

Worst bit is that doing the regular "find all entities with desired components" query was many times more expensive than the simple hash-map based lookup. Saving was also expensive, but could be delayed until needed unless the information in a component was needed for querying. The game was still running at 40-60 fps, but I'ld desire a higher margin.

I have committed a branch with the system, but will be discarding the idea and instead moving on to just persisting the entity state of the POJO entity manager (through protocol buffers, to either json for debugging or binary), borrowing on some of the work I did for the OrientDB experiment.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
:(

Well thanks for trying and tossing the code on the web. Learning experiences are always good :)
 
Top