Planning the multiplayer architecture

[begla]

I'm currently thinking about implementing a first prototype of the multiplayer aspect of the game and I've started to restructure parts of the current architecture to support networking more easily. This prototype is very important in my opinion, since any further changes on my list might have a direct influence on the network performance (thinking of multiple depth layers for example) and have to be optimized accordingly.

With the mentioned changes, the forthcoming server application can use an instance of the class LocalWorldProvider to generate worlds (and chunks of course) and the class World is responsible for all the fancy rendering stuff, mapping the player to the world and so on and so on.

My architectural goal is the following: a client application should only differ from the local version in the type of WorldProvider assigned to the World.

To summarize: A local game uses an instance of the class World with an assigned instance of the class LocalWorldProvider as a world provider. A server application works the same way (but the World class can be skipped since no rendering will be needed). The client application will use an instance of World in combination with some kind of new world provider (a RemoteWorldProvider maybe) bound to a remote server. This new provider will manage the requesting of new chunks from the server, applying remote changes sent from the server. The generation of chunks can be split in an arbitrary amount of threads, since the current architecture supports the complete parallelization of the chunk generation process. The goal is to minimize the overall data transferred. If a chunk has not been touched on the server, it can be easily generated locally (to reduce the server load). Modifications of chunks will be transferred in the form of setBlock(x,y,z, 0x66) and the lighting will be calculated locally and so on and so on. Thinking of... Lighting can even be completely ignored on the server, since it can be fully reproduced on the side of the clients. But that's another story.

Any thoughts on this approach?

I also need some ideas regarding the network protocol. I would start using an self-specified protocol using TCP or UDP. But maybe there is a much more sophisticated way that takes away all the hassle? Maybe Rasmus got a good idea here... But wait – I'm sure he has.

 

[Cervator]

I saw the provider stuff pop up and am eager to talk more architecture on networking

First I'm wondering if you have looked around any for publicly available libraries for networking? I'm only aware of two, but then I only found them indirectly and I haven't really spent any effort to figure out what's out there in Java.

The two I'm aware of and have read up a little are SpiderMonkey (the networking component of jMonkeyEngine) and RedDwarfServer, which is far more advanced, has an interesting task-based approach, and does trivial sharing of objects across the network (with probably not-so-trivial review to optimize performance - especially since it is TCP only, not UDP).

I'm not sure either of those fit our needs ideally, especially since they're both sort of their own thing. There are bound to be more options out there, maybe more powerful ones too. While we could probably do our own networking I'd like to think we could cut out a lot of effort and trouble by finding a nice framework, similarly to using LWJGL

Trying to see how Blockmania would run on RDS is still one of the little side projects I wouldn't mind trying out (especially also to just learn more about RDS), but I'll probably not find enough time for that for a while

One additional quirk is what considerations might come out of a future "fancy" server infrastructure, if we do any kind of premium setup or integrate with an existing service like Steam (I liked how well the multiplayer worked in something like L4D2 with public dedicated servers - but that's "beyond" a simple client-server infrastructure)

Another thought to consider is how we want to do storage of block data, inventory data, etc. I picture a persistent world where blocks exist in a few distinct states:

* Pristine - generated and untouched by any player (can still see it and jump on it all day without any impact, could regenerate from seed with no loss)
* Interacted - basic block that has been interacted with by a player, such as placed blocks making up a small house in the middle of nowhere. Fully abandoned structures like this might over time decay back toward pristine, potentially leaving ruins around for a while to show the world changing (likely an advanced content feature and/or might be a performance tweak we can use to not store a ton of old custom chunks nobody cares about)
* Customized - may be the same as interacted, but for custom blocks created by a player (so not basic placed blocks, but a custom block made up by mini-blocks in some arrangement) - not sure if this makes sense for separate storage considerations. Imagine a wall destroyed by player action (or just some mined blocks), leaving a clutter of physics-enabled mini-blocks laying around not being picked up by a player. Short-term those should slowly revert to static blocks, probably snapping to a custom block of an approximate shape of the clutter. Long-term they could "erode" into pure basic blocks, even tho erosion generally doesn't make terrain more blocky - it makes sense for data storage tho!
* Tagged - blocks that are part of a player's estate could be tagged with ownership of some sort, probably linked to the portal-estate system. You'd only be able to tag blocks in a limited radius around a central object that defines your estate, itself likely linked to a village Portal (think Spawn). These blocks need to be permanently stored in their custom state and could be protected against other players (anti-griefing) or the environment somewhat. That middle of nowhere outpost? No protection for you, need an active player near or it'll decay over time and make the server happy. Bonus: Make only minions able to tag blocks (by some ceremonious process that takes a little time)

I'm not as gung-ho on multi-level Z chunks as I was at first, it could be neat, but we'd probably be better off just making considerations for maybe supporting the addition of it in the future rather than try to actually do it up front. You could totally start with just one Z chunk and just release "SURPRISE UNDERWORLD!" one day that can be generated right below the players' feet in an existing world. Just release a new "tech" by which you can dig through the bedrock layer (which really could just be standard rock of a certain hardness you just don't have tools able to overcome yet)

 

[Adeon]

Stop, stop, Let's describe all the points step by step. Otherwise I can confuse

So, the server apllication must:
1) Interact with database legacy system:
1.1) Read the information about the required game object.
1.2) Read the information needed for initializing of the player.
2) Interact with the player:
2.1) Register a new player.
2.2) Identify the player who has just been connected to the system
3) Provide the opportunity for the players to communicate with each other
4) Store the list of the players, which are located on the map.
5) Take messages from the player addressed to the game world.
6) Send messages from the game world to the player.
7) Register the disconnections of the players and delete them from the list of the players, which are located on the map.
8) Forced as needed disconnection of the player (from the game).
9) Store the data about the game map-objects.
10) Store the data about the players.
10.1) Player’s id
10.2) Player’s login
10.3) Player’s password
10.4) Id chunk, on which the player was located before the latest quit
10.5) The data considering the inventory of the player

The client application must:
1) Provide the connect with the main server.
2) Interact with main server. (Thanks to you, Captain Obvious )
2.1) Send the request for registration in main server.
2.2) Send the request for entering the game.
2.3) Send the request about the current state of the player.
2.4) Receive the data about the current state of the player.
2.5) Receive the data which is needed for the connection to the server.
2.6) Receive messages about the current state of the object in the game map.
2.7) Receive messages from the server about the disconnections of playes

Now, I suggest to supplement this list, and after that to discuss the implementation of each item

 

[begla]

Nice list! Thank you! Could you add this list to our Wiki?

I'll focus on some optimizations I really want to remove from my to-do list the next couple of days. The multiplayer prototype will be the next step after that.

I already searched for some full fledged multiplayer frameworks written in Java, but I did not find anything I would recommend to use in the context of this project. But I'll continue the search. :!:

 

[begla]

I'm not as gung-ho on multi-level Z chunks as I was at first, it could be neat, but we'd probably be better off just making considerations for maybe supporting the addition of it in the future rather than try to actually do it up front. You could totally start with just one Z chunk and just release "SURPRISE UNDERWORLD!" one day that can be generated right below the players' feet in an existing world. Just release a new "tech" by which you can dig through the bedrock layer (which really could just be standard rock of a certain hardness you just don't have tools able to overcome yet)

Our chunks are currently twice as high as the chunks in Minecraft and it does not even influence the generation and updating performance that much (keeping in mind that chunks contain twice as many blocks and lighting calculations have to be applied to twice as many cells). The world is currently not that deep and many of the available cells are rarely used or not used at all. So it might be good to shift the ocean level at some time in the near future.

I'll put some thought in optimizing the nibble arrays even further to reduce the overall memory overhead. This might come in handy when chunks have to be transferred by the networking part.