Implementation gestalt-worldgen

[chapp007]

@Cervator and I have been discussing moving the world gen code into a separate project called gestalt-worldgen. The original thread is here: http://forum.terasology.org/threads/entity-system-code-in-engine-and-gestalt.1599/#post-13921. I started this thread so we could have a focused discussion about how to implement the move.

To be clear, the goal is to move out the idea of WorldFacet, FacetProvider, WorldRasterizer and the glue between those like WorldBuilder. Moving this out would be useful to others creating voxel games. I'm actually creating a game in jme3 right now so I would immediately benefit and verify the usefulness of such a move.

Here are some things that I have identified so far that need input from the main developers:

1. Region3i is in the engine module and is heavily used by the world gen stuff. It seems that it would fit nicely in TeraMath. The package name could stay the same.
2. WorldRasterizer needs CoreChunk to be able to call setBlock. To set a block you need Block, BlockManager and maybe other things which brings to light that we need a world-model to represent the world. They can probably all be interfaces and for example the Block interface doesn't have to have a getMeshGenerator() since it's not needed by WorldRasterizers. Any interfaces that get created for world-model could be implemented by their concrete counterparts in engine. For example, the new org.terasology.gestalt.world.model.Block interface would be implemented by the existing org.terasology.world.Block class in engine.

The good thing is that this will be a separate project and has no risk in breaking anything until we point engine to the new module. So once I have some code pushed up it will be easier to discuss what needs to be done.

 

[Cervator]

Thanks for the initiative and thread I moved it into Core Projects since that's good for stuff actually covering some specific sort of implementation.

Region3i is indeed used all over the place, but yup so long as the package name stays the same it should slide right back in. No theoretical API break there, I think.

WorldRasterizer - yeah that's more tricky. We do need some more thinking and design for the world model, maybe in two major ways:

• The different layers, like world -> chunk -> block. I take it if we abstract the layers a bit more and use those in gestalt-worldgen really they could be used for non-blocky worlds if somebody really wanted to? Yet somebody could ignore some of the layers if not needed in that specific implementation case. I wrote up a design thread with some thoughts on possible layering going far above what we have right now, it might be an interesting read. It also gets into having multiple worlds to track, which may matter less to world gen of one world
  
• The different phases. One problem right now is that there's really only one big phase: world gen consisting of the facets that result in chunks and blocks. Some work like Dynamic Cities use some background entities to simulate stuff in the background, which kinda-sorta works with the entities living in the global store without being attached to a location, but that's somewhat of a hack. It would be nice to add an actual phase where the world is merely a non-chunked map, like the world map in Dwarf Fortress. First you generate that (or a part of it in an infinite world), potentially even give the player options (starting location?), do weather simulation over time and let it trickle down to any actual chunks that have generated over time (changing biomes etc)

There are a few other world gen threads scattered about the forum. It has been an active topic over time

@msteiger would be good for feedback on Region3i and worlds in general. @Florian and @Rostyslav Zatserkovnyi too like mentioned in the other thread, as well as again @Immortius or @Josharias if available (especially on the world gen tutorial). @Cpt. Crispy Crunchy could probably just in on background simulation a la Dynamic Cities

I love the updates to the TutorialWorldGeneration module! Keep it up! Let me know if you'd like an invite to Slack sometime (or that YourKit license), or join us on IRC.

(Edit: Badged as Contributor for the great work so far already! Will add to game credits next update)

 

[chapp007]

Region3i is indeed used all over the place, but yup so long as the package name stays the same it should slide right back in. No theoretical API break there, I think.

I'm pretty sure that one is ok too since engine has a dependency on TeraMath so when someone depends on this new version their classpath will effectively be the same(Unless they depend on a separate version of TeraMath and override the transitive dependency that engine pulls in, which would be a bad idea for them to do anyway I would think)

I wrote up a design thread with some thoughts on possible layering

To me part of the elegance in the current architecture is that it's simple to understand. If Terasology tried to support other world types(non-blocky worlds), then it might be so many levels of abstraction that it would be hard to follow and maintain. Right now someone like me can come in and read the code and in a relatively short time understand what's going on. That's my initial impression but it may just be that my brain is still digesting what exists today and can't think past that just yet. Plus I need to look into Dynamic Cities and see what that's doing and maybe I'll understand the problem you are trying to solve a little better.

How about this as a plan of attack. I'll create a project in my github account that will sort of prototype what I'm thinking of moving out. That may raise even more questions from me and the community or it may clarify stuff for us. Either way, a throw away prototype is usually good in situations like this. That way we all have code to look at and talk about. While I'm playing with that hopefully that will give others time to chime in with their thoughts on how we should move forward.

I love the updates to the TutorialWorldGeneration module! Keep it up!

Thanks! I plan on working much more on the documentation. I'm kind of learning then documenting so it's a slow process for me but I'll eat the elephant one bite at a time.

Let me know if you'd like an invite to Slack sometime (or that YourKit license), or join us on IRC.

I'm currently using the Java Mission Control flight recorder to profile the app. I'm not familiar with YourKit but if there are advantages to using it over JMC I'd be happy give it a go. I've never used Slack but you can send me an invite when you have time. I'm sure it will come in handy as I become more involved. Better to get the boiler plate stuff out of the way now.

Badged as Contributor for the great work so far already! Will add to game credits next update

I don't feel like I deserve that but I'm honored. Thank you so much.

 

[Cervator]

How about this as a plan of attack. I'll create a project in my github account that will sort of prototype what I'm thinking of moving out. That may raise even more questions from me and the community or it may clarify stuff for us. Either way, a throw away prototype is usually good in situations like this. That way we all have code to look at and talk about. While I'm playing with that hopefully that will give others time to chime in with their thoughts on how we should move forward.

Yep that's fine. I don't want to over-complicate it, but figured a "chunk" really just represents, well, a chunk of world. It doesn't technically need to be representing blocks, although both ours and your jme thing do The main missing piece for me there is "sector" to better capture non-chunk/block stuff without going global to a world.

I'm currently using the Java Mission Control flight recorder to profile the app. I'm not familiar with YourKit but if there are advantages to using it over JMC I'd be happy give it a go. I've never used Slack but you can send me an invite when you have time. I'm sure it will come in handy as I become more involved. Better to get the boiler plate stuff out of the way now.

If you can get JMC working and put to use then by all means go for it! We had to whitelist some classes to get YourKit to not trigger security errors within the module sandboxing. Other than that it is just a cool shiny toy that normally costs money. Slack invite sent! Although I may not be online much longer tonight. It is like good old IRC just v2.0 and focused on teams.

 

[chapp007]

Ok gradle experts will laugh at me I'm sure but I'd rather go ahead and ask instead of wasting time on something this simple.

I'm setting up the gestalt-worldgen-proto project and I'm having trouble resolving the TeraMath dependency. This isn't a multi-module project and I just created it with the "New Project" wizard within Intellij. Here's my build.gradle.

group 'org.terasology'
version '0.1.0-SNAPSHOT'

apply plugin: 'java'

sourceCompatibility = 1.8

repositories {
    jcenter()
    mavenCentral()
}

dependencies {
    testCompile group: 'junit', name: 'junit', version: '4.11'
    compile group: 'org.terasology', name: 'TeraMath', version: '1.4.0'
}

Here's the relevant part of the error when I run "gradlew build".

Searched in the following locations:
https://jcenter.bintray.com/org/terasology/TeraMath/1.4.0/TeraMath-1.4.0.pom
https://jcenter.bintray.com/org/terasology/TeraMath/1.4.0/TeraMath-1.4.0.jar
https://repo1.maven.org/maven2/org/terasology/TeraMath/1.4.0/TeraMath-1.4.0.pom
https://repo1.maven.org/maven2/org/terasology/TeraMath/1.4.0/TeraMath-1.4.0.jar

 

[Cervator]

Sure, that one is indeed easy. We have our own Artifactory that hosts our custom libraries and such. Throw in another repo so you have something like:

repositories {
    jcenter()
    mavenCentral()
    // MovingBlocks Artifactory instance for libs not readily available elsewhere plus our own libs
    maven {
            url "http://artifactory.terasology.org/artifactory/virtual-repo-live"
    }
}

It can be helpful to look around at some of the existing projects to see how they are configured, although some of them are a bit on the complex side

 

[chapp007]

Still not working in gradle after adding that repository. I switched to use maven for dependency resolution and added the repository and it seems to be working. Since this is a prototype I don't want to get hung up on details like this since we can always switch it over to gradle later.

I'll let you know when I have something for us to look at. Might be a couple of days because I've got a lot of "day job" coding going on.

Thanks for the help!

 

[chapp007]

I put the project out there at https://github.com/jchappelle/gestalt-worldgen-proto.

Not much to see right now. In doing it I learned a lot about the real dependencies. The main one is the dependency on the entitySystem package in engine. I think the first step is going to be finishing up the entitySystem migration to gestalt. Otherwise we are going to spend a lot of time trying to maintain compatibility by creating abstractions and complexity.

Once the engine is completely dependent on the gestalt entity system then we'll be able to make the worldgen code depend on that.

I'm going to forge ahead with integrating it into my game. I'll also keep poking around in Terasology and contributing as well.

 

[Cervator]

Good to hear about the progress, and yep we can Gradle whenever. Looks like you're using Eclipse at this point?

Entity system extraction is continuing with an occasional update from @Immortius but I have no real idea when it might be finished.

Onward and upwards!

Edit/reminder: Talked to Immortius who suggested this shouldn't actually be within Gestalt (scoped to just be the very base game essentials like ES, modules, and assets, maybe also physics - and math?). So may need a new name. "Terra" is probably too generic and obvious, but still ...