Tweaking Stackable Chunks

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Immortius would there be any value in merging in the vertically stacked chunks before the new world gen approach is ready, so that the code doesn't go dusty and in case it helps get some more eyes on getting existing world gens updated just for height ? Or is that what you aim for with addressing the Perlin approach? Certainly the tree setup hits challenges
We could, if we were willing to drop SecondPass generators from all existing world generators. So no trees and anything else that uses that feature. I've been assuming that we would want to maintain at least some of the quality of the world generation over the introduction of vertical chunking. Otherwise the existing generators could be used.

Edit: On a side note I find it interesting how the facets are almost like Gradle dependencies and the project tree. Right down to "figure out the configuration first then do the action" so to say. Maybe a silly observation, but it is past midnight so ..
Yeah, the same sort of issues crop up again and again don't they.
 

Marcin Sciesinski

Code ALL the Ages!
Contributor
World
Architecture
Just a thought - would it be possible to allow adding the dependencies via JSON overrides? Say Facet A depends on Facet B, but when a Facet C is introduced in a mod, the developer might wish that Facet A also depends on Facet C?
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I probably didn't explain this well enough, but:

Facets do not depend on other facets. Facets are basically (temporary) data stores, holding whatever information is produced by the providers.
Facet Providers depend on facets (and produce) Facets. This dependency information is used to order them.

So for the case you describe, what happens is you have one or more providers that create Facet A, and that those providers depend on Facet B - so Facet B is needed for Facet A's creation. The providers producing Facet B are ordered before the providers producing Facet A. A module introduces a new provider that updates Facet A and depends on Facet C, which causes the providers of Facet C to be ordered before the providers for Facet A (or at least those providers requiring C). This is handled by the Requires/Updates/Produces annotations.

None of this framework uses JSON - there are no assets involved, it is purely classes. What I haven't looked at is exactly how modules plug in the new providers - or remove providers if necessary.
 

Marcin Sciesinski

Code ALL the Ages!
Contributor
World
Architecture
I think maybe an example would help me understand.

Anyhow, maybe better would be to have it more explicit, so FacetProviders would have: @BeforeFacets(array) and @AfterFacets(array), and @ProducesFacet(array?).

I'm afraid that you won't be able to sqeeze anything between two facets, an example:
1. Using your approach - I had a FacetProvider1 saying (produces A), and FacetProvider2 saying (requires A, produces B), I will not be able to squeeze anything inbetween Facet A and Facet B.
2. Using my approach - I have a FacetProvider1 (produces A) and FacetProvider2 (produces B) saying @AfterFacets(A), then I will be able to introduce a FacetProvider3 saying (produces C), @AfterFacets(A), @BeforeFacets(B).
 

msteiger

Active Member
Contributor
World
Architecture
Logistics
I ran into similar problems while developing "Cities". The generation pipestart is triggered from the end - the chunk generator - who need to gets data from different providers in a thread-safe(!) manner.

ChunkGenerator -> Sector -> Settlements -> Buildings -> BuildingPart

To avoid creating the same data over and over again, I used Guava's LoadingCaches:
https://github.com/Terasology/Cities/blob/master/src/main/java/org/terasology/cities/common/CachingFunctionGuava.java

They are thread-safe and can be constraint by maximum size and age.
I used Guava's Function<F, T> as a basis to define input in a generic way and wrap them in CachingFunctions.

The settlement generator in Cities currently needs two things that are not yet available in general WordGenerators atm:
  • Surface Height - iterating over blocks was slow and often not possible, because not all relevant chunks were created for the entire town
  • Blocked (rectangular) areas such as lakes, but also generated structures (roads, other buildings)
It would be important that both could be either update the underlying data or give an updated version to the next layer
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Sounds essentially like the "mustRunAfter" and "mustRunBefore" options from Gradle, to fine-tune dependency order :3

Those do not define a hard set contract like outright dependencies, but the configuration phase does the best it can to tweak the order to respect those definitions

I can see how the facets and providers are a little like components and systems, giving the super flexibility we have in the ES. Adding explicit ordering like that might sacrifice a bit of flexibility (slightly higher coupling), but maybe akin to Gradle's "soft" contract it could help order things better

Also reminds me of the pre and post steps in the ES lifecycle
 

Marcin Sciesinski

Code ALL the Ages!
Contributor
World
Architecture
Cervator, the loss of flexibility is probably a bit of a choice between:
1. Minecraft model - where mods are very independent and player essentially could (though with the arrival of modpacks does not anymore) cherry pick the mods - in that case you need very abstract facet model.
2. Modpack model - where mods define only their logic and an "integration mod" gathers them together, configures and defines world generator - in that case you don't need an abstract facet model, since you will know all facets in advance when defining the world.

I'm all for the second model, and while it sacrifices the flexibility offered by first model as players can't just randomly pick a mod and add it to enrich their experience, it seems people less and less do that even in Minecraft, probably for a very good reason.

This would of course require the Terasology to change, and not allow player to choose mods to play, but instead introduce a meta-mod level artifact, that gathers these into playable piles of mods and configures them.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Oh I think that's been the plan all along :)

The player shouldn't pick modules directly, with little integration mods / game modes that deal with modules.

Although, with that impact considered for facets, would you end up better off extending / replacing FacetProvider1 to add your stuff for facet C, rather than squeezing a new facet between A and B, if you know that'll be specific to your "mod" ? Leaving the new enhanced FacetProvider1 available for others in turn (or more accurately the one module that replaces/enhances it within the set of modules that makes up your mod)

Something about that feels off, although it may just be me explaining/understanding poorly. I suspect the balance is tricky either way :)
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I think maybe an example would help me understand.
Ok, here's an example. May be a little hand-wavy, but hopefully will help.

In the initial generation module, we have:
Code:
@Produces(SurfaceHeightFacet.class)
public class PerlinSurfaceHeightProvider extends FacetProvider {
 
    public process(GeneratingRegion region) {
        SurfaceHeightFacet facet = new SurfaceHeightFacet(region.getRegion(), region.getBorderFor(SurfaceHeightFacet.class);
        region.put(SurfaceHeightFacet.class, facet);
    }
}
 
@Produces(SolidityFacet.class)
@Requires(@Facet(SurfaceHeightFacet.class))
public class SolidityProvider extends FacetProvider {
 
    public process(GeneratingRegion region) {
        SurfaceHeightFacet heightFacet = region.getFacet(SurfaceHeightFacet.class);
        SolidityFacet facet = new SolidityFacet(region.getRegion(), region.getBorderFor(SolidityFacet.class));
        region.put(SolidityFacet .class, facet);
    }
}
So this is basically an initial surface height calculation, followed by converting surface height into a 3-dimensional region of what areas are solid (which would then allow caves and underground dungeons to be carved out). Solidity provider is ordered after PerlinSurfaceHeightProvider by its requirement for SurfaceHeightFacet. Ideally what we want is if anything modifies SurfaceHeightFacet, it is slotted before SolidityProvider, even if this involves new facets. So lets say we wanted to add stone circles, that check and smooth the ground where they are. This would be done like so:

Code:
@Produces(StoneCircleFacet.class)
@Updates(@Facet(SurfaceHeightFacet.class)
public class StoneCircleProvider extends FacetProvider {
 
    public process(GeneratingRegion region) {
        SurfaceHeightFacet heightFacet = region.getFacet(SurfaceHeightFacet.class);
        StoneCircleFacet facet = new StoneCircleFacet(region.getRegion(), region.getBorderFor(StoneCircleFacet.class));
        region.put(StoneCircleFacet.class, facet);
    }
}
Updates means that the provider needs to be done after the initial production of a facet, but before it is Required by another provider. So that slots it in between the two. It can also be use the other way, to pull a new facet into process of creating a facet:

Code:
@Updates(@Facet(SurfaceHeightFacet.class)
@Requires(@Facet(SeaLevelTemperatureFacet.class))
public class ColdMountainProvider extends FacetProvider {
 
    public process(GeneratingRegion region) {
        SurfaceHeightFacet heightFacet = region.getFacet(SurfaceHeightFacet.class);
        SeaLevelTermperatureFacet temperatureFacet = region.getFacet(SeaLevelTemperatureFacet.class);
    }
}
Again, this would be run before SoldityProvider.

There is still some loose ends in this. Assuming StoneCircleProvider checks surface height as well as updates it, you would want it ordered after major surface height providers so that nothing comes along and vastly disrupts the surface height afterwards. This suggests either a simple priority system or explicit ordering instructions against specific facet providers, or both. There is also the potential that something may update two facets, where the producer of one facet required the other, and various other unresolvable edge cases (recursions).

It is also worth noting that implementation-wise providers are not really added into a linear list and run in order. When a facet is requested (by a world rasterizer or otherwise), all the facet providers needed to generate that facet are run - and only those providers. If another facet is requested, all the facet providers required to generate that new facet are run, if they haven't been already.

I'm afraid that you won't be able to sqeeze anything between two facets, an example:
1. Using your approach - I had a FacetProvider1 saying (produces A), and FacetProvider2 saying (requires A, produces B), I will not be able to squeeze anything inbetween Facet A and Facet B.
2. Using my approach - I have a FacetProvider1 (produces A) and FacetProvider2 (produces B) saying @AfterFacets(A), then I will be able to introduce a FacetProvider3 saying (produces C), @AfterFacets(A), @BeforeFacets(B).
It is unclear to me why you want to squeeze FacetProvider3/FacetC between FacetProvider1 and 2? FacetProvider2 doesn't use C, so it can't be important to have C available at that point. If you have some future FacetProvider4 that further contributes to Facet B and uses Facet C, then yes C needs to be generated before that step, but I cannot see any value in it being available explicitly before or after FacetProvider2 - only before FacetProvider4 is required. It isn't a matter of everything touching A runs before everything touching B, the ordering is looser than that. It is about having guarantees where necessary, and allowing things to be ordered more loosely where they are not.

Given this, with your approach I feel BeforeFacets is unnecessary. No facet provider needs to run explicitly before a facet is generated, because if it does it cannot be using it - in which case it doesn't matter whether it runs before or after that facet is generated. It is more the case facet providers need to run after a facet is generated, because they require that facet to do their work.

Cervator, the loss of flexibility is probably a bit of a choice between:
1. Minecraft model - where mods are very independent and player essentially could (though with the arrival of modpacks does not anymore) cherry pick the mods - in that case you need very abstract facet model.
2. Modpack model - where mods define only their logic and an "integration mod" gathers them together, configures and defines world generator - in that case you don't need an abstract facet model, since you will know all facets in advance when defining the world.

I'm all for the second model, and while it sacrifices the flexibility offered by first model as players can't just randomly pick a mod and add it to enrich their experience, it seems people less and less do that even in Minecraft, probably for a very good reason.

This would of course require the Terasology to change, and not allow player to choose mods to play, but instead introduce a meta-mod level artifact, that gathers these into playable piles of mods and configures them.
I guess I'm thinking something in between? In the future, I envision Terasology will have GameTypes and Mods, where mods apply to specific gametypes or gametypes that meet some general requirements (usage of specific modules). Both of these things build on top of modules. A GameType will likely either have a fixed world generator or impose some requirements on the world generator - so this will guarantee some Facets will be in use. But at the same time, I would like to see Mods able to new features on top of this, or replace how existing Facets are generated and rasterized. Players will not be able to cherry pick individual modules when this is set up.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I ran into similar problems while developing "Cities". The generation pipestart is triggered from the end - the chunk generator - who need to gets data from different providers in a thread-safe(!) manner.

ChunkGenerator -> Sector -> Settlements -> Buildings -> BuildingPart

To avoid creating the same data over and over again, I used Guava's LoadingCaches:
https://github.com/Terasology/Cities/blob/master/src/main/java/org/terasology/cities/common/CachingFunctionGuava.java
This is interesting, although I don't think it would have solved the performance issues. Perlin noise usage by the perlin generator uses a subsampling technique, so a grid of values is read and then interpolated to fill the rest. This means it is much more performant to be able to request a region than individual values. Additionally, if different providers need slightly different regions (same region but slightly larger), that would still result in the same data being generated multiple times - when all you really need to do is generate the largest region. But I'll keep this in mind for future use. :)

The settlement generator in Cities currently needs two things that are not yet available in general WordGenerators atm:
  • Surface Height - iterating over blocks was slow and often not possible, because not all relevant chunks were created for the entire town
  • Blocked (rectangular) areas such as lakes, but also generated structures (roads, other buildings)
It would be important that both could be either update the underlying data or give an updated version to the next layer
I'm heavily interesting in having surface height available, to help drive spawn locations and sunlight calculations.
I don't think lakes are generally rectangular, but it sounds like you are talking about "feature" facets, which would be that sort of higher-level structural abstraction (collection of bounding areas).
 
Top