Per-block data storage

Mooncalf

New Member
Contributor
Design
Well, I have little to no idea about the technical application, but I would like to suggest a form of damage tracking for individual blocks.

Eventually the game will end up with some sort of collateral damage to the environment (i.e. EXPLOSIONS!) or more directed violence against blocks, which prompts the problem of how we keep track of how damaged a block is and how resistant a block is against a source of damage. To give an example: Throwing a stick of dynamite at a dirtwall will produce a hole, throwing it at a steel wall will only produce a smudge.

Now I see two ways of doing this:

1.) We track each blocks individual "hit points" which depend on the block type.

I figure this is memory intensive and in most cases redundant, as most blocks will be at perfect health most of the time, but it allows for smooth tracking.

2.) We assign a "hardness" value to each block type. Now everytime the block is under attack the damage is compared to the "hardness" of the block type: If the hardness is overcome, the block is exchanged for its "damaged" counterpart otherwise the block perseveres undaunted. We can then continue this chain (also by bleeding over excessive damage) until there are no more damaged variants available at which point the block simply vanishes and joins his fallen brethren in Blockhalla.

This frees us from assigning individual memory space to each block in the world, but makes the damage process a somewhat rougher "all-or-nothing" deal and we will need to use multiple slots for one block type and its damaged variants (unless there is a better solution for storing the damaged versions); however, most blocks won't actually need to have a damaged variant (like sand or dirt), yet the ability to have them will allow us to do some additional shenanigans, like having a damaged steam pipe fog up the environment, or weakened wooden floors that collapse under the weight of a creature.

If we would go with solution number 2 we could also implement a way to temporarely track smaller damage increments to a block that fail to individually overcome a blocks hardness, but combined can damage it; effectively when a block is attacked it generates an entry that tracks the total damage of all attacks against it that stays with it until either the total damage overcomes the hardness of the block, the block is removed, the block is unloaded from memory (typically by having the player leave the area) or when a certain amount of time has passed.

I'm thinking here of situations like felling a tree with an axe or shooting a stone pillar with a machine gun.

Of course the hardness idea can also be combined with the first variant.


Regarding water/liquids:

Is it technically feasible to treat bodies of liquids as individual enitities? By which I mean that instead of putting a bunch of individual water type blocks in the same area and calling it a lake we actually have a lake entity that determines how much blocks around it are filled with water and to what height?

To elaborate:

First we determine if an area can hold a liquid (e.g. a hole in the ground).

Now we generate a "liquid" entity associated with a central empty block at the lowest z-level of the hole.

This entity contains all the necessary information required for the liquid, like type of content or volume.

From this block onward we determine how many empty blocks are in "reach" on this z-level. All these blocks are stored within the liquid entity as the first z-layer.

The total amounts of blocks in this layer determine the total amount of volume this entity can store.

Now as long as there is water left in the entity, all blocks of this layer will be filled with water.

If water is added in excess of this capacity, the water level rises, at which point the game will check if the next z-layer above the current filled one can hold a liquid. If "no" all the excess water is lost (assumed to have drained into the environment), if "yes" the next layer is added to the entity.

Should this higher layer connect to more empty blocks in lower z-layers, these are added to the lower layers in the entity, creating new ones as necessary (it would probably be prudent to number the z-layers in relation to the worlds z-layers and not its own starting layer to prevent negative values).

This new layer now has its own volume, which is added to the total volume of the entity.

Now whenever the total volume of the entity is filled to a certain threshold (like 50%) of the volume of the next unfilled layer, this layer is also filled with water blocks and these blocks are removed again, when the water level falls below that threshold.

Should a block within this liquid body be filled with a solid block, the block is simply removed from the list of the entity and the volumes adjusted accordingly. If this was the block associated with the entity, simply associate the entity with the next viable block in the liquid body; should no viable block remain, the entity is removed.

Everytime a solid block is removed that is in vicinity of the liquid body at the same z-layer, the layer is recalculated. Should this cause a layer to be open to an area that cannot contain a liquid (i.e. the space would be too vast), it will start to drain out at a rate determined by the amount of open spaces it can drain out of (might even do water pressure calculations for that) unless the leak is sealed again. If a layer is completely drained in this manner, it is removed from the entity. If it was the last layer, the entity is removed.

The entities can have additional values, like how fast they regenerate or lose their supply, or how much surface area they have (for determining gain through rainfall) and also if they are connected to another entity (like a river or a pump).

This system would allow us to control bodies of water on a volume basis, without doing constant recalculations and individual tracking of blocks.

We can also fracture the z-layers into smaller sections to allow "smoother" filling of blocks (like DF).

Also we can use similar systems for rivers and oceans (with oceans being infinite and limited in layers).
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Good news from Blockhalla everyone!

We've already got a great per-block damage system, and it is independent of per-block data storage :)

The Entity System Immortius prepared creates a single entity per damaged block that stores the exact damage and does other per-block magic. That way we only have one entity per block "activated" in some fashion, which gets us the best of both worlds: Per block (that matters) data storage + efficient memory usage. Block types already store hardness and such which can be used in the calculations.

On liquids - my ever-mentioned-but-not-implemented-yet Block Dynamics system would cover that using per-block data storage, but done in a multi-purpose fashion that shares said data with other features. Threads here and here, but also refined further elsewhere. On treating a body of liquid as its own entity, that's what I used to call a meta-block and we need a CompositeBlockComponent to really crack into the multi-block field to see what we can do. I like the idea of tracking larger objects like that for other reasons as well, like defining regions etc.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Yeah, at the moment when you want to damage a block you request its entity - if one doesn't exist a temporary entity is created. This entity can be damaged or otherwise interacted with. When the block returns to full health through regeneration, or if it is destroyed, the entity is removed (unless it has other data like a chest). The stats of the entity (amount of health, regeneration rate, etc) come from the block definition.

On liquids - I feel their low level representation needs to be handled as currently, due to their prominence - that is, as arrays of data at the chunk level. That doesn't preclude some lower granularity entities and systems that influence or control the liquids though.

I'm actually thinking of trying handling liquids as non-blocks, but with data still stored at the chunk level. That is, each block will have a liquid type (water or lava) and depth (0-7) which together uses half a byte. Advantage is that this can exist along side the blocks, so you can have seaweed or other plants under water, or have water flooding stairs and other partial blocks.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Plants under water - that's so oddly revolutionary I wonder why I never even thought of that :rofl:

I'm sure there's a way to fold that together with my desire to have stuff like a secondary resource counter (liquid), integrity/pressure, etc. Several other "liquid" potentials as well, like oil, tar, fine dust from mining, even unhardened cement. To minimize memory usage - do we need both light nibbles inside a liquid or can we sacrifice one on the altar of liquid typing?
 

Mooncalf

New Member
Contributor
Design
Nice to see that I'm completely lagging behind the current technological development of the game :D.

How would the current damage system fare when presented with massed demand (e.g. firing wildly into the area with 1200 rounds/minute)? Are there any limits to it?

For the liquids the main point for me is the ability to handle volumes of liquids as a resource that we can balance around without having to consider individual blocks or causing constant recalculations of water block placement.

Namely if I start to pump water off into a reservoir I don't want the pump running dry because the closest water blocks have been used up and you need to wait for neighbouring blocks to replace them and the thing will also constantly hog system ressources even when the player goes off site, unless we extrapolate off site calculations based on time, which leads to the discrepancy between what a pump can theoretically do and what it does in practice while observed by the player.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Given the current block regeneration rates you would be looking at maybe 60 active block entities at a time (generating 20 damaged blocks a second, regeneration from a small amount of damage takes maybe 2 seconds), which is a piddling amount.

As for liquids... I couldn't say with out a bunch of experimentation and prototyping. The main problem I see is rectifying the macro water entities with block changes, how to deal with water moving off of the active world, and how to store this efficiently.

Cellular systems on the other hand are more attractive to me, because they are much simpler to represent and process, and with the right rules you can produce complex macro-level results. Perhaps augmenting the liquid data with some flow information might produce the same results - when you drain water from an area you propagate flow back through the body until you reach the origin, and you start removing water from there instead?

At any rate for now I'm just doing a fairly basic implementation that I know will work - infinite water sources and all that. Complex water is a much lower priority than a lot of other things.

Plants under water - that's so oddly revolutionary I wonder why I never even thought of that :rofl:

I'm sure there's a way to fold that together with my desire to have stuff like a secondary resource counter (liquid), integrity/pressure, etc. Several other "liquid" potentials as well, like oil, tar, fine dust from mining, even unhardened cement. To minimize memory usage - do we need both light nibbles inside a liquid or can we sacrifice one on the altar of liquid typing?
Well, if we extended the data to a byte, that would give you 32 content types, and a depth of 0-7 And we could squeeze a little more out by recognizing that we don't need to have multiple content types with a depth of 0. Or by having types with less depth range.

Not sure about sacrificing light nibbles, if you need one you need both, and light goes into water at least.
 

dei

Java Priest
Contributor
Design
Art
Already speaking of steampunk everywhere we definitely need one more: Steam! (needs to be combinable with plants etc too)
Water flowing on lava/fire could produce steam, certain underwater plants some burnable gaz etc...

Cervator: +1 for the underwater plants. Magic-Faction could have water- and later lava-turbines, Steampunk-Faction could use steam- and later oil-turbines.
Plants/Objects in Lava, Cement and (Oil) don't make much sense so these could be normal blocks with fluid pressure on them. Counting Steam, eventually Oil we would have two liquids apart from water. For water I think we would need as many bits as possible (oceans and lakes can be much deeper than oil and gaz reserves). So I would use 7 bits for water pressure (first bit is indicator if its water, = 128 Pressure levels) and the rest of bits distributed between liquid-types and their pressures.

This way we could implement a nearly realistic water-ecosystem: When rain falls every surface-block fills up with water a certain amount per time. Then this water naturally (block-based DF-water simulation linked in another thread) forms rivers, streams and lakes until it reaches an ocean, which is defined as a constant-level sink/source. Each block-type could define a water-permeability so water can seep into the ground and water-sources (not infinite) form naturally (eg. where earth changes to impermeable stone in a hill). That would form dipping ceilings and even dripstone someday etc.
Apart from that plants could be dependent on and drain water-level in the ground, so deserts can't be overgrown by plants (if there is no underground river or pipe).

It would need much balancing but I think it would be worth the effort.

Pipes with Turbines would give the game a reeeaaly huge economic factor (hydroelectric powerplants, oil-pipelines...).
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Lots of good stuff. We also have the growth simulator woodspeople did to integrate and use to soak up some of that good water that sinks into the ground :)

Interesting angle to use one bit to allow for one "magic" liquid then a bunch of secondary liquids with fewer special features.

Steam could be present in pipes, yes, but I wonder if it would ever be anything but an effect? You wouldn't have "pools" of steam in the world, nor would it be held in dirt as a secondary resource, probably would just be a graphic effect if it gets loose and otherwise a fairly simple representation of it in pipes? Other types of gasses might exist underground, but not be an obvious thing to put in pipes... hmm.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Cervator: +1 for the underwater plants. Magic-Faction could have water- and later lava-turbines, Steampunk-Faction could use steam- and later oil-turbines.
Plants/Objects in Lava, Cement and (Oil) don't make much sense so these could be normal blocks with fluid pressure on them. Counting Steam, eventually Oil we would have two liquids apart from water. For water I think we would need as many bits as possible (oceans and lakes can be much deeper than oil and gaz reserves). So I would use 7 bits for water pressure (first bit is indicator if its water, = 128 Pressure levels) and the rest of bits distributed between liquid-types and their pressures.
When I mention "depth" I mean within a block. You don't really need 128 different depths within a block.
 

dei

Java Priest
Contributor
Design
Art
When I mention "depth" I mean within a block. You don't really need 128 different depths within a block.
Yes you need! Different depths could also be named pressure (with the top 5 or so used to fill a block only partially by the liquid would be an option). If you look at a U-Formed tunnel that is 100 blocks deep and fill one side with water you need to ensure that on the other side the water level is at the same height in a balanced state as described by http://w-shadow.com/blog/2009/09/01/simple-fluid-simulation/. Else you would have to handle water as an entity which has conciousness of its level, mass etc, which causes many problems when water is splitted etc.

Or did I understand you wrong with the depth-idea?
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Ah I see. What you are talking about is a separate thing, although it has merit. What I'm thinking with depth is merely how full a block is, and would enable something like Minecraft's sloped water. Pressure falls into the more complex liquids bucket I'm not really thinking about at the moment.
 

overdhose

Active Member
Contributor
Design
World
GUI
ooh I can't wait to actually create some rice fields in Terasology now :D

Magic-Faction could have water- and later lava-turbines
wouldn't they rather get it from a different source, like plants and trees for example. Don't use 2 resources for everything.
 

Nym Traveel

Active Member
Contributor
Art
World
Omg, just read through this whole thread - my head spins :/

Quite interesting ideas here, can somebody bring me up to date: who's implementing what idea right now? (especially interested in the water-stuff ;) )
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I'm currently doing some work on the block system - this will include the ability to have extra per-block-type data.
 

4D enthusiast

New Member
Contributor
This thread was dead for years before I worked on this, but just in case anyone reads it: this feature has been implemented. For details, see o.t.world.chunks.blockdata.ExtraBlockDataManager, and for examples of how it's used, see biomes or FlowingLiquids.
 
Last edited:
Top