Inactive Liquid Simulation

[Cervator]

As a small addendum - better up to date example than digging through the messy old threads. This is getting dangerously close to derailing the thread so if a big non-liquid discussion takes place it is probably better to do over in the block storage thread - but liquid is a critical part of it.

I'm pondering if it would make sense to merge some of the core systems. Lighting and "block integrity" in particular were prime candidates for this in the past as solid (opaque) blocks wasted two entire nibbles of light data (in the old days at least, before the changes from Panserbjoern) that could almost perfectly be used for block integrity instead (only solid blocks provide structural support for the world)

That gets trickier when you get to something like a Greek column block shape - clearly that should provide structural support, yet as a non-full block it also is lit up. I was thinking maybe in that relatively rare case (the ratio of Greek column blocks vs natural blocks in the entire world remains pretty low) the lighting could take over and the column makes itself an entity to handle support needs (and interaction with nice handy levers that break the column at just the right time)

Other similar interactions exist, like liquid rising around said Greek column. Right now that wouldn't work as both the column block and the liquid use the primary block ID byte. So liquids might need to grow independent from the block ID - maybe we should store the liquid type in the liquid per-block data described in the top post here, or even in a singular "block network" type entity instead of block-level data in the first place.

I could go on and on about theorizing the exact combinations of block data, extra data, and entity data but ... yeah that gets too complicated, so I'll leave it at this. Maybe we need a handy little table with the different combinations

 

[UberWaffe]

IMO it's best to wait for the next push, before really digging into the code though. I'm currently implementing UpdateRegion interaction and had to change a lot on the system end.

Gotcha. I'll wait for the update before I start scratching around with the code.

And yes, specifically stating that the system is separate from any liquid implementation (i.e. it just provides the means) is probably not clear from the requirements... so:

Attempt 2 (Additions in bold, changes in bold underline):

Spoiler

The system MUST be able to:

1. Provide an API to module makers for implementing any liquids.
2. Allow simulation of individual liquid blocks to calculate liquid flow.
3. Provide a means to propagate liquid blocks based on calculated flow.
4. Allow liquids to flow upwards (i.e. like some gasses).
5. Allow liquid blocks to contains varying amounts of liquid.

The system SHOULD be able to:

1. Have no noticeable impact on engine performance, even when concurrently simulating up to 1000 blocks. (<- Utterly arbitrarily chosen number for now.)
2. Allow modules to tie into useful events relating to liquids (i.e. upon an entity entering or leaving a liquid, liquid amount changes, etc.)
3. Be able to provide additional information on a liquid type (not per block, but per type):
   • Mass of liquid per liquid amount
   • Name of liquid
   • Additional type specific data

The system COULD be able to:

1. Allow simulation of groups of liquid blocks to calculate as a whole (I.e. an entire chunk of a single liquid type interact with another entire chunk of the same liquid type as if both where just single blocks).

The system WOULD LIKE TO be able to:

1. Simulate liquids over areas without physically filling blocks with liquid blocks. (I.e. aquafiers, moisture content, flow around non-full blocks, etc.).
2. Perfectly simulate liquids and flow dynamics.
3. Prevent the social and moral decay of humanity.

Other similar interactions exist, like liquid rising around said Greek column. Right now that wouldn't work as both the column block and the liquid use the primary block ID byte. So liquids might need to grow independent from the block ID - maybe we should store the liquid type in the liquid per-block data described in the top post here, or even in a singular "block network" type entity instead of block-level data in the first place.

I could go on and on about theorizing the exact combinations of block data, extra data, and entity data but ... yeah that gets too complicated, so I'll leave it at this. Maybe we need a handy little table with the different combinations

I personally think that perhaps that level of liquid simulation (i.e. liquids sharing space with other block objects) should be an aim for a future update. Otherwise scope creep might choke version 1.0 in its crib.
See WOULD LIKE TO requirements above.

It is however, a very good point to bring up (i.e. how to store liquid related data).
I personally do not know enough about the various data storing methods, nor particularly lean towards any one of them. As long as you can store the needed data somewhere (amount of liquid in a specific block, the flow directions and flow rate needed for the calcs) then all is good.

As to what data to store:
I think Dizzy's got all the info covered in his data-bits for amount, direction and flow rate. Any other liquid related information I can think of would all be liquid-type related, and thus not stored on a per-block basis.

Also, currently there are 10 bits free in Dizzy's bit data double-word. If it is to be used for liquid types (i.e. sub-ID) then that gives 1024 liquid types per blockID used for liquids.
Maybe useful?

 

[Linus]

As to what data to store:
I think Dizzy's got all the info covered in his data-bits for amount, direction and flow rate. Any other liquid related information I can think of would all be liquid-type related, and thus not stored on a per-block basis.

Also, currently there are 10 bits free in Dizzy's bit data double-word. If it is to be used for liquid types (i.e. sub-ID) then that gives 1024 liquid types per blockID used for liquids.
Maybe useful?

Well actually, Immortius pointed me out that the extra data is stored as shorts and not as ints. So there are only 16 bits to store data in the world instead of 32. The liquid still worked as I expected it to work, because the flow magnitude is recalculated at every update anyway, so it doesn't really need to be stored in the world. So if the magnitude is left out, we actually only have 2 bits of permanent data left ATM.

 

[UberWaffe]

Ah, okay.
So then I suppose it means we will essentially use one blockID per liquid type (which might be better anyway).

Perhaps make a function to allow a modder defining a liquid to make use of the last two bits as they please.

(For instance, I want to use it to do some sort of 'extra liquid amount' for oil, or perhaps each bit for 'busy cooling' and 'busy heating up' for magma, or whatnot.)

Looking forward to the next release.

 

[Immortius]

One thing I would like to try in the future is having no blocks for liquids, and having liquids as an extra layer over the top of blocks instead. This would require the chunk mesh builder to understand liquids, but opens things up for layering liquids over other blocks (like underwater plants, stairs, etc).

 

[UberWaffe]

The system WOULD LIKE TO be able to:

1. Simulate liquids over areas without physically filling blocks with liquid blocks. (I.e. aquafiers, moisture content, flow around non-full blocks, etc.).

Covered.
For a future version anyway. That is essentially what WOULD LIKE TO requirements in MoSCoW means. Stuff for future versions.

I assume that such a layer implementation would still have data available (in some fashion) per block, so Dizzy's CA approach and logic would still be applicable, just storing/fetching data in a different fashion.

 

[Cervator]

I'm all for leaving the liquids mixing with blocks as a "WOULD LIKE TO" - I wasn't hoping for a soon-ish implementation of that part at all, but wanted to bring the topic up for discussion as it has a fundamental impact on architecture for earlier requirements / goals

I wonder if it would be possible to do away with liquid typing in anything resembling a block ID or extra data storage, instead treating the liquid system a bit more like the block network Marcin Sciesinski came up with - where an overall entity tracks the "global" stuff along with positions for the rest. If we're tracking an entire body of liquid (ignoring chunk loading issues for a moment) there really wouldn't be a need to store the liquid type more than once. Mainly we'd store the quantity said liquid network has per block.

That would help on the macro scale as well - tracking large bodies of water at a lower detail level, like an entire river (even unloaded if there's a cause to process something). And beyond allowing liquids to mix with blocks even opens the potential for overlapping liquids, like a thin layer of oil atop water

An upgraded rendering system being smart about liquids would only need to worry about the top liquid if viewed from above or the bottom liquid if viewed from below. Sideways views - maybe a little more tricky. Individual liquids don't need to care about the layering order, that can be calculated by density when needed.

Then just wait till somebody decides to set an ocean on fire after an oil spill

Edit: Quick addendum of a scenario to think about: A large still ocean. Most of it doesn't move and all is the same liquid type. Just a large volume of positions, most of which are "full" and fully immersed by identical neighbors. Maybe an octree-based block selection of some sort could capture the area covered, spending more data along the edges where the liquid data might matter more. Octrees are more problematic on land where the blocks might change a lot (especially when we seed mineral types again), but large bodies of liquid on the other hand ....

Disclaimer: Possible crazy talk by the not-so-technical guy

 

[Linus]

Hey guys, this is just a quick post to let you know why there haven't been any updates lately. The last few weeks have been crazy busy for me, because I'm trying to finish my bachelor thesis. I also just moved into a new apartment this week, so the little spare time I have left is mostly spent on unpacking the last boxes.

If all goes well, I will be able to continue working on liquids at the end of next week.

 

[Cervator]

DizzyDragon - understandable! I was indeed beginning to wonder where you were at, but your cause is a fairly regular occurrence in an all-volunteer project. Good luck with your thesis and move!

 

[Linus]

Update:The system architecture is now finally ready to support CA updates affecting multiple UpdateRegions, thanks to a new UpdateRegionManager. The actual sharing of data between regions still needs to be implemented, but with the new architecture in place, this shouldn't be too much work.

The UpdateRegionManager keeps track of which regions need to be updated. The rule for updating regions is pretty simple:
if a region has changed, update that region and all of it's neighboring regions at the next update call.

All regions are updated synchronously to prevent weird behavior at the region borders.

The code on github is now again compilable and runnable, so if you want you can try out a (buggy but playable) early alpha of water. The following blocks will trigger the CA system: CabSource, CabWater10, CaWater, CaLife.

 

[Linus]

Update:
Yay, finally, CAs can now finally spread into other regions. As a consequence, water source blocks are a lot more fun.

Changes made:

• UpdateRegions can now communicate with eachother via the UpdateRegionsManager
• Data buffers are now swapped at the correct time (after all regions have finished their update step)
• Cleaned up code and removed a lot of duplicated code by moving it to superclasses.

Bugs:

• When a CA spreads into a new region, the new region doesn't always get triggered to update.

 

[Cervator]

Awesome! It is really sweet to see DF style water working in-game, complete with it being numbered blocks, somehow that makes it more real

The CaLife might need some sort of limiter, that one went pretty nuts, haha. But everything is running hugely better than last I tested. Excellent progress!

Edit: Made a highlight build of it and will share on the social outlets tomorrow.

Btw might want to get the latest from develop sometime before you drift too far apart

Edit2: Might want to make a brief updated video sometime? That'll get more attention than a direct build download. Especially with the latest code from upstream

 

[Skaldarnar]

The build looks great. Got a bit messy with some pools and water falls
The numbered blocks have its own style, I like that somehow

A bit weird - in the world created first everything works fine (despite the region thing you already mentioned), but creating a new world and placing water blocks/source won't trigger anything (even the default CA is not working). Do you have any idea what could be the reason?

 

[Linus]

Super quick update:

• Bugs are gone
• Need to make everything work in the restructure branch

 

[Wolfgange]

I'm not sure if anything in here would be of any use, but I found this document about efficient mass water simulation. I know that most/all of the ideas discussed are too in-depth for Terasology, but could some of the boat parts be used for when/if Terasology adds boats?

And I doubt it, but would it be possible to use the current Terasology water simulation as the depth of the water for calculations, but add ripple simulation to the top layer of the water for when objects fall on/into the water (but it would not ultimately affect the amount of water stored within each water block)? The ripple calculations are also discussed in the link. If what I am explaining is unclear, I can explain in farther detail.

 

[Cervator]

I think it is more of a question if somebody will run with it and actually do it - there is a ton of good papers and ideas out there, thanks for digging up that one

Your ripple point I take it is akin to the current simulated waves (still based on what blocks are water) could also simulate objects floating on the simulated waves. Sounds kinda tricky and way down on the priority list - but again if somebody really wants to see water simulation that detailed .. go for it!

 

[Skaldarnar]

@Linus any chance you can get the Cellular Automaton up-to-date? Is what is in your liquidSystem branch your latest code? I'd like to get this back on track, especially since other games have shown that sufficient water simulation is possible within a voxel world (have a look at Seed of Andromeda, for instance).

 

[Linus]

The code in the liquidSystem branch is indeed the latest code. However, iirc, the branch only contains a bit of code that sets up OpenCL and a way to keep track on what blocks to update. I had a local backup of the working code for CA water, but two of my backup drives died last year, so it is likely that the code is lost forever.

I could send you the update rules I used for the finite liquid blocks if you want. You can just try to plug these rules into the SimpleLiquids blockUpdate function and see how well it runs.

 

[Skaldarnar]

Sure, that's the plan for SimpleLiquids I just need to figure out how to handle the updates properly (I tried it with a single queue, but it did not went that well....).

For the update rules, I've found several blogs and ideas, like

• http://www.dwarfcorp.com/site/2013/06/19/how-water-works-in-dwarfcorp/
  
• http://w-shadow.com/blog/2009/09/01/simple-fluid-simulation/
  
• http://www.gamedev.net/blog/1580/entry-2256263-2d-cellular-automation-water-simulation-source-code/

Your system did calculate updates only for a specific region, didn't it? I think this would really make sense as an extension for the SimpleLiquids module.

 

[Linus]

My liquid system did the block updates per region, instead of per block.
So instead of a queue of blocks, you would use a queue of regions.

To update a region, the required data would first be copied into a (3D) array, the input array.
Then you would just run an update kernel on each block. The kernel reads from the input array and
writes into an output array. By comparing the input and output array, you can see which blocks have changed and you can pass these changes to the (existing) batch block update function.

I would suggest to write the kernel in such a way that it only writes to its own block output data. I was able to do this by splitting the kernel into two smaller kernels, one for down flow and one for side flow, and run those in sequence.

I flooded entire worlds with this, so I guess it is efficient enough.
Also, you wouldn't have to change the algorithm if you want to move it to OpenCL at a later stage.