Name: Per Block Data Storage
Summary: What core data do we store at a per-block level? Block ID, light values, extras
Scope: Engine
Current Goal: Refactored central data storage, including support (but not necessarily functionality) for Block dynamics and secondary block resources (oil, gas, moisture, plant growth resources...). Possibly making light storage more multi-purpose (for solid blocks). Possibly framework for mods to register new per-block data values (expensive in memory usage)
Phase: Design
Curator: Cervator
Related: #137, #139, #140
Making an issue for this in GitHub - https://github.com/MovingBlocks/Terasology/issues/139 - and leaving details and discussion here![Smile :) :)](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Current per-block arrays in Chunk.java:
* One-byte Block ID - this may fall short of capacity in the long-term, especially if Custom Blocks (special shape, possible composite texture - created by a player) need unique block IDs
* Half-byte Sunlight - lighting calculations
* Half-byte Light - like-wise (I figure, anyway!)
* Half-byte States - currently used for tracking how far liquid can flow before it stops
Begla on IRC at some point brought up an issue about how Java deals with memory storage for small stuff, and without wanting to put details here since I don't remember exactly there was something about ending up using 4 bytes anyway. So under some circumstances we might want to just use an int and do something fancy to store several things in it at once. For reference a 16x16x256 chunk with a single int per block would use 262,144 bytes of memory - which is both scary and sooo tempting. Mix in the possibility to alter the chunk size (perhaps even by the up/down axis) and who knows...
Either way, we'll likely end up changing the current setup, so thread here to discuss what we need and how to do it! My personal stance / wish list:
* Possibly using a short for block ID to not get cramped by the byte limit. Or if parsing an int, 12 bits would probably be enough (4096 values)
* Changing the state concept into a secondary "resource" value per block, exact meaning depending on context. Making an issue for this system too. Examples: Visible liquid height in liquid blocks (or partial blocks like stairs / slopes), moisture in air (weather potential), moisture in dirt (growth system / fun with aquifers), oil and gas in deep minerals (bring your mining canary). Really looking forward to this one - and it makes perfect sense as it does something in every block type.
* Unsure about this next one, but - block integrity persisted? viewtopic.php?f=4&t=171&p=977#p977 (EDIT way later: This was either this or that thread) explains the concept, but not sure yet whether the integrity values should be calculated as needed (rarely) or stored (as they could be left on disk most the time). Liquid nibble + integrity byte = mudslides![Big Grin :D :D](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Latter point brings up another part of the puzzle. Do we have two types of data - the always-loaded in memory (chunk block IDs) and some completely separate block meta-data only loaded as needed? Also keeping in mind the potential to store some stuff (creatures & concepts) as entities in an ES instead.
Summary: What core data do we store at a per-block level? Block ID, light values, extras
Scope: Engine
Current Goal: Refactored central data storage, including support (but not necessarily functionality) for Block dynamics and secondary block resources (oil, gas, moisture, plant growth resources...). Possibly making light storage more multi-purpose (for solid blocks). Possibly framework for mods to register new per-block data values (expensive in memory usage)
Phase: Design
Curator: Cervator
Related: #137, #139, #140
Making an issue for this in GitHub - https://github.com/MovingBlocks/Terasology/issues/139 - and leaving details and discussion here
Current per-block arrays in Chunk.java:
* One-byte Block ID - this may fall short of capacity in the long-term, especially if Custom Blocks (special shape, possible composite texture - created by a player) need unique block IDs
* Half-byte Sunlight - lighting calculations
* Half-byte Light - like-wise (I figure, anyway!)
* Half-byte States - currently used for tracking how far liquid can flow before it stops
Begla on IRC at some point brought up an issue about how Java deals with memory storage for small stuff, and without wanting to put details here since I don't remember exactly there was something about ending up using 4 bytes anyway. So under some circumstances we might want to just use an int and do something fancy to store several things in it at once. For reference a 16x16x256 chunk with a single int per block would use 262,144 bytes of memory - which is both scary and sooo tempting. Mix in the possibility to alter the chunk size (perhaps even by the up/down axis) and who knows...
Either way, we'll likely end up changing the current setup, so thread here to discuss what we need and how to do it! My personal stance / wish list:
* Possibly using a short for block ID to not get cramped by the byte limit. Or if parsing an int, 12 bits would probably be enough (4096 values)
* Changing the state concept into a secondary "resource" value per block, exact meaning depending on context. Making an issue for this system too. Examples: Visible liquid height in liquid blocks (or partial blocks like stairs / slopes), moisture in air (weather potential), moisture in dirt (growth system / fun with aquifers), oil and gas in deep minerals (bring your mining canary). Really looking forward to this one - and it makes perfect sense as it does something in every block type.
* Unsure about this next one, but - block integrity persisted? viewtopic.php?f=4&t=171&p=977#p977 (EDIT way later: This was either this or that thread) explains the concept, but not sure yet whether the integrity values should be calculated as needed (rarely) or stored (as they could be left on disk most the time). Liquid nibble + integrity byte = mudslides
Latter point brings up another part of the puzzle. Do we have two types of data - the always-loaded in memory (chunk block IDs) and some completely separate block meta-data only loaded as needed? Also keeping in mind the potential to store some stuff (creatures & concepts) as entities in an ES instead.