Discussion: Custom Blocks

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
I was thinking about putting an item in the issue tracker for this, but realized it is too early and I don't have enough of an idea of what shape it will take to really put any sort of specification together on it. So here's a forum thread instead for the basics.

To get some quick concepts out there:

* A Custom Block is the idea of a single block that is made up by sub-components (like a fancy stair block)
* Currently all blocks in the world are stored as simply Byte values to disk, so we have a limit of basic blocks of 256 (air at 0 is the only hard coded one, I'm thinking Custom Blocks could be id 255) - that won't be enough to cover all permutations of custom blocks possible for a player to put together
* The blueprint concept is a whole block selection that can be used in various formats (like the copy paste terrain tool, eventually workshops/rooms)
* The schematics concept is a sub-block construct that makes up one whole block, but is made by tiny versions of normal blocks.
* A blueprint of a certain size (say 2x2x2) can be turned into a schematic by shrinking it, which can then create a custom block in the world
* A blueprint can contain schematics / custom blocks

A few big questions:

* How do we render blocks that aren't boring squares with whole textures for each side? Some blocks can cheat with transparency in the texture (like cacti technically look smaller than a whole block but actually are whole blocks with a small invisible part)
* Do we create some advanced blocks separately from custom blocks? A good example might be a door, which has a non-standard bounding box, a state (open / closed), the ability to be made up by multiple blocks (1x1x2 standard door, 2x1x2 double door, bigger doors) - doors are clearly meta blocks, but are they custom blocks so players can easily create their own?
* How do we decide between custom blocks dynamically created by players (like a fancy staircase) and non-square standard blocks (like doors or some furniture). You could approximate the shape of a chair with a custom block, but how would such a block compare with an actual pre-defined chair? Texturing might also be tricky between the two if you aren't using boring blocks with the same texture on each side, likewise orientation.

A lot of this is still rough in my head and likely confusing - that's what discussion is there for helping clarify :)
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Let me describe what I currently do in NightBlock. It isn't perfect, but it may give you some ideas.

In NightBlock a block is composed of several elements:

BlockShape

This determines the shape of the block. These is done as a composite of 7 sub meshes - one for each side, and a core mesh. Each of the sides is also marked as being a full side if it fills the entire side, or incomplete. When the chunk mesh is being generated, the core mesh for each block is always included but sides are only included if they are not adjacent a block with a full side blocking it. So the chunk mesh is stitched together out of the relevant mesh parts of each block.

I built the block shapes in blender and then wrote a script (well, modified an existing one) to export it as a simple ascii format.

Alignment

If a block is symetrical on every axis this isn't needed, but something like stairs can be rotated. In NightBlock alignment formed part of the block definition so stairs facing different directions were different blocks (with different indexes).

Material

I haven't developed this fully in NightBlock, but the final part of a block definition is the material used to display it - the texture tiles (if using a texture atlas) or texture indices (if using a texture array) used by each sub mesh. Having material separate allows having the same shape with different materials - wooden, stone, and dirt stairs for instance. At the moment I only allow a single tile for a block, and modify the UVs of the block mesh to correct tile. With a texture array I'ld add an index to each vertex describing which texture to use. Don't know which system you use at the moment.

Block Definition

The final stage of defining blocks is then to add combinations of shapes, alignments and materials to the block table. The various block information is rotated to conform with the provide alignment, uvs altered and everything set up ready for efficient chunk generation. Blocks can later be found by searching using combinations of the three elements.

Block Controllers

The way I approached multi-block constructs in NightBlock was at a higher level than the world - the world just knew about the existence of door-top and door-bottom block, but they had no special significance. The game logic handling the placement of doors and interactions with them knows to treat them as a group. In NightBlock I use an entity/component system to handle game logic, so would add an entity with a BlockController component linking it to the set of blocks defining a door, and a Door Behaviour component. Then when the player interacts with any of the blocks in the door, the entity handles the interactions and applies the correct changes to the involved blocks. This is also useful for single blocks that have advanced, stateful behaviour like furnaces.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Sounds great, actually. I'll have to defer to begla on some of the fancy 3D stuff like meshes, but it sounds about right, and we do need to deal with shapes next :)

Your Block Controller explanation makes me happy as that's about exactly what I meant to achieve with Meta Blocks, which makes me think I'm not totally insane :D

What I can say about our system is the piece I worked on - dynamically defining blocks as far as material and texture go, which with the new Groovy-based BlockManifestor is loaded block by block and texture by texture out of tiny definition & png files. That's to make it super easy to extend the game as there are no hard coded IDs outside of Air (empty) so far, and only ever whole directories are loaded, so there's no need to even spell out what files to load.

The same definition files can also very easily hold other config for blocks of the relevant types, including meta-block stuff like behavior for a tree (growth & reaction to being chopped at). But I keep thinking the exact definition of what's what (Block vs MetaBlock vs possibly Entity) may still need a little work, so this helps quite a bit!

Could you detail your use of Entity and Behavior objects a little more. as with the door example? I know we have an Entity object in code, but so far I hadn't thought about using it with a MetaBlock construct like a door - I was planning to have a collection of sorts for each MetaBlock type to hold all those specific objects, maybe that would end up serving the same purpose. And there's a BlockGrid or grouping like that in there somewhere.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Cervator said:
Could you detail your use of Entity and Behavior objects a little more. as with the door example? I know we have an Entity object in code, but so far I hadn't thought about using it with a MetaBlock construct like a door - I was planning to have a collection of sorts for each MetaBlock type to hold all those specific objects, maybe that would end up serving the same purpose. And there's a BlockGrid or grouping like that in there somewhere.
Sure, but it is a little complex :). Entity in my case is extremely different from the existing entity class in Terasology, where it is simply the base class for anything that exists in the world. With NightBlock I was experimenting with using an Entity System, which is an alternative approach to using a class hierarchy for game logic. Entity Systems focus on flexibility and reuse through the use of composition and configuration. They have four main building blocks

Entity

An entity is simply an identifier. In NightBlock it was a long, but it could be a GUID or hash or whatever. I also had an Entity class, which wrapped the identifier and provided helper functions, but fundamentally an entity's only purpose is to be a unique identifier around which a collection of components are grouped.

Component

A component is a set of data describing an aspect of an entity. For instance, a common component in NightBlock is Location, that describes the position and rotation of the entity. Some components I had were Mesh, Light, Item, Weapon, Particle, PlayerInput, Character and Collision. Components are pure data - they don't have any game logic. They should also not inherit each other - if you need data from two components you just put both components on an entity.

Entity Manager

This is what ties components to entities, manages their lifecycle and handles queries like fetching specific components for entities or iterating over all components of a type. A simple manager can be done just using a collection of hash maps and other simple collections, but it is also possible to build the manager around a database - a standard rdbms or something shiny like OrientDB.

There is a decision to be made about whether entities can have multiple of the same component or not - I decided not to allow it as having two locations doesn't matter, and if havng multiple of something does make sense the component's content can reflect that. This restriction also simplifies managing entities.

Systems

With all this data clumped around entities, some Systems are needed to make use of it all. I had a few specialised systems I'll get into later, but the primary pattern is to have a number of System classes each of which finds all the entities with a desired set of components and processes them. A Mesh Renderer system would find each entity with a Location and Mesh component and render them, a Physics system would find each entity with a Location and Physics component and calculate a physics update, a Player system would process input for each frame and update the Character component on any entity with a Player component.

I enhanced this concept with the ability for systems to find newly added or pending removal components to handle those changes, which was important because I was working on top of a more traditional engine so had to update the underlying scene graph. Better yet would have been to track changed components so that they could be searched for explicitly.

Entity Prefabs

Entity prefabs are very similar to entities - they are a collection of components connected to an identifier. Unlike entities they are never processed by a system. Instead they are used as the template for spawning new entities. Components on entities can reference entity prefabs - for instance, a weapon component may refer to a prefab to spawn when it is fired. This might be an arrow for a bow, or a bullet for a gun, or a cat for a catapult.

Event handling

This isn't touched on by many of the articles on Entity Systems, but I came up with what I think is a reasonably good solution. I have a special type of System for handling events - these are registered against a type of component. When an event is "sent" to an entity, the components connected to it are iterated and the event handler for each component type is triggered. For instance, when an item is used, the "Used" event is "sent" to the entity. For each component that entity has, the appropriate handler is sent the event and the entity. If that item has a Heal component and a Teleport component then the event handler for each is triggered and it may heal the player's health and teleport them to a safe location.

I'm still experimenting with how this is all best used. One thing I'm messing with is having components with a list of actions they will take when an event occurs - such as playing a sound, spawning an entity prefab, or removing a component. With the right set up it should be able to support easily designing potions and magic items with different abilities, or applying traps and locks to doors, chests or other usables.

So how does this apply to blocks

Well, the only special block support I handle so far are doors. A door entity has a BlockController component that has a list of blocks positions it controls, and a Door component that has all the information required for how to tranform the door from its close to open state and back, and the list of actions to take when opened or closed (i.e. play a sound).

The BlockControllerSystem updates the world to know that the set of blocks listed in a BlockController is handled by the entity. If the player interacts with any of those blocks, a Use event is sent to the entity.

When the door event handler receives a Use event it reacts. When opened the door blocks are removed and an open door mesh added to the door entity. When closed the door mesh is removed and the blocks being re-added. This is because my door sits in the center and needs to hang across block boundries when open - in Minecraft doors sit against a side so only need to be replaced with blocks rotated 90 degrees.

Benefits and Costs

What I see as the potential benefits of an Entity System are a lot of flexibility in defining objects via configuration - by specifying what components an entity has, and what settings those have. Also it is simple to slip in new compnents and systems to get new behaviours.

The cost - well, it is a non-OO approach so you lose some of the benefits from that. The data in the components is not encapsulated, and you cannot react immediately to changes to the data. It also is quite a different mindset - instead of building, say, a monster that fires explosive frogs, you instead build the components and systems needed to support it.

Further reading

T-Machine has written a series of articles on Entity Systems. There is also a wiki on the subject, although it is a little bare.

Sorry this turned out very wordy, I probably should throw some diagrams in.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Good stuff! An interesting read. I'll have to re-read it again later when I have more time and access to look at our code to brainstorm. Thanks! :)
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
I've been chewing through parts 1-5 on Entity Systems (is there a 6 hiding somewhere?) and it sure is a meaty topic. In a small way it seems familiar to what I was wanting to do with MetaBlocks - assigning a meta-ability to blocks in Groovy config ending up at a sort of multi-inheritance, but with controller classes instead each owning and iterating over a data set of all blocks registered to that meta-ability.

I'm happy to skip 5 years of refinement on that though and just look more closely at an existing pattern/paradigm if it is relevant as is :)

It seems very eager to be heavily married to relational DBs - what do you think about that? I was thinking about what potential the object-oriented DBs would hold, but admit I might already have broken that by playing with meta-object concepts

Also an interesting thing to think about in regards to multiplayer / networking, as event-driven frameworks like Node.js seem to be picking up steam. I'm not that interested in Node, but Red Dwarf Server seems to be using a similar setup in pure Java explicitly made for MMOG type setups (just never completely finished, alas). Not sure how all that interacts and what potential is there, needs more thinking!

Neat to see references in the wider blog there to Mythruna on jME3 (been aware of that since it started), EVE (woo spreadsheets in space!), and DevOps, all topics interesting to me. Thanks for the links :D

I'm going to check out the wiki and links from there another time, although yeah, it doesn't look like there's all that much in the wiki, wonder if the author's fascination with the topic petered out? Anyway, I'm sure there's lots more out there elsewhere.

I'd be happy to see more ideas on how to apply this to Terasology. And catapults firing cats :geek:

Going to also leave this here for myself just as a reminder: BlockSelection
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Cervator said:
It seems very eager to be heavily married to relational DBs - what do you think about that? I was thinking about what potential the object-oriented DBs would hold, but admit I might already have broken that by playing with meta-object concepts
Well, I had NightBlock running at one point on OrientDB acting as a POJO/Object database (it supports graph and document modes as well). I would pull the Components out of the database straight as objects. This was working alright, performance would probably be better using OrientDB in document or graph mode. Certainly this seems aimed at when you have a lot more data, or looking to support clustering for scaling out. It does have the benefit that you can use queries to get things like recently changed components or components with specific values. And of course the database can be embedded into the application.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
I've got Entity System research coming out my ears now - lots of material :D

Still need to think about it in more technical detail vs. the codebase. Have you seen the Artemis framework and developed an opinion of that?

Certainly seems like it has potential (both concept and maybe Artemis), although as that primary author predicts my OOP-side is making me twitch here and there.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Okay - more reading and thinking and I'm getting a better feel for how entity systems compare to other stuff, along with some of the risks involved.

What makes me happy is that we could still totally drive the config out of Groovy with about the same setup. Where I would "register" a Block to one or more MetaBlock types in addition to its "native" Block config just instead becomes chunks each corresponding to different components, including Block itself, "Door", "Tree", "Portal" etc. The Groovy Manifestors still load stuff, they just store the details differently.

Which is kind of cool because suddenly you can visualize defining creatures and other things the exact same way, with some reused components and some unique. On the other hand you could also end up with a lot of very similar components, or some components that aren't reused anywhere else anyway. Although you could always leave some things together in one Component and split it later.

Block would become a Component (with some stuff split out); Plant, Tree and Liquid possibly others (matching the Block sub-classes that are distinct enough), but the differences are so minor it makes my OOP-side twitch again. Also makes me question how re-usable they'd be.

Individual Groovy files would be defining prefabs / templates - PortalBlock.groovy for instance would register a prefab entity that has the components Block, Location (probably? Although currently we track location elsewhere), (Renderable)Mesh (split out from Block), BlockSelection (or BlockController, but does that really control anything?), and Portal (which is the one thing I'd add under the MetaBlock design)

PortalManager would be an example of a System, controlling the logic behind Portals. While I would simply add a collection of Portals there, with a pure entity system you'd store them all in a giant pile somewhere, like in the Artemis framework class responsible for that, and simply ask for all entities with the appropriate Component makeup when you wanted to iterate over Portals (which you could do with the timer-based entity system in the framework, every x ms for triggering GelCube spawning). OOP tenses tingling again, wanting to separate out object types in collections, but then sure you've got all Portals in one place, but now you have to go check multiple collections to get all Renderables or duplicate some multi-component objects in storage...

When creating an actual Portal you'd generate an entity ID, associate it with the appropriate components (which would be defined in the prefab), have it added to the entity store / manager... and you're done.

It would be a big balancing act to get it right, that I can see. Most the criticism I found was related to over-architecting or getting stuck in the minutiae and I can easily imagine that. Part reason I did the Groovy prop files for Blocks was being able to super easily define lots of properties in a hierarchical approach that could then be stored in an OOP tree starting from Block. If you were to do that with ES you could, but the question is how tiny Components you'd end up with.

In the Groovy definition approach to creatures you could eventually include very detailed anatomy config, which in the OOP approach + Manifestor setup could be fairly easily defined and interacted with down to the fingernail level (like in DF) - but I wonder how'd you do that in ES, make Components right down to "Finger" and assemble those into creatures? I suppose you could.

Okay, I'm pretty sure I'm ranting now and getting too technically deep into the current code base and even design that's still only stuck in my head and nowhere else... :geek:

(and none of this actually relates directly to the idea of Custom Blocks, but that's then where you just add that BlockShape / custom Mesh thing..)

Edit: One particularly nice article for an intro to the concepts that can help make stuff click is http://gameprogrammingpatterns.com/component.html

Edit2: Ooohh, I could get my OOP fix in still by using that for the templates / prefabs - as long as it doesn't touch the running code, it isn't cheating then to still call it ES, is it? :D
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Cervator said:
Still need to think about it in more technical detail vs. the codebase. Have you seen the Artemis framework and developed an opinion of that?

Certainly seems like it has potential (both concept and maybe Artemis), although as that primary author predicts my OOP-side is making me twitch here and there.
I had a brief look at artemis, and decided there would be more value in writing my own - primarily in gaining a good understanding of how entity systems work. If I was to start over again I may well use Artemis, although I'ld more likely build something on top of OrientDB.

Cervator said:
Okay - more reading and thinking and I'm getting a better feel for how entity systems compare to other stuff, along with some of the risks involved.

What makes me happy is that we could still totally drive the config out of Groovy with about the same setup. Where I would "register" a Block to one or more MetaBlock types in addition to its "native" Block config just instead becomes chunks each corresponding to different components, including Block itself, "Door", "Tree", "Portal" etc. The Groovy Manifestors still load stuff, they just store the details differently.

Which is kind of cool because suddenly you can visualize defining creatures and other things the exact same way, with some reused components and some unique. On the other hand you could also end up with a lot of very similar components, or some components that aren't reused anywhere else anyway. Although you could always leave some things together in one Component and split it later.

Block would become a Component (with some stuff split out); Plant, Tree and Liquid possibly others (matching the Block sub-classes that are distinct enough), but the differences are so minor it makes my OOP-side twitch again. Also makes me question how re-usable they'd be.
Well, one question to ask here is do Plant/Tree/Liquid have data? If not, why not make them properties of Block rather than Components unto themselves. Or perhaps, do you just have a single "Growth" component with data describing how the block expands outwards. The settings may differ between plants, trees and liquid, but the common element to them is their growth.

For NightBlock my inclination was to keep individual blocks outside of the Entity System completely, as they can be stored and worked with much more efficiently as a unified group. In particular, render chunk generation is quite specialised and distinct from rendering any old mesh. I hadn't done any dynamic blocks types though, and can see some benefit to having an entity for each individual tree.

Individual Groovy files would be defining prefabs / templates - PortalBlock.groovy for instance would register a prefab entity that has the components Block, Location (probably? Although currently we track location elsewhere), (Renderable)Mesh (split out from Block), BlockSelection (or BlockController, but does that really control anything?), and Portal (which is the one thing I'd add under the MetaBlock design)

PortalManager would be an example of a System, controlling the logic behind Portals. While I would simply add a collection of Portals there, with a pure entity system you'd store them all in a giant pile somewhere, like in the Artemis framework class responsible for that, and simply ask for all entities with the appropriate Component makeup when you wanted to iterate over Portals (which you could do with the timer-based entity system in the framework, every x ms for triggering GelCube spawning). OOP tenses tingling again, wanting to separate out object types in collections, but then sure you've got all Portals in one place, but now you have to go check multiple collections to get all Renderables or duplicate some multi-component objects in storage...

When creating an actual Portal you'd generate an entity ID, associate it with the appropriate components (which would be defined in the prefab), have it added to the entity store / manager... and you're done.
Yes, I would agree with most of that. I'm not sure how Artemis does its internal storage, but mine was Map<Class<? extends Component, Map<EntityId, Component>>. This means that you could easily iterate over all entities with a given component type associated with it, or get a specific component type for a specific entity. Remember that entities are just an identifier, not a container. Components are only attached to each other by association.

I would point out that the Portal component/system is a generic thing that spawns some entity prefab, in some area and at some rate, all configurable. It needn't even be on a metablock, it could be on a mob, or anything. The only components it really needs to be related to is Location.

It would be a big balancing act to get it right, that I can see. Most the criticism I found was related to over-architecting or getting stuck in the minutiae and I can easily imagine that. Part reason I did the Groovy prop files for Blocks was being able to super easily define lots of properties in a hierarchical approach that could then be stored in an OOP tree starting from Block. If you were to do that with ES you could, but the question is how tiny Components you'd end up with.
Yeah. I think some things are better for entity systems than others, and I don't think I'ld do blocks as a collection of components. I'm not sure I would do them as a OOP hierarchy either, but thats another story.

In the Groovy definition approach to creatures you could eventually include very detailed anatomy config, which in the OOP approach + Manifestor setup could be fairly easily defined and interacted with down to the fingernail level (like in DF) - but I wonder how'd you do that in ES, make Components right down to "Finger" and assemble those into creatures? I suppose you could.
It wouldn't make sense to me to have different components for different parts of the anatomy. The first warning light that goes off is a person has multiple fingers, and an entity can only have one of each component type (in my implementation anyhow). In fact I would suggest each piece of anatomy would be an entity in its own right with appropriate components for its function (probably a general health component, maybe a hold component for hands, a connection component to relate them to each other, a wear component for chest, head, ring fingers, feet and legs).

Edit: One particularly nice article for an intro to the concepts that can help make stuff click is http://gameprogrammingpatterns.com/component.html
Well, technically that's a component system rather than entity system as T-Machine means, as those components have behaviour rather than just being data. Unity3D has a system similar to that one. They are very similar concepts though.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Immortius said:
Well, one question to ask here is do Plant/Tree/Liquid have data? If not, why not make them properties of Block rather than Components unto themselves. Or perhaps, do you just have a single "Growth" component with data describing how the block expands outwards. The settings may differ between plants, trees and liquid, but the common element to them is their growth.

For NightBlock my inclination was to keep individual blocks outside of the Entity System completely, as they can be stored and worked with much more efficiently as a unified group. In particular, render chunk generation is quite specialised and distinct from rendering any old mesh. I hadn't done any dynamic blocks types though, and can see some benefit to having an entity for each individual tree.
Data, yes, sort of. I was experimenting with the setup and theorizing about what special blocks may become more distinct over time. It is all placeholdery at the moment, just showing off ideas. Only thing I could think of at the moment for Liquid was "viscosity" and maybe later a more advanced liquid simulation could ask for more. Plants and Trees grow, and yeah, I was wondering if a single Growth Component could do it, but the two may diverge quite a bit - which is fine in OOP, but not so much with a single Component

For general "primitive" blocks yeah, I agree, ES is over kill when even the entity ID would be several times bigger than current memory requirements of one byte per block. I was wondering what to do about that, but I suppose in theory you can leave primitive blocks as they are and only use ES for what I call MetaBlocks - that is, any block-related concept that's fancier than a primitive. Then if a MetaBlock requires specialized rendering, it does the Mesh component thing and gets attention for a secondary rendering sweep of some sort?

Immortius said:
Yes, I would agree with most of that. I'm not sure how Artemis does its internal storage, but mine was Map<Class<? extends Component, Map<EntityId, Component>>. This means that you could easily iterate over all entities with a given component type associated with it, or get a specific component type for a specific entity. Remember that entities are just an identifier, not a container. Components are only attached to each other by association.

I would point out that the Portal component/system is a generic thing that spawns some entity prefab, in some area and at some rate, all configurable. It needn't even be on a metablock, it could be on a mob, or anything. The only components it really needs to be related to is Location.
I think Artemis was about the same, tho I'm getting all the different snippets mixed up some, I think the main author had some nice snippets in a later blog post too that made it really nice and handy to get back the right references. For whatever reason the idea of every entity in a giant pile still makes me nervous, but that's probably just the OOP talking.

I hadn't thought about splitting Portal, but that might make sense. I have some more goals for it, but separating out the "Spawner" piece sounds totally doable :)

Immortius said:
Yeah. I think some things are better for entity systems than others, and I don't think I'ld do blocks as a collection of components. I'm not sure I would do them as a OOP hierarchy either, but thats another story.
I'm thinking a mixture of ES and OOP might work - and after that author is done feeling a disturbance in the force from me saying that I'll elaborate by splitting it out thoroughly between ES MetaBlocks and OOP templates :laugh:

Or try, anyway. It sounds like an interesting experiment. Still not sure if that would be the sole ES or if maybe it would make sense to have more than one for completely other parts of the game. Do you have any further ideas on how to structure that piece?

Immortius said:
It wouldn't make sense to me to have different components for different parts of the anatomy. The first warning light that goes off is a person has multiple fingers, and an entity can only have one of each component type (in my implementation anyhow). In fact I would suggest each piece of anatomy would be an entity in its own right with appropriate components for its function (probably a general health component, maybe a hold component for hands, a connection component to relate them to each other, a wear component for chest, head, ring fingers, feet and legs).
Good catch, and I think I let my original Groovy ideas bleed through here. I was thinking about making a super easy way to track individual pieces of anatomy by only defining what deviated from a general type, organized hierarchically. So say a "normal" hand has 5 standard fingers, for a creature with 4 fingers you'd just set fingers=4 at the hand level. If a single finger did something special you just set finger[0]=[something special]. Again, totally experimental, but I was wondering how ES would handle that, if it is even appropriate. You point out a more functional approach to that, which probably makes much more sense :)

Immortius said:
Well, technically that's a component system rather than entity system as T-Machine means, as those components have behaviour rather than just being data. Unity3D has a system similar to that one. They are very similar concepts though.
Yeah - I'll give you that one for sure. I realized it was different, but for some reason it made the "one-step-further" entity thing "click" for me :geek:
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Bump - Immortius' growing BlockShape system has given me more to think about on this topic, and it is getting pretty crazy :)

Copying it here from GitHub for discussion:

Immortius said:
What I am envisioning at the moment is that block shape will just be an option in the block definitions, along with an "alignmentType" to specify what rotations are desired, and that it won't have any affect on the block indexing or persistence otherwise - except that the additional shapes will push things towards the 256 block limit. Existing blocks types may also have be changed to have shape (cube, billboard), depending on how the tessallator handles them (whether there are efficiencies that would be lost).

For instance:
BlockDef Air
BlockDef Grass (shape=cube, alignmentType=single)
BlockDef Stone (shape=stairs, alignmentType=rotateAroundY)

Results in byte to block:
0 - Air
1 - GrassCube
2 - StoneStairNorth
3 - StoneStairEast
4 - StoneStairSouth
5 - StoneStairWest

The loaded blocks would have the necessary mesh information in them, in a form suitable for optimal addition into a chunk mesh.
That hadn't even quite clicked for me - that fits in so perfectly, right down to defining multiple blocks with minimal text (one shape term causes four unique blocks to be manifested above). At first I was still just thinking that we'd have Meta/Custom Blocks hold any special details for fancy blocks, but yeah, absolutely nothing prevents us from baking shape into the Block behind the byte ID stored in the chunk.

Aside from that pesky 256 block limit, anyway. I figured we could avoid that by using ID 255 = CustomBlock with a secondary lookup to catch the more sophisticated blocks that could be put together by players, even dynamically in-game. But seeing the shape trick made me wonder how to differentiate between what has to go into a secondary lookup and what gets to stay in the "native" 256 block range. An initial "Any constructed block, not natural blocks" falls apart pretty quickly when you consider something like a tree stump, which could be an example of a natural block with shape (half-block or so) - but as more trees get added, you'd need a unique ID per tree species.

That's part of why I thought shape would have to be something "more" since it can so inflate the possible count of unique blocks in the world. Likewise with custom blocks put together by "mini blocks" even if not shaped oddly, because even if you can just fit together 2x2x2 mini blocks into a normal blocks the number of unique outcomes rises sharply as more base block textures get added. Imagine accounting for a simple staircase block of every orientation and every material you could build stairs with.

I'm going to have to ask for you 3D wizards to chime in on what's preferable for rendering performance - it seems we have a few options!

1) Do away with the idea of hokey ways to allow expansion beyond 256 possible block combinations and simply expand to using a short instead of byte?
2) Come up with a better convention of what is a vanilla block and what is a custom block (secondary lookup of some sort)
3) Use a different storage system entirely (internal DB?) and take advantage of something fancier? SQL performance, entity systems, etc
4) Also throw in considerations for other things? Liquids? Complex shapes (like columns)? Dynamically crafted meshes? Collision detection? Would multiple Z-level Chunks help / be cool? Non-axis aligned objects?

#1 is clearly the most simple, but may also do terrible things to performance. I'd like to think 32k unique block IDs would keep us going for a while tho... :D

#2 is in between and interesting. CustomBlocks would be pretty rare in the grand scheme of things, tho with a sharp contrast between natural areas (almost entirely vanilla) and built-up areas (could hold a high number of fancy player-made blocks). Figuring out a workable convention probably wouldn't be too hard and could be fiddled with dynamically depending on what's enabled (fancy structural blocks, fancy tree system, fancy mineral system)

#3 ... is mainly there just to provoke thought. I know next to nothing about the arcane magics of 3D rendering and high performance I/O and imagine trying to do anything fancier for every block everywhere would kill performance. Even if you have it backed by SQL it can't compete with single-byte blocks, can it? At the same time we will hit more fancy systems down the road that will end up suddenly exploding what we might like to store about individual blocks, such as "block ownership" on a multiplayer server.

#4 might be too early of a thought. I'm wondering what a liquid simulation system would take when going beyond whole blocks of liquids. Even the 7-stage (or so) directional (tilted block top) approach in Minecraft seems substantial, yet still remains pretty primitive. How crazy would it be to try to go beyond that in realism and what sort of storage would you keep it in? I added the stuff after "Liquids" while crafting the overall post so will just leave those there...

I'm definitely prone to frameworkitis/architecturitis so I'm going to stop talking now before I get into triple-short composite primary keys that could be both x,y,z coords able to support a 1,000+ square kilometer world while also serving as entity IDs... :geek:
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Cervator said:
1) Do away with the idea of hokey ways to allow expansion beyond 256 possible block combinations and simply expand to using a short instead of byte?
Well, certainly simplest. I guess anything between 8 and 16 bits is possible as well, although that requires some bit manipulation magic. The methods would all pass around short, it would just be the storage doing that magic though.

I haven't had a look at how chunks are handled at the moment - whether they are just straight arrays (which is nice and fast) or something fancier (i.e. the technique hinted at here).
I'd like to think 32k unique block IDs would keep us going for a while tho... :D
64k you mean. :p
2) Come up with a better convention of what is a vanilla block and what is a custom block (secondary lookup of some sort))
I haven't had a chance to catch up on what you are thinking of doing with Custom Blocks yet. Depending on how they are used they could be handled by an entity - or by appending to the block table for a world as and when they are needed.
3) Use a different storage system entirely (internal DB?) and take advantage of something fancier? SQL performance, entity systems, etc
I'm not sure I would want to store block data in a db as anything except as a binary blob. To give some context, to build the mesh for a chunk of the world more or less requires iterating through every block in that chunk and also some some blocks in adjacent chunks. So the data really all needs to be in memory, and any sort of persistence needs to allow bringing the whole chunk into memory quickly.

For additional block data it might make more sense, depends what the use case is. For things like ownership, might be best storing outside the block as some sort of Octree or region based structure. Or a Map of location to info.
4) Also throw in considerations for other things? Liquids? Complex shapes (like columns)? Dynamically crafted meshes? Collision detection? Would multiple Z-level Chunks help / be cool? Non-axis aligned objects?
For collision detection, I haven't looked at how it is handled at the moment yet. I know you use jBullet, but I also know the chunk rigid body isn't used for player collision detection. I can allow the specification of collision shapes per block via blender if that is useful.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Aw, that book preview is limited and only shows pages 43 and 45, skips the one you linked to :(

And yeah, 64k, not 32k - at first I was remembering that a short is signed, unlike a byte, but nothing really prevents us from using negative IDs.

immortius said:
I can allow the specification of collision shapes per block via blender if that is useful.
I find your ideas intriguing and I wish to subscribe to your newsletter :geek:

And by that I mean I don't know a thing about it, nor much about how we collision detect yet, but it is a neat topic and it sounds like Blender and fancy Groovy generators for definitions of various sorts could be very useful :)

Edit: Octrees look handy. I've missed / rusted so much during my years of non-development. Boo!

Edit2: More stuff seemingly relevant to interests, etc: http://en.wikipedia.org/wiki/Cube_2:_Sauerbraten - seems like that sort of octree-based cube subdivision is exactly what we could use for CustomBlocks, i.e. blocks that have a detail level lower than one full block. And the game engine is open source so we might be able to see a good implementation example: http://sauerbraten.svn.sourceforge.net/ ... src/engine (I see "entity" references too, in the cubeloader, but not looking enough to see if that relates at all to an entity system)
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Cervator said:
Edit: Octrees look handy. I've missed / rusted so much during my years of non-development. Boo!
Page 44 had a diagram that showed an octree-like data structure for block storage. It was a series of lookup tables, each of which subdivided a block into 8 references down to lower level lookup tables. The storage efficiency came from identical sections merely pointing to the same lookup table. So if the top structure represented a 128x128 block, and two of the 64x64 subblocks were solid dirt, then the 128x128 level lookup table would have two references to a single 64x64 level lookup table, which would have 8 references to the same 32x32 level lookup table, and so forth down to a single 2x2 lookup table which would contain the actual bytes.

Seemed like an interesting idea anyhow.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Definitely does. I asked Begla on IRC and he said he had tried octrees for block storage before, but that they didn't work out - maybe at the scale we need them at? I'm bad at remembering details.

Does seem like chucking large amounts of Air for example into those octrees rather than store them individually should take less memory - but then looking it all up might take more effort. Oddly enough it sounded like it somehow managed to take more memory.

Still interested :)
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Could be really worth writing a prototype!

Octrees can be used to speed up rendering and finding relevant chunks in the view frustum without iterating through all cached chunks. Storing individual blocks using a Octree will absolutely kill the memory overhead (and decrease access times from O(1) to...). But maybe I've missed the actual strategy. I'll have to read through all your posts more carefully – just returned from work and dinner is coming up. ;) Will finish the new rendering pipeline first before getting into other stuff.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
So I'm finally tinkering with entity systems / meta blocks / block structures and other fun stuff now. Lots of thinking and testing there to really make sense of it, especially since there are subtly different but related concepts like a meta-block construct representing a specific portal in the world (so each block position would use absolute world coords) but also defined blueprints with relative coords and block IDs for each position, and even "soft" blueprints that might define a shape to dig out that must be bordered with stone/soil (but any stone/soil)

Some related concepts might work with an entity system and some might not. But one thought that came up I wanted to check - on the issue with being capped at 255 block IDs (and "Air"), can we leave any arbitrary blocks empty in the main chunk array (so "Air", really) and rely on a second pass (sourcing from an extended block id collection of some sort) to render any custom blocks that can't fit within the main basic block storage? So instead of setting blocks that can't fit into a normal ID to a key value like 255 to signal to the system to look elsewhere, just ignore them and rely on the secondary pass to render?

I'm sort of figuring no, since the render process probably has to do some things in order (opaque things, occlusion, etc). I also don't really know if it matters since the lookup has to happen either way.
 
Top