Textures, blocks, and Groovy, oh my!

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
So here's the first check-in for my take on how to use Groovy to load Block data in a very dynamic fashion, supporting both internal and external blocks and allowing us to do away with hard coded block IDs, the big terrain.png, assorted hard coded data in stuff like the plant growth simulator, and eventually create a way to make creatures, manage meta-block persistence, and all kinds of craziness.

A lot of puzzle pieces fell into place while working on this - what was just a quick take on how to switch the Groovy around so the applet could easily work again turned into "Oh, we can architect half the game around this stuff!"

Big disclaimer tho - it is still mostly a concept in example code, doesn't do anything other than print some stuff it loads out of tiny Groovy "properties" files (really classes with closures that look kind of like prop files), and actually doesn't even load the game as I'm not at that step yet. System.exit ftw! Look at the log output.

But the bigger the potential grew, the longer it was going to take before I can commit, so here's an early preview and a lot of fancy words - first the code: https://github.com/Cervator/Blockmania/ ... 0907e7cd2b

The idea is this: http://groovy.codehaus.org/ConfigSlurper is neat. It essentially allows you to very easily and intuitively write half prop files, half Groovy scripts, that can be compiled into readable classes. Behind the scenes closure magic is involved and actually compiles stuff into a rather ugly assortment of inner classes and junk like that, but the goal is to only use the magic on game startup - that way performance hits aren't an issue like some of the earlier attempts to get Groovier blocks.

By going back to a vastly more modular approach (similar to back when each block was its own Java class), yet gaining the flexibility and minimal code of Groovy, we end up loading blocks sort of like from "pretty" XML in a way. A sub-typing system for Blocks is also introduced, and the terrain.png is being sliced into tiny little pieces. Eventually the goal is to replace the Default.groovy and completely avoid hard coded block IDs and atlas positions in a static texture file. Instead we get a per-world Block Manifest file that simply ties IDs with block definitions (which are stored by Java in the end, not Groovy), along with a world-specific terrain.png

That does away with the hassle of manual block IDs, which are in the way for easy user contributed add-on blocks. I included a new add-on block that'll be loaded by this system when it is complete - and consists of a 151 byte Groovy script + a 930 byte PNG. That's all the user adds. Now, outside of "free build infinite blocks" mode the game will need a hook to put blocks to use, which is where the snazzy little ability of Groovy's come in - you can include code in these "Groovy Prop Files" to register the block as a certain type the game can recognize, like a plant. The growth simulator (https://github.com/begla/Blockmania/blo ... lator.java) for instance currently hard codes stuff like sunlight needed, biome details, etc, but you could change that to load block data and introduce a mushroom that overrides the default "needs sunlight" for its specific case.

By leaving defaults at each level we only indicate exactly what's needed for each block, with additional/changed defaults for subtypes - for instance liquids have a unique "viscosity" value, and plants have a "growth" section. Oh yeah, that's where we start moving from plain block definitions to game logic. Take a look at grass, for instance: https://github.com/Cervator/Blockmania/ ... 2b#diff-12

On there the quick new way of indicating faces is sketched out (the BlockManifestor would read the given props and overwrite defaults in the dynamic atlas collection), with the references resolving to individual image files under com.github.begla.blockmania.data.textures.blocks (might need better pathing / path indication). Then under the plant section (which could also include data on what a plant can be used for) there's a growth section detailing how and when grass will grow and even evolve (gotta catch em all!). In this case "grass" only has one growth stage, which is when it goes from dirt (growth simulator picks a spot and decides it is suitable for grass) to size 1 grass. However, it has an "evolve" section included, which will allow the growth simulator to again hit that block and create something else - in this case tall grass, yellow flowers, or red flowers, at differing weights. If the probability for that triggered (here 95% less likely than the initial growth from dirt to grass) the block would be transformed (or have a billboard placed on top of it) indicating the new growth.

Trees could be made to slowly grow the same way, I've put an example OakTree in there that admittedly currently would grow from nothing to its max stage of 50 in one go - since we (and Minecraft) spawn trees straight to adult size. But nothing prevents us from slowly having the tree grow in stages, using the growth simulator. Trees also tend to have leaves, so there's a reference to an "OakLeaf" block in there, which is both an attachment to a tree and its own independent block. Which is where the system starts getting hazy, as the potential bleeds into other areas. An OakTree is indeed a Tree, but an OakLeaf is not (yet they're in the same package). But you could start imagining OakTree instances with leaves and stuff attached stored in a BlockGrid actually containing its state in the world as it grows - and gets chopped down by having a key block removed, physics-enabling the rest of the tree in rapid succession rather than leave it standing...

Then you're half-way into defining blueprints for other things like lairs, you could use a similar system for creatures, and I'm about half-insane at this point (and it is a tad late too!). Start persisting the relevant BlockGrids and you can start saving data for Portals and such. The split can be maintained between binary default/native data (what comes with the game) and plain-text add-on user contributed content. Not quite sure yet how to best save everything, but there is also potential for keeping track of Serialization version levels with this stuff (for auto-upgrading worlds with newer versions of blocks / meta-blocks like trees or portals, or what not...)

I'm going to go back to working on a way to then splice together the terrain.png from individual blocks activated for the current world and actually loading the right data into BlockManager so the world will actually work again backed by the new system. I've only written some initial details on a few blocks, but hope to get Stuart involved in this to familiarize him with coding, in particular since the related content there down the road will be relevant to his interests (creature & lair design, etc). I'm also going to try pushing it to the Nanoware organization and see how two people collaborating on the same branch works. So while I make the system work more he can start putting together more details on the blocks and the plants etc that the blocks represent.

I know there's a lot more potential here, and it'll take a while to work through it and flesh out where it is going. But I wanted to get it out there early for review, architecture thoughts, and collaboration. Even if it probably sounds crazy right now :)

Oh, and on an even more technical note: Making peace between non-compiled Groovy scripts and fully compiled Groovy Classes to do the split between external/internal was fun! At first I didn't realize Groovy was just making sneaky Java classes look like text-based prop files, so I thought compiling them might screw them up. But trying to include plain *.groovy files as resources into the jar file (and be able to load them as classes/resources via the ClassLoader) would not allow any of the Groovy we do want compiled to also compile, with standard settings (could probably hack it with custom Ant etc, but that then sort of negates the ease of Groovy in the first place...). But now all that works! Internal classes can Groovy it up all day, while external scripts would be executed by the GroovyManager if need be to register their blocks etc to appropriate Java managers.
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Sounds like an awesome concept with a lot of potential! I'll keep tracking your progress and give feedback wherever I find something notable. :)

I'm currently working on an advanced rendering mechanism that utilizes occlusion queries. This – in combination with splitting chunk meshes into multiple sub-meshes – allows us to drastically reduce the overall polygon count (varies based on the viewing scenario). This mechanism can be easily configured (even the amount of sub-meshes per chunk can be changed dynamically) to allow smooth frame rates on almost all setups.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Sweet! More "space" for performance = more crazy stuff we can do in said space.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
More thoughts on this topic - disclaimer: I might just be crazy, sleepy, or stating the obvious (or any combination there-of)

Thinking in particular on how to employ a similar approach to other objects, in this case for instance blueprints for an assortment of block compilations. This could be for player constructions, building blocks for creature lairs (typical rooms), etc. If we write the "recipe" to this in a Groovy object and like blocks read them with a Groovy "manifestor", instancing matching classes (maybe a typed BlockGrid?) while writing a world manifest file to tell that world what simplified values (byte index values) saved in the world files correspond to what objects.

The plain Groovy files could very easily be used by players to develop tools and other similar blueprints, much more easily so than binary data saved to disk, yet with the manifestation process we still only save primitives to disk and maintain optimal performance (aside from on game startup). We also retain the flexibility to edit "what's what" in the world entirely simply by altering the manifest rather than saved world values. This could be useful for both version updates and power users altering a world in big ways to fit their wishes.

I wonder if this is just me restating roughly how stuff normally works anyway. Oh well, back to coding :)
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Blargh, maybe getting an impromptu cold is wrecking havoc on my ability to think or triage, but now I'm at the point of loading + splicing images to make the dynamic terrain.png - and again I can't for the life of me hit just the right combo of path + resource loading to make it work :D

Code:
println "Full name: " + path + "/" + i
java.net.URL imgURL = getClass().getResource(path + "/" + i);
println "URL is: " + imgURL
Texture tex = TextureLoader.getTexture("png", ResourceLoader.getResource(path + "/" + i).openStream())
println "Texture is: " + tex
Output for that little snippet:

Full name: com/github/begla/blockmania/data/textures/blocks/Dirt.png
URL is: null
Texture is: org.newdawn.slick.opengl.TextureImpl@11c19919

So it loads if I just copy paste the current Texture loading from TextureManager, but if I use the same path to try to get a URL (or a Stream for that sake) to make a BufferedImage from the normal class loader strategies I always get null, argh :)

Wonder if I could just use Textures for splicing rather than go through BufferedImages? Also, does it matter if texture.png is just a really wide 16 pixel high image? As in, just one image on the y axis.
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Loading plain Java buffered images like I did here is not an option? At least it works there. :D Had to tinker quite a bit at first till I found a good way to load those files if they've already been moved into the final .jar-files.

Code:
BufferedImage yourImage = ImageIO.read(ResourceLoader.getResource("com/github/begla/blockmania/data/textures/changeMe.png").openStream());
One example can be found here: https://github.com/begla/Blockmania/blo ... Block.java

Using the Slick Textures for anything besides rendering in OpenGL will be all but efficient. Those thingies internally save only the texture "pointer" to the texture that has been pushed onto the graphics card. You could read those bytes back from the GPU but that... Is not the way to go.

See here: http://slick.cokeandcode.com/javadoc/or ... xture.html

I do not really trust Slick because of the weird convenience implementation of texture binding that caused those problems Anton ran into earlier.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Ah-hah, so I can have the best of both worlds and use that ResourceLoader and feed it to a BufferedImage. I was wondering about that and looked around for a way to use Slick to give me a BufferedImage. But it was late and the fever might have been clouding my brain :D
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Cervator said:
Ah-hah, so I can have the best of both worlds and use that ResourceLoader and feed it to a BufferedImage. I was wondering about that and looked around for a way to use Slick to give me a BufferedImage. But it was late and the fever might have been clouding my brain :D
:D Hope you get well again soon. :cry:
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
No rain, snow, ice, nor gloom of dark or sickness of body shall keep me from delivering commits!

https://github.com/Nanoware/Blockmania/ ... 83ec344880

Splicing together the individual block images works now. Have to make it intelligent next so the blocks know where in the spliced image their textures are :)

Also need to find the best way to save the image, being that it is being generated before the world, so the saved worlds dir doesn't look to be ready yet. Right now it just gets dumped temporarily in project root.

Oddly enough the news feed on https://github.com/organizations/Nanoware got this submit right away, but completely missed one between that one and the gravel test commit.

Think there'd be any issues with terrain.png being 16xVeryLong pixels rather than a more normal square? We could use a simple index int rather than a Vector2f to indicate the texture location.

Edit: What's up with the 0.0624f scattered all over the generateDisplayList() in Block anyway?

Code:
GL11.glTexCoord2f(calcTextureOffsetFor(SIDE.TOP).x + 0.0624f, calcTextureOffsetFor(SIDE.TOP).y);
I'm figuring a one-texture-high terrain.png would allow us to always have y =0, but then that 0.0624f makes me go huh :)

Edit2: Another commit in, showing the new face mapping mostly done. Also found this - handy for figuring out what's what in our texture set: http://www.minecraftwiki.net/images/7/7 ... nGuide.png
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
More architectural / design musings while I'm stuck at work!

I think I'm getting the block hierarchy and "meta-blocks" sorted out more in my head now. Main example / debacle is how to define a "Tree"

We've currently got tree trunk blocks and tree leaf blocks (need to rename OakTree to OakTrunk). Both are in a blocks/plant/tree package. This poses a challenge as leaves aren't trees, although they heavily relate to trees. I was previously thinking about downgrading them to plants, but then wasn't sure if they were plants since their growth is indirect from trees (defined in the trunk Groovy block)

Moving the leaves to a plant/leaf package makes more sense as tree definitions in a plants/tree/[species]Trunk would easily be related to its leaf over in a similar package. And leaves could count as plants, as they could grow something, like fruits. Possibly also more leaves if we can have leafs further than one block from a tree trunk / branch (in theory tree branches could be defined as horizontal trunks so big trees look less silly)

The overall relationship of the tree-type blocks to a Tree object could then be made using a meta-block reference to an object class. I'm figuring this could be done with a "blocks.meta=Tree" property in the Groovy file, along with a new MetaBlock interface that objects such as "Tree" or "Portal" could implement, and the Block class could own a reference to a MetaBlock that would be filled if the block being loaded has the meta prop. The interface would require methods that would be executed if the actual block is affected in a way that may impact the overall meta-block. In the Tree example this would be if a OakTrunk block was removed, as the whole Tree then arguably could be expected to fall over (go physics-enabled). This would not happen if it was an OakLeaf block, which would also be registered as a Tree meta-block :geek:

The difference would be the base class - OakTrunk = TreeBlock, OakLeaf = PlantBlock. Both are part of a Tree, but only the removal of a TreeBlock could impact the whole Tree to fall over.

Say when a tree is generated in-game the block coords it is made up by gets stored in a BlockGrid or something in a Tree class. Each Tree is added to a Tree Collection. Now if an OakTrunk block is destroyed, which thanks to its meta-block relation is a Tree, the game logic could intelligently go look in the Tree Collection for a Tree that contains that coordinate and instantly know what other blocks make up that tree for staggered physics-enabling of the associated blocks.

A fun (but not necessarily practical!) thought experiment then is that you could make every block that's part of a tree "shake" if part of the trunk is being damaged. That might be cute for small trees, but a little less so for the huge ones we currently have, and not really worth the effort ;)

That then leads to other sorts of block classifications like workshops (really, rooms), estates, and other fun things. Those aren't really meta-blocks as that's pure an "about the block(s)" concept, but similarly they're objects in the end made up by blocks, that need to know what's happening if a related block over yonder is being affected somehow. You could potentially even classify natural world features like rivers, ravines, etc, to be able to logically cut the world into parts that could then be named by exploring players and referenced by NPCs and such should we go with an RPG angle. Small collections aren't bad for storage, but bigger ones like that might be. But it would be one way to define a series of blocks that all must have been within range x of a player before that player has fully "explored" the feature and earned the right to name it.

*continues muttering about geeky topics*
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
So much interesting stuff going on here. I will look into your commits more deeply this weekend if I find the time. Can't wait till this one goes live. Especially in combination with some more physics enabled fun stuff! :p

The ominous 0.624 equals 1/16f with a small additional offset to avoid overlapping texture borders. Texture quads on the GPU are always accessed with floats in [0..1].
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
More commits and stuff! Very close to integrating with BlockManager now, then just need save/load of the Manifest (and finish adding blocks). After that the applet should be very close. Trying hard to resist messing with MetaBlocks and adding boat loads of new/different blocks (must keep out of milestone 1!)

And ah-hah on 0.624 :)

Currently the boolean loading in BlockManifestor.prepareBlock flags null warnings in IntelliJ, despite working fine. Goes away with an unnecessary cast. I might add that just to get rid of the warnings - the minimal extra overhead prolly doesn't matter since it is pre-game loading anyway.
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Just had a quick look through all your commits. Great stuff with great potential! :) We'll have to keep an eye out how well those extremely wide texture perform on some GPUs though.

Any idea when you've got a first prototype ready for me to play around with? :D

EDIT: Will upload a finalized version of the liquid simulator soon (and maybe a small video?). Then I'll start working on procedurally generated lava and water streams.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Cool on the liquid simulator - I've got a minor todo in the block loading system to figure out what to do with / where to put the liquid boolean, if it is being used. I figured LiquidBlocks would be a notable/useful grouping, but that one boolean might still want to live in Block if needed there :)

As for the block loading system itself - almost done! I probably should go finish the AI exam first, but right now I can't resist doing a little more... very close to having the stuff in BlockManifestor ready to drop into BlockManager. Going to line up and compare the content of the old + new variables before attempting a cut-over. I figure there might be some hard coded references to hunt down before the game can launch with the new dynamic stuff.

If any GPUs take issue with very wide short textures we can dynamically generate a more square block manifest instead - that's just a little more work :)
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Cervator said:
If any GPUs take issue with very wide short textures we can dynamically generate a more square block manifest instead - that's just a little more work
Great!

Walking around between some randomly generated water and lava streams here... Uaahh. Needs some more tweaking then I'll make it public. :ugeek:

EDIT: Dangerous!
 

Attachments

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
I need to check out that branch, especially before you make it less dangerous - we need to code a volcano with a lava stream like that as part of a normal eruption! :D

I've been rushing more and more as I got closer to finishing this, and that's while also trying to do a bunch of other stuff (AI exam done, yard work, prep for trip to NYC on Friday, etc...). It KINDA works now, and I've pushed the code. As I put in the check-in comment I think the logic is just about right at this point, but the generated image isn't quite right (or the new atlas positions are off), and as a result the world looks kind of psychedelic ;)

I'm suspecting the new ImageManifest.png is not quite in the right format, or the sub-textures are off. Stuart was using Gimp and seems to have gotten transparency right, while I was just using boring ole Paint where I'm pretty sure the partial textures are all on a white background. Then after being processed in code the actual transparent sub-textures end up with a black background, while old terrain.png has white for transparency.

In BlockManifestor.saveManifest() I suspect maybe there's a different constant to use for the BufferedImage?

Code:
BufferedImage result = new BufferedImage(width, 16, BufferedImage.TYPE_INT_RGB)
Anyway, it is past midnight and I have to go to work tomorrow. And I'm still not over the cold. Argh! Feel free to tinker in the organization branch, if you'd like to try different format options or what not. I'll try to add some polish tomorrow on the stuff I rushed through tonight :)

Edit: Oh, while hunting down hard coded references to specific blocks or blockType byte values I came across a spot in LocalWorldProvider.setBlock under LIQUID SIMULATION that made me pause. All the 0x0 places are handled by just hard coding Air in this loader to also be 0, but this spot has a (byte) 7 in it that I think is not actually a blockType, but the javadoc insists it is:

LocalWorldProvider.setBlock

Code:
            /* LIQUID SIMULATION */
            if (BlockManager.getInstance().getBlock(newBlock).isLiquid()) {
                c.setState(blockPosX, y, blockPosZ, (byte) 7);

..... hunting down c.setState

     * @param type The block type
     */
    public void setState(int x, int y, int z, byte type) {
What exactly is the state? the setState method seems to insist "byte type" is block type, but there's also a _blocks for actual blocks... I'm figuring it is a different sort of data for the same block, like how there's a sunlight counter
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Edit: Oh, while hunting down hard coded references to specific blocks or blockType byte values I came across a spot in LocalWorldProvider.setBlock under LIQUID SIMULATION that made me pause. All the 0x0 places are handled by just hard coding Air in this loader to also be 0, but this spot has a (byte) 7 in it that I think is not actually a blockType, but the javadoc insists it is:
That's just another "nibble-array" I've introduced for storing state information for any available block type. This is currently only used for liquids to store which distance a liquid block has traveled relative to the original source block.

Might become useful for creating switches, doors, ... later on.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Ahhh - neat. Might want to update the Javadoc for that when you get a chance :D

Getting to work late today, alas! But I think it was worth it :geek:
 

begla

Project Founder and Lead Developer
Contributor
Architecture
Logistics
Cervator said:
Ahhh - neat. Might want to update the Javadoc for that when you get a chance :D

Getting to work late today, alas! But I think it was worth it :geek:
Whoops. Will fix it! :)
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
So yep! The generated image isn't quite in the right format for some reason.

For starters TYPE_INT_RGB lacked the transparency, but switching to TYPE_INT_ARGB just let some of the textures go screwed up with transparency in places :)

I tried renaming the terrain.png and placing it where the generated image goes, which actually resulted in valid textures (so yeah, logic works at least mostly) though of course the totally wrong textures, as the atlas coordinates are way off, but that worked in an amusing fashion as I was wandering a brick/tnt hillside while looking at an ocean of gently swaying roses :laugh:

I tried some of the other format constants in BufferedImage, but for all I know the ImageIO writer or something else is off - for instance the DPI/PPI (or what not) differs between the generated image and terrain.png when I compare in Gimp (Paint shows something else?). Not really my expertise - able to take a quick look, Ben, and see if you can apply your graphical magic to get us to a working format? :)

Here's a screenshot from the fun screwed up world:
 

Attachments

Top