Dev meeting Saturday March 12th. Architectural topics beyond engine v1.0.0

[Cervator]

Bit of short notice on this but I've been mentioning it on IRC and Slack with several key people confirmed available already. This is architecture first, GSOC second.

World time planner link - 1 pm Saturday US West, 4 pm US East, 10 pm Germany, 2.30 am Sunday India (sorry!), 8 am Australia (Sydney)

Location: #terasology on Freenode IRC (just hit the chat tab here on the site if you don't have an IRC client)

I'll be around for hours before and after for general topics like GSOC student help, maybe some play testing on @minnesotags' server for the final build candidate, and so on

Sunday March 13th is the v1.0.0 engine release. Monday is start date for GSOC student proposals.

Agenda:

• Final check for officially releasing engine v1.0.0
• Any more GSOC mentors?
  
• Immediate future beyond v1.0.0 - Playability
• Longer term architecture

Final release check for engine v1.0.0

This is just a formality I expect will be brief. Final chance to raise any concerns before we freeze the engine/modding API for v1.0.0 until the next major release (months away). I don't expect any concerns and realize we don't even technically need the modules super stable as that doesn't affect API.

Any more GSOC mentor volunteers?

We've got enough for two students now, but could use more just in case. I doubt we'll get a third spot but still, a few more even backup mentors would help our numbers in justifying a second slot

Note: You can volunteer as a backup mentor just to help represent, without commitment beyond checking for questions in the forum or elsewhere if you get a chance and can help. Pretty much "IRC help" level. We have two primary mentors with actual time committed.

Immediate future beyond v1.0.0

We've been doing architecture at this point for years. Time to switch gears! Releasing engine v1.0.0 and entering Alpha at the game level will be a good time to step out of stealth mode and get some attention for the game.

It'll also be a time to probably focus on a new playability milestone - currently we're missing a fair amount of simple content and expected usability features like being able to drop an inventory stack outside the inventory to drop it. Some of the remaining v1.0.0 issues will go into that milestone and they should be good material for our new contributors (not GSOC level, but maybe more challenging than simple issues)

As @Florian recently pointed out continuing to work architecture without stable content for actual players can lead to burnout, and probably has. I'd like to see if we can get some people focused on this and some issues submitted on GitHub for what we're missing for a journey to Beta and a better experience for end users.

Longer term architecture

We have several areas still pending refactoring and some potential for future options

• gestalt-entity is being extracted - @Immortius is working this but with no set timeline for when it might get used for Terasology (could even get used for Destination Sol first?)
• NUI might be getting extracted and/or get an editor of some sort (a GSOC student is looking at that), and again might get applied to Destination Sol. Exactly how much gets extracted?
  
• Finishing CoreRegistry -> Context? Extract it? Do any libs need it? Where does it go?
  
• Rendering still needs an overhaul. @manu3d has slowly refactored away but we need cleaner overall architecture like the DAG Pipeline approach. We have several GSOC students interested in this or issues that depend on it (like refreshed Oculus support)
• @Marcin Sciesinski has reappeared out of nowhere with shiny toys as he tends to do from time to time: an incubating LibGDX-based re-envisioning of Terasology's base engine, with assorted tweaks to the entity system and rendering. Main highlight: not a repeatedly refactored engine that started as a tech demo.

Question is: which if any do we pursue, when? Would it destabilize or waste too much effort in the short to mid term? And could any involve effort from GSOC students or risk getting in the way for GSOC efforts?

For instance the DAG Pipeline issue has been identified as a possible new GSOC item, while also likely blocking several other items like the Oculus or far rendering items. I don't personally know what level that fits at. Would it work the same with LWJGL and LibGDX (higher level architecturally) or would any future plan to consider LibGDX waste that effort? Should it be built on Marcin's new approach instead for a v2 or even v3 engine release? Or can we refactor the current engine without a major release and just slowly improve what we have?

In the perfect world I would love to see us build a pile of awesome content on top of engine v1.0.0 including a huge Playability milestone getting us a fun game with lots of activity. Content that would then all be compatible or nearly effortlessly adjustable for a v2.0.0 engine release with some of the above improvements.

If the impact on existing content is minimal we could probably do another major release within 6-12 months. But if the impact is major it may have to wait further, or maybe we shouldn't even do it. The good news is that the engine is getting smaller and smaller with content and libs being extracted out. As always I am looking for discussion and hopefully consensus, we have better experts than I on evaluating our architectural future

 

[Cervator]

Here's a Google Calendar link that might also be useful for time zone stuff Yeah that totally converts poorly for some reason. Dunno what's wrong with it! Go by the posted times above instead, the site should help convert to more timezones if needed

And as a quick clarification this meeting is not mandatory or even primarily related to GSOC - students need not attend or worry that not attending affects your chances to get picked

Mostly the talk will be about architecture and planning, which might indirectly affect some GSOC topics.

 

[Cervator]

Another bit to add for the rendering topic by Waterpicker from IRC:

The rendering module of flow Caustic and render combined are a DAG pipeline based on LWJGL

• https://github.com/flow/caustic
• https://github.com/flow/render

Flow is a continuation of the Spout project we talked to once or twice long ago. Could perhaps be some architectural inspiration there if they already have a working DAG pipeline. Way back when we talked last we were ahead in the rendering department, maybe that has swapped by now

Edit: They used to have a demo at https://github.com/flow/reactsandbox (React is their physics lib) but the GitHub page there seems to be broken. The demo used the rendering stuff.

 

[manu3d]

Thank you for the flow caustic links. I only spent 2 minutes browsing the code and got mightily annoyed by the complete lack of comments/javadocs, at least in the classes I went through. But I'm not particularly agile/fast in reading other people's code. Perhaps it's a perfectly viable option. I certainly won't be able to tell by tomorrow.

Regarding the meeting, I'm still GO for it, but my 2 years old daughter is spending at least tonight in the hospital due to what is probably a viral infection that triggered an usually powerful immune response - enough to raise doctor's eyebrows and command a whole battery of very uncomfortable/painful tests. She -should- come home tomorrow but she might have to spend additional nights there depending on further exams. Either ways I should be able to attend anyway, i.e. having the laptop with me at the hospital and relying on the fact that normally at that time she sleeps, ill or not. There is however a chance I might have to drop out of the meeting or arrive late, if something unusual happen. Just so you know.

 

[Cervator]

@manu3d very sorry to hear! I hope she does well and feels better soon. No worries about possible meeting attendance issues, you know that kind of stuff doesn't even rank in importance vs RL responsibilities like that

In other news I struck through the Google Calendar link as it seems to convert to other timezones weirdly poorly. Go by the posted times in the top of the thread instead please.

 

[Marcin Sciesinski]

I was asked to give some pointers of where the rendering is done in the new engine I've been working on. The class in question is at:
https://github.com/MarcinSc/TerasologyPrime/blob/master/core/src/com/gempukku/terasology/graphics/environment/ChunkMeshGenerator.java

Please note, that at the moment it is not extendible, but my plan is to allow registering custom vertice producers that will be responsible for appending to a chunk mesh for specific block types without shapes/textures defined in common blocks. I will probably start off by creating a tree renderer, similar in style to:

These trees will not have a static model, making all trees look the same, but instead will be generated using an L-system. These trees will become part of the chunk mesh to speed up the rendering.

 

[Waterpicker]

Another bit to add for the rendering topic by Waterpicker from IRC:

• https://github.com/flow/caustic
• https://github.com/flow/render

Flow is a continuation of the Spout project we talked to once or twice long ago. Could perhaps be some architectural inspiration there if they already have a working DAG pipeline. Way back when we talked last we were ahead in the rendering department, maybe that has swapped by now

I wish to add an addition to this. I found an example of how render and caustic can be used. It rather simplistic but shows the GraphNode in action.

https://github.com/flow/examples/blob/master/render/src/main/java/com/flowpowered/examples/render/RenderExample.java

 

[manu3d]

Looks promising. Before we move forward with our own DAG-based rendering pipeline we should definitely look in-depth into this. Thank you for the example. And @Marcin Sciesinski : looks very nice!!!

 

[Marcin Sciesinski]

It would be beneficial to check, if the DAG stuff is compatible with OpenGL ES, as this is what libGDX permits for compatibility, if we were to use it in the future.

 

[Marcin Sciesinski]

First successful (in my opinion) test:

 

[Cervator]

Alright, belated after meeting report here - solid meeting

@Marcin Sciesinski already posted a thread on his further work elsewhere. But to go over the meeting points chronologically real quick:

1. The v1.0.0 engine release is in progress, but as expected the prep is taking a while. Got a ton of notes written up but the changes just keep coming
2. More GSOC mentors: Marcin volunteered as backup. We just about have enough to ask for three slots if we get three good proposals, but a few more backups would be nice. Let me know if anybody is interested.
3. We have a post v1.0.0 milestone on GitHub called Playability with a focus on anything making the base game easier and more fun for players. Filling in the gaps you might expect from other games, balancing out gameplay (like in JoshariasSurvival), offering more tutorial-like play like with Gooey's Quests, and so on.
   • We'll start trying to host multiplayer test sessions in the weekends to gather more of these usability features and bug reports. Did some initial getting back up to speed tonight with @minnesotags' server but still need a bit of work (and a release)

As for the big topic of architecture I think progress was made and technical details shared but that remains a big and complex topic that might even be a tad intimidating if you're just trying to learn some code for GSOC

Bear in mind I'm not a 3d wizard or architect as I try to summarize so forgive me for getting some terms/details wrong - please correct me!

• A DAG pipeline is doable, although it remains to be seen if it is truly within scope of a GSOC item
  • Good example to review provided by @Waterpicker: https://github.com/flow/examples/blob/master/render/src/main/java/com/flowpowered/examples/render/RenderExample.java
  • The approach is high level enough to where it could be hooked up to two different levels of renderers (see below) and survive a substantial engine refactoring (such as switching from LWJGL to LibGDX one day)
  • We could probably work this item in the near future if there is interest, in-place with the engine as is, maybe re-implementing some current visual effects as nodes (or what not) in the pipeline.
• We could aim long term for two renderers, although this is further out in the future (pretty sure it would come after a DAG pipeline?)
  • First a low level OpenGL ES 2.0 compatible renderer with very basic effects that would be able to run on low-end hardware and perhaps even mobiles (via LibGDX)
  • Later a higher level optional version supporting fancier effects based on newer OpenGL when available. This can probably remain a pipe dream without causing much grief until we have multiple bored high level 3d wizards
• Terasology's entity system in its current shape is likely going into stable legacy, with plans to eventually replace it with an external version that has been refactored nicely. This is as opposed to refactoring it in place.
  • The gestalt-es that @Immortius is working on remains an early effort and contains some neat advanced features like transactions. Still more thinking to do, especially after the meeting
  • The variant that @Marcin Sciesinski started on from scratch also needs more work and differs a bit. Marcin is making adjustments to have it fit better as a candidate for Terasology and may try a trial implementation.
    
  • In either case networking is (or will become) external from the ES. Maybe the two ESes will converge with the network implementation differing? I dunno, but it seems like potential
• CoreRegistry to Context for current Terasology is still a big task (ref @Florian) - maybe Marcin's Spring-like Context approach could be used here and lead to multi-world support in the process? Linking the sector thread as maybe related.
• LibGDX and a fully "clean" engine (like Marcin's experiment) may depend on first fitting an external ES to Terasology-as is then tweaking more libraries so there'll be fewer conflicts with LibGDX, Android, etc. Rendering in parallel could improve and eventually we run out of old stuff to replace.
  
• World gen is something I didn't really ask much about. Might be something to keep on the radar, probably one of the biggest pieces left in the engine if ES and NUI get extracted?
• @Rostyslav Zatserkovnyi has been looking at NUI for extraction and gestalt-asset-core for use in Destination Sol (I think!) which would be all kinds of cool. There are some remaining attachments to the existing ES that may be tricky (may need some advice from @Immortius there)

Putting all that into a more time-specific format:

• Short term (and going forward permanently, really) we focus on stability, content and usability
• Mid-term we may have two related GSOC viable topics, but still comes down to advice from the experts and proposals from students. We also might see some ES work happen.
  • DAG pipeline with @tdgunes may be doable within current engine without impacting content. If it is considered out of scope for GSOC maybe the particle system item can still fit here.
    
  • Library work by @Rostyslav Zatserkovnyi might result more in usages outside Terasology (external libraries, editor, Destination Sol), so again likely little impact on content
    
  • @Immortius and @Marcin Sciesinski may do some neato ES architecture prep, too early to say how it goes. Sounds like a soft implementation into the engine may be possible, or a slightly harder one with a bulk update of modules.
• Long-term we could hit another major release for a few possibilities:
  • Full implementation of a new external ES including likely Context changes
  • A new cleaner low level renderer could be built on top of the new DAG pipeline offering less arcane 3d wizardry
  • LibGDX considerations could be possible with overall improved architecture, rendering, and enhanced libraries checked some for compatibility with LibGDX and maybe even Android

Does all that sounds decently representative of the meeting and our options at hand? Keep in mind these are loose goals some likely spanning years.