Architecture review / planning for engine v3 and beyond

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
#1
Maybe architecture meeting Sunday: fill out this poll if interested / able to make it. DS stuff at its conclusion, maybe.

With engine v2 just about out the door I've been thinking about architecture and trying to sort out what we should try to prioritize for when. Started this some time ago while wondering what we should squeeze into v2 and what we could manage to get in there - not nearly as much as the wishful thinking would call for :)

Rather than do one of the usual link and thought dumps I thought I'd try something different. About a year and a half ago (it has really been that long?) I posted a thread on Roles & Projects meant to help organize volunteers, and I've poked at the associated Trello board since. Most recently I turned it into a bit of a roadmap experiment as well, complete with both longer term engine goals, GSOC projects, and the aforementioned projects & volunteering roles.

As of this post I've got an initial dump of architecture topics across three lists: major goals for a v3 engine, secondary goals (less critical), and possible items to push out to a v4. There's also a small list of unscheduled/on-going projects I entered earlier, some of which could probably moved around a bit. But the thing is: I'm still hardly an architect and mostly just remember things in foggy detail. Each card on there could probably be its own whole forum thread, and some of them have entire forum threads or GitHub issues. I don't want to replicate that in Trello but I do want to group it all in a place for a better overview.

With that in place I'd really like to invite some more informed feedback, ideas, and - gasp - maybe even some volunteers willing to stick their name on the card :)

(To clarify: not aiming to replace GitHub issues at all, but the more repos and issues we get the harder it is to get an overview and a good feel for who is interested in what. Trello is way easier to use for that, especially when one card can represent a dozen different smaller things scattered all over GitHub)

Two things drive this, one short term one long term.

Short term: we had quite a few architecture / API breaking things floating around toward the end of v2 which made it tempting to keep it open longer to fit more stuff in before we lock down API again (bad, scope creep, bad!). The final entry that'll make it into v2 is probably the block family PR, unless somebody really highlights something crucial over the next week. But discussing that topic, especially in the context of @asie's plan A for GSOC (content framework overhaul, especially around block interactions and storage/shape), really got into a pile of related improvements, which I think are now all on the Trello board. We resisted going for more of those in the short term and Asie switched to Plan B (computer all the things)

Long term: v2 has been a really long time coming, both sitting around for a while (two topic branches from last GSOC) and being merged to develop quite a while ago with months to simmer without really finalizing easily. And v1 was released more than two years ago so some API breakage or other major things got pent up for a while, waiting. Originally I had hoped we could get all remaining "formal launch" type architectural changes in and base a game beta on engine v2 (I know, so naive!), even including the gestalt-entity-system reintegration. Now I'm pinning my hopes instead for that being v3.

Ideally I'd like to have a shorter life for v2 than v1, giving v3 about a year to cook and release around the start/end of next GSOC. Stuff I'm pushing out to v4 has a focus of being less about breaking API or majorly changing how visible things work and more about reworking internals behind the scenes to improve the development process further while optimizing the game. So v4 could come after a hypothetical "launch" although it may almost be meaningless to go beta and later "prodution" live with an open source project like this.

In summary
  • v2 engine release is imminent, but still a far cry from "feature complete"
  • v3 engine should IMHO be focused on getting to "feature complete" for something suitable to launch / build on for a long time (and could bring huge breakage)
  • v4 engine should focus on behind-the-scenes improvement without destabilizing anything
I'd love to hear feedback here, on chat, in associated forum posts and issues (please add links where able!), and maybe at an architecture meeting this Sunday if enough interested people complete this poll. Even if not I expect to spend Sunday on architecture, v2 finalization, and maybe the DS community project after if there is time & interest for that as well.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
#2
Alright, I'm not going to expect high attendance at the meeting, but after noticing Josharias committing to what I think is 6 am his time I can't leave him getting up early in vain ... :D

So we'll get an early start Sunday, but no obligation to make the earliest time entry in the poll (about 10.5 hours from this post - 9 am Eastern time). I'll catch up with others throughout the day as different people are around