GSoC Proposal Draft - Terasology Launcher

[majordwarf]

Hi,
So I've written the first draft for the launcher on how I'd approach the following idea, any feedback is appreciated.
The proposal is 9 pages so a TL;DR -

• A new launcher using web technologies [ React-Redux for frontend and NodeJS for the backend and Electron to build multi-platform executables. ]
• Implement the already existing features of the launcher into the new one.
• Add some new features [ Displaying blog posts, Java Detection, File validation etc. ]

Draft : Link

The draft was shared earlier on Discord, forgot about posting it on the forum.

 

[Cervator]

Went over it again, added more feedback

I still really like your timeline table format, best one this year so far!

Still personally reluctant about the very idea of rewriting the launcher in a different language, but that's a personal opinion and one I'm happy to be convinced out of. So far the proposal seems fair enough

 

[Michael]

My first thought is what if we want to share some code between the Terasology engine and the launcher. There might be ways to add some consumable stuff between java and the electron but that seems like a pretty complicated solution. I do concede that java lacks a really good GUI library. Maybe this is for the best that we build the app using Electron.

Do we still have problem with the JavaFX dependence because I would assume that would be a matter of packaging the app better. Maybe rewriting the app is not really necessary at all since we end up throwing out one set of problems for another. I would rather the current launch be improved then thrown out.

These are my few cents on the issue of just starting over with a new launcher.

 

[Cervator]

Yeah JavaFX is still a problem child, I'm really curious if there'd be simple options to embed the JRE and avoid that one issue - if that's easy and fixes everything that's been a hassle with JavaFX ... then we're sort of back in business?

 

[Michael]

I'm kind of confused why it would be necessary to drop the entire codebase for an electron app. :?

 

[Cervator]

I'm going to try to summarize some discussion on chat, both from Discord and between mentors:

Pros for using web stack:

• Make the launcher more lightweight
• Avoid troubleshooting JavaFX issues
• Attract new developers (those more eager about web tech)
• Integrate with other web sites of ours, like the module page
• Big community around simple UI applications (easier to find some help online than for JavaFX)
• Probably more future-proof compared to JavaFX
• Possibly we could share React-based utility between a web tech launcher and some of our websites if using React?

Pros for using Java/JVM:

• Current launcher already exists and has a set of features that work (but of course also others that work less well ...). Not starting from scratch
• Stays in the Java ecosystem; retains knowledge about tech present at org level. Lower maintenance burden to have fewer tech stacks
• Less confusing ecosystem - JS is exploding and a new framework comes out every week
  • While sometimes less is more one problem is missing a good 2D UI framework which led us to JavaFX years ago. Newer alternatives?
• Potentially more ready flow of developers between the Launcher and the other game projects (dubious if this has been a big deal in the past)
  • We're decidedly a Java shop primarily, wider tech could reach more people but make the talent pool more shallow overall?
• Already have i18n set up in a similar fashion as the Terasology engine
• Reuse of existing libraries and frameworks, retains possibility of using new things in more places (like smaller libs such as config)

Other related thoughts

• As noted above - could we examine other Java 2D UI libs that have come out since JavaFX? Any potential in LibGDX somehow? Or NUI?
• Potential in using/adding another JVM language to access more utility? JS could potentially get mixed in that way, but that may be a mess
• How hard would it actually be to embed a JRE with the launcher (in which case also use it for the game) or the game? With or without JavaFX
• If we consider a ModuleManager Library eventually then it could be put to use in a Java launcher, but probably not in a web tech launcher
• The current code for talking to Jenkins to get version info is messy and needs replacement anyway but a new back-end structure isn't actually in place yet. We should probably auto-publish releases for both engine and modules to the GitHub release page. Should we fetch releases from there? What about pre-releases or snapshot builds? Jenkins? Artifactory? Promote to Bintray somehow?
• DestinationSol already comes bundled with a JRE, which is a requirement for proper integration with Steam anyway. But the way it is organized isn't necessarily ideal - I don't think we've ever upgraded the JRE it ships with, for instance, and the files are just directly in the main repo. There's also old bundled JRE work for Terasology, but it is uncertain whether any of that is helpful or hurtful

Ultimately I think it comes down to the following - which case wins in getting us most benefits in the end?

1. Rewrite from scratch in web tech to access better frameworks and maybe reach more resources / devs. Risk: Fragmentation / lowered reuse, higher initial effort. Will we actually end up with less launcher than we had before and an equally stalled project, or will we be able to bring in enough added resources to resume steady progress toward something better than current state?
2. Fix the JavaFX issue by bundling the JRE, probably a substantial task, but might leave remaining time to then push forward with new features. But would this just leave the project stalled and mysterious due to the dependency on JavaFX? Would it be hard to work on?
   1. Maybe worth noting: we should eventually add support for bundling a JRE somewhere.
3. Bypass the JavaFX issue entirely but stay in Java using some other UI framework. Focus on just replacing JavaFX, use spare effort on new stuff. Could this somehow achieve the best of both approaches by using something new without throwing away all the old?

 

[Skaldarnar]

A quick round of searching the web revealed jreznot/electron-java-app which combines Vaadin with Electron. Curious question: is there potential for a web-based launcher, i.e., having the launcher experience by just visiting a website? What are the limitations a browser enforces (cross-site stuff, downloading files, opening files or executing scripts on local hard drive, …)?

I would like to add the point that, besides attracting more devs, the JS community around simple deskop apps offers many tutorials (yes, there are both good and bad ones). I would like to make sure that, whichever UI technology we switch to (if any) provides enough material to troubleshoot typical errors…

 

[Cervator]

... having the launcher experience by just visiting a website?

That'd be neat! Reminds me the old stuff like the game applet or talk about using webstart, now both long gone.

And yeah, full agreement on the JS ecosystem definitely being hot and full of good stuff these days

 

[majordwarf]

I was looking into the frameworks as we had a long conversation on discord and as @Cervator mentioned JS frameworks keep popping up now and then. And looking into bundling JRE with electron, some of the stuff hasn't been updated in months. This made me think again is it optimal to use it? and the issues that will be caused in the long term. Seeing there had been many minor issues occurring make me want to reconsider the state of the proposal. Choosing a more mature and stable platform would be better than change tech and re-writing the code every 4-5 months when there is no support.

Looking into wrapping JRE with launcher I came across Launch4J: http://launch4j.sourceforge.net/
This bundled JRE later could be used by the game too if we want it that wat.
Also, all the fancy UI stuff can be possible with JavaFX as it was pointed out by @Skaldarnar in slack conversation.

• Integrate with other web sites of ours, like the module page

Stuff like this can be done using plain Java with some HTTP calls and fetching data.
And JavaFX being developed by Oracle I don't see it going to void in anywhere near future. I agree there's more support for web-dev but one thing I came across was that sometimes electron being based on chromium can end up hogging memory and that's one of the last things we want to do. Also as said sometimes Js frameworks are neglected as new one pop around, electron has been having issues with delta updates and it had been a long time issue and still not fixed.

There's Xamarin if we want to avoid bundling JRE as it based on C#

I guess more research will be required in this area.
I'd love to dig deeper into this after GSoC and implement a better robust solution if possible.

 

[Skaldarnar]

We had launch4j in the discussion in the past, and even though don’t recall the details I’m pretty sure there were issues either with bundling for different platforms or something else. You might want to go through the forums and old GitHub issues and PRs to get more information on this.
In addition: is launch4j still actively maintained?

Do you have some references regarding electron and delta update support?

 

[majordwarf]

Launch4J was last updated on May 2018 but seeing it gets an update once a year it seems active.
As for the Electron delta upgrade, the following issue was started in November 2018, LINK and pretty much no development. This being just focused on Windows, Linux and Mac have remotely no support for delta updates. Mac issue: LINK & LINK

As said the electron route seems like a temporary solution to me. I'm looking to dig deeper into forum and old issues once done with exams.

 

[Cervator]

I still definitely think we have some (hard) architectural decisions to make here, just like with the module site. I remain drawn to Java although I don't want to lose track of one founding principle of the launcher: it was meant to be super simple to launch, specifically so it would work in cases where the full game might not, giving the user better odds to find useful troubleshooting information (like capturing typical log statements and telling the user in plain English (or even a translated language) that their video card likely isn't good enough).

There another helpful part is that the CrashReporter is hooked up to both the launcher and the game, making the usage easier, both can upload logs directly to the telemetry service etc.

That simplicity was also a choice behind JavaFX in the first place since we were trying to minimize needed libraries for the launcher, expecting the JRE to make JavaFX readily available. Then that didn't happen

Even if we were to go with webtech I'd expect a similar focus on simplicity and minimalism. Which probably even knocks out using our own NUI instead of JavaFX since I think it is OpenGL-based through LWJGL.

One crazy option, and hear me out here, we could even use ... Swing. Or AWT. Scary, I know! But if we're putting too many buttons and things in the launcher we might be doing it wrong

Launch4j came up, yeah, I think going to a custom but tiny exe was an attempt to throw in JRE detection or better handle command line args / output? Or maybe it was one step toward bundling the JRE? Been a while.

I'm still happy to consider all the options as we keep learning and poking at things

 

[AvaLanCS]

Late to the conversation, but replying to "Potential in using/adding another JVM language to access more utility?": groovy would be a suitable candidate since it is very close to java syntax-wise, so it is easy to make the jump for java developers, but has enough convenience features to make up for the effort of integrating it.

For example the Groovy HTTPBuilder library can do a http get/post with 10 to 20 lines of code, while the same is around 60-70 lines of code in java7, and it requires a lot more knowledge of http headers, while the groovy lib sets them to the right value with a few configuration parameters.

 

[Cervator]

Groovy is sweet, and great for utility. It is however also slow as molasses

Great for build scripting and such, although I suspect Kotlin might even be trying to catch up there since it can now be used in Gradle as well, potentially with better code completion & validation

 

[Skaldarnar]

I really like the conciseness some languages bring to the table for solving specific tasks. HTTP requests are one example, handling files and streams is another, processing data in lists or sets another, ... Java is slowly catching up on many of these features, although it will likely never reach the same specialization as Scala or Kotlin do.

When considering different languages we should keep in mind what they are/were intended for, and whether that fits our picture. I would not recommended a language like Groovy for full-fledged game development, but if you just want to write a small in-game script it might be a viable alternative...