The Great Convergence

[Immortius]

• Should the applet have a module.txt? Should any facades? Not really needed after PC went internal, Applet is never getting pulled from Artifactory, Android ... dunno

I would say no. They're not really modules. Maybe they should have something similar, but beyond giving them a version number I can't think what else it would be fore. Engine only has a module.txt because it has a built in module.

• Should repos be prefixed with their type? Right now facades would be ("AppletFacade") and modules are not ("BlockNetwork"). It may not matter if they aren't mixed (modules under Terasology org, facades under MovingBlocks org) but may be confusing in user repos that have both. However, how many users are likely to create facades?

Dunno. I assume there will only be a small number of Facades.

• Should the utility dir be broken apart and some stuff put back in default locations? Would fix the Gradle Wrapper (plugin) within IntelliJ. I didn't have any solid reasoning for squishing everything in there, just seemed tidy at the moment.
  • https://github.com/Nanoware/Terasology/tree/redoCore/utility

I think so. It is a little dangerous to have generic names like "utility" and "misc" in a project, because people tend to bundle thinks into them simply because they don't quite fit elsewhere - also makes it trickier to find specific things. I'ld suggest blender_addons could be moved out, and possibly moved into a separate project? The raw block shape blender files certainly can be moved out to a different project or some sort of repository for raw assets. The remaining items are all build related, so perhaps then utility could be renamed build-support?

• Any comments on capitalization in module names, facades, etc? Examples
  • https://github.com/Nanoware/Terasology (should pull up the correct branch)
  • https://github.com/Terasology/Core
  • https://github.com/Terasology/BlockNetwork

I'm fine with camel case. The engine should be case-insensitive when resolving uris, and the folder name isn't even used in those cases anyway. I don't know how case-insensitive retrieving them from git is - you may have to make the project names all lower case if it cannot cope with incorrect casing.

• Would it make sense to change the author field in module.txt to a list?

On the author field, could go either way - how many authors are we expecting a module to have, outside of a few core modules (which might just have Moving Blocks as the author)? It sort of depends on how we intend to use it in the UI and in the any module repository - at the moment it has a single line in the UI.

 

[Cervator]

Okay, some more organizing and polish done. Moved some stuff to TeraMisc but mainly got stuck trying to get the Gradle Plugin working inside IntelliJ - still no luck. Even with the Wrapper selected correctly the plugin doesn't seem able to fully parse the project setup (probably the crazy dynamic stuff). No tasks ever load within IntelliJ's JetGradle window and refreshing the project fails. Probably a similar reason why trying to open the project through the Gradle file (rather than with the generated IntelliJ files) fails.

Everything still works from the command line so the Gradle project is solid. Plugin works for the launcher fine so it must just be complexity. Maybe somebody else can check it and play with it a bit sometime - in the meantime having the plugin enabled doesn't seem to be hurting anything.

New line-up is just a tad bit more messy with the utility dir gone but that's probably just some sort of minor OCD presenting itself

 

[Cervator]

Focused on Jenkins today and its level of completion at this point is pretty solid for a first roll-out

I enabled code analytics / unit testing / etc for the engine job, which is going to be the updated Terasology job for develop. Also tweaked the pathing to go with the recent moves - which are actually reverts of old moves. Still need updates in Jenkins as the engine job attaches a usable Terasology.zip but also a little framework of build/config/artifact stuffs.

The sample BlockNetwork module build job grabs all that stuff so it can compile while only fetching its module source from GitHub. It uploads its artifact to Artifactory as a snapshot, as well as attaching it to the Jenkins job.

The zip on the main job again contains modules - listed as a parameter in the Jenkins job. Those are fetched directly from Artifactory.

PC Facade and Core module are in the main repo so those are bundled that way. They aren't sent to Artifactory anymore, nor is the engine jar. Unsure if they're needed there, but easy to add later if so.

I matched another Launcher piece by mkalb and enabled PMD for the fun of it. Also another static analysis. Lots of overlap but hey, graphs! Easy to tweak later.

With that I think I'm about out of stuff to procrastinate with. Next up goes something like this:

1. Extract modules from develop
2. Extract modules from multiplayer
3. Push Develop to Stable (last stable legacy build) + new branch "Legacy"
4. Push Multiplayer Stable (forced overwrite)
5. Change our published applet to come from the stable branch instead - leaves time to work on applet in develop
6. Branch a "restructure" from multiplayer, apply the actual restructure steps
7. Enable all the jobs and make sure it all works
8. Push Restructure to Develop (forced overwrite)
9. (Weeks later) delete multiplayer branch (leaving time for those based on it to migrate)

If nothing gets in the way this could be Monday or Tuesday evening. I'll announce on IRC when I am about ready to take a final cut from the multiplayer branch, in case somebody is actively working on something and want to push it first.

Any comments feel free

There is of course lots more coming and I'm going over existing posts here again and again to catch all the feedback so nothing gets lost.

 

[Cervator]

Small update: Ran short on time tonight. I did pull a copy of multiplayer and start, so I can update my step-by-step instructions, but I haven't pushed anything yet. Extracted one module as a test but leaving it at that for tonight. Likely the main branch I pulled will just be a final draft, I can redo that part very easily and quickly after finishing with the modules.

 

[Cervator]

Final draft is up:

Commits: https://github.com/Nanoware/Terasology/commits/FinalDraft (just one commit behind multiplayer branch, not counting PR #718)
Modules: https://github.com/Terasology (extracted all from multiplayer and tweaked)

I was resigned to leave it at this for tonight as I ran into a quirky reflection error, but only on running from the .exe. But then I also hit it in IntelliJ. But then it went away with a full rebuild. And later the .exe worked too

I thought maybe it related to a new reflection lib that made it into this version (was not in last time I pulled from multiplayer) but maybe I'm just seeing things. Perhaps a final quick review and test? Easy:

git clone git@github.com:Nanoware/Terasology.git
gradlew idea
gradlew distApp distModules distAppZip

IntelliJ is born with a "TerasologyPC" run config, although if you rush it the background compilation may not have finished yet. Try from there, gradlew run, and via facades\PC\build\distributions (then either the exe under app or extract the zip and try from there)

You can fetch all the modules as well, to make a module dist with extra modules you use the following:

gradlew distModules -PextraModules="BlockNetwork,Signalling,Fences"

Currently distApp, distModules, and distAppZip are not linked via task dependencies as the parameter approach above breaks compilation in the same execution - something about resolving artifacts out of order. The Jenkins job uses two different steps - easy enough.

I would retest from a fresh workspace but I can't even reclone it myself right now due to crappy internet

If it looks clear I'll do the needed updates in Jenkins etc tomorrow so I can overwrite the develop branch after re-applying from latest multiplayer

 

[Cervator]

DONE

Well, kinda done. The restructure branch is live under MovingBlocks based on the HEAD multiplayer commit

It builds in Jenkins and works. Mostly, anyway - it gets shipped with just BlockNetwork and Signalling right now, as I haven't made Jenkins jobs for the others yet, and due to a versioning quirk an older binary of Signalling is used that still asks for BlockNetwork 1.0.0+ while the shipped BlockNetwork has been set to 0.1.0. So you get to see the shiny version incompatibility error!

I haven't overwritten develop yet (crappy internet gave me a way late start again tonight) but this branch should be suitable for everybody to use for future development. When develop is updated pushing to there instead is trivial. In theory you could pull the restructure branch into a multiplayer workspace, it should be a fast-forward merge, but it is likely to clash if you have any changes in said workspace since src moves to engine/src. Doing that inside IntelliJ may also cause fisticuffs due to modules appearing/moving around/disappearing while IntelliJ tries to keep on top of things.

Probably safest to just clone into a new directory and go from there. gradlew fetchModule[name] should work for everything in the Terasology organization (better name?) but there are only builds & binaries for BlockNetwork and Signalling. You do not need anything other than "gradlew idea" or any one other task to get started, nothing mandatory to fetch.

I updated my handy dandy restructure issue tracker with what's left, goal is to finish off the First Unstable milestone along with the develop overwrite and the First Stable by when this code goes to the master branch. Feedback appreciated.

Pinging people that may have pending changes - if you submit them to multiplayer you'll make Gooey cry if I need to reapply the restructure again (I'm getting to where I can do that in my sleep though!) or you can do that and copy-paste them over at your convenience. There is effectively no functional changes between the multiplayer and restructure branches, only structural changes.

begla Immortius Skaldarnar x3ro socram8888 Jtsessions glasz nh_99

More Jenkins magic tomorrow and a news post then or closer to weekend with MOAR LINKS

Here's the step-by-step restructure for anybody nerdy enough to be interested (it was essentially done twice with the extraction skipped for round two):

1. Make new workspace with fresh clone, add and checkout target branch (multiplayer)

1. Mkdir​

2. git clone git@github.com:MovingBlocks/Terasology.git​

3. cd Terasology​

4. git checkout -b multiplayer remotes/origin/multiplayer​

5. git branch restructure​

6. git checkout restructure​

2. Move applet class

* mkdir target dir in command below​

* git mv src/main/java/org/terasology/engine/TerasologyApplet.java applet/src/main/java/org/terasology/engine/TerasologyApplet.java​

3. Move main PC class and create facades

* mkdir facades/PC/src/main/...etc​

* git mv src/main/java/org/terasology/engine/Terasology.java facades/PC/src/main/java/org/terasology/engine/Terasology.java​

* git mv launchScripts facades/PC/launchScripts​

4. git mv src/template/VERSION templates/VERSION
5. Move Core module

1. mkdir modules​

2. git mv mods/core modules/Core​

6. Commit checkpoint
7. Extract blockNetwork module

* Make a copy of the Terasology workspace (might not be needed with some Git magic, but it is easy!)​

* git filter-branch --prune-empty --subdirectory-filter mods/blockNetwork restructure​

* cd ..​

* mkdir BlockNetwork (note capitalization change)​

* cd BlockNetwork​

* git init​

* git pull file://E:/Dev/Terasology/Git/MultiRestruct/FinalDraft/TerasologyExtract​

* Create actual repo "BlockNetwork " under "Terasology" org​

* git remote add origin git@github.com:Terasology/BlockNetwork.git​

* git push -u origin master​

8. Extract signalling module

* git filter-branch --prune-empty --subdirectory-filter mods/signalling restructure​

* etc ...​

9. Extract more modules

* git filter-branch --prune-empty --subdirectory-filter mods/... restructure​

* etc ...​

10. Extract Applet

* git filter-branch --prune-empty --subdirectory-filter applet restructure​

* cd ..​

* mkdir FacadeApplet (to rename away from plain "applet")​

* etc ...​

11. Delete all the extracted stuff

* git rm -r mods​

* git rm -r applet​

* git rm -r rawAssets​

* git rm -r blender_addons​

12. Commit checkpoint
13. Move src into engine/src

1. mkdir engine​

2. git mv src engine/src (had to close ALL IntelliJs and/or Explorer instances)​

3. Also include the old root "libs" (contains tera-ovr.jar in multiplayer - should put into Artifactory later)​

4. git mv libs engine/libs​

14. Commit checkpoint
15. Update Gradle Wrapper files (from first restructure effort)

1. Simply copy existing files at gradle/wrapper and the root dir gradlew + gradlew.bat from latest restructure test to new dir​

16. Commit checkpoint
17. Copy in Gradle build scripts (from first restructure effort) + updated templates

* engine/build.gradle, facades/PC/build.gradle, the subprojects.gradle for facades and modules​

* build.gradle for modules/Core (from templates/modules.gradle)​

* ide.gradle, modules.gradle, settings.gradle (overwrite), utility.gradle​

* Also copy in updated .gitignore and fletch together any diffs if necessary​

* build.gradle (overwrite) - extensive changes first few times, but less each time​

* new templates directory​

18. Set IntelliJ back up and make sure all looks well

1. gradlew idea​

19. Adjust some Gradle stuff (and related) for the restructure:

1. We don't use codenarc (that's Groovy), but did introduce PMD - update the config dir from latest restructure draft​

2. Libs may have changed in the main build.gradle, should check and update the engine's build.gradle (new home for primary libs)​

20. Commit checkpoint (won't necessarily compile or run at this point)
21. Tweak actual code needs for the restructure

1. ExtensionModule needs "classes" to "build/classes"​

2. PathManager needs a few tweaks for pathing, compare to latest restructure​

22. Commit checkpoint (game should find and be able to run with the Core module now)
23. Test fetching modules back into the main workspace and do some related restructure tweaks

1. Add .gitignore files to existing modules​

2. Update capitalization​

3. Drop all versions to 0.1.0 (eventually will upgrade via Jenkins promotion)​

4. Adjust Signalling's dependency on BlockNetwork (case fix and version tweak)​

24. Checkpoint commit - and done!

 

[mkalb]

Cervator Your gradle task "ideaActivateCheckstyle" at the restructure branch has wrong values. Please look at the launcher build.gradle. I don't have time to make a Pull Request.

 

[Cervator]

Cervator Your gradle task "ideaActivateCheckstyle" at the restructure branch has wrong values. Please look at the launcher build.gradle. I don't have time to make a Pull Request.

Thanks - do you mean how

entry(key: 'property-1.samedir', value: 'config/checkstyle')

Had the value missing? And test classes was false?

I don't mind submitting the fix, if those are the two you mean. Unsure how that was lost.

Not really sure what that breaks either, I can run Checkstyle on a file within IntelliJ okay, it seems?

 

[Cervator]

mkalb - how's this?

https://github.com/MovingBlocks/Terasology/commit/478afb0647e3721d4d5f4fffce3b2a0e7a89df6f

I tested the null check and the XML handling is solid enough to do without, so I removed it. Can do to Launcher as well, but will wait for your approval

Immortius also hit an issue I didn't document right, there's a new "**" pattern in the .gitignore now which is a feature introduced in a recent version of Git. If you have an older version of Git the .iml files in the PC facade and Core module may not get ignored correctly. Just update to latest Git to fix!

Edit: I guess technically now IntelliJ runs Checkstyle with test classes included, while command line excludes them. Minor todo item for later

 

[Cervator]

Okay, I put in a temp fix for the main build not fetching the latest module snapshots, also helped to actually rebuild Signalling at 0.1.0 so the 1.0 wouldn't always qualify first

Deleted some old builds in Artifactory and redid the module jobs in Jenkins plus prepped jobs for the ones I hadn't done yet

Try out the latest build zip - I have reason to believe it should ship with all the multiplayer modules at the latest versions, but I still have internet trouble and most the time can't download much faster than if I had dial-up REALLY need to get that fixed. Luckily I can crawl through most browsing and light Git activity.

 

[Cervator]

Just did some fairly minor stuff tonight, got caught up in making pretty screenies.

• develop branch has been used to create new legacy branch, which has its own build job
• develop branch has been pushed to master one last time
• Immortius fixed a chunk thing I then backported to the multiplayer branch so that can be pushed to master tomorrow
• Applet now builds from the "legacy" branch - I figure for now better to showcase the old content rather than new multiplayer with less content. Can download to play multiplayer

Tomorrow

• Push multiplayer to master (overwrite), later delete obsolete branch
• Push restructure to develop (overwrite), later delete obsolete branch
• Update Launcher with any needed lineup changes
• News post
• Vikings marathon

 

[Skaldarnar]

Thanks for Vikings, perfect timing if you try to prepare for exams

 

[Cervator]

Okay, I've looked at the launcher some and tentatively replaced the multiplayer entry with a legacy entry in my personal fork. I cheated and took shortcuts though, would be good with some review and additional tweaks (like full renaming of multiplayer stuff and better German translations). I can run it and see the (empty) change history for the legacy branch.

• Noted a todo where I think we need to add "saves" as a new data directory in multiplayer
• Another where I'm not sure what to do with the expected location of the zip file - while it is the same name it is in a different path. Currently it seems to be a global variable - should we split it into a per-job variable?
• Maybe we should add a single commit (to the readme probably) to the legacy branch so it'll show up in the Launcher. I can do that tomorrow
• I added a series of build jobs for the appropriate branches under Nanoware/Terasology - maybe that would be useful for testing the Launcher itself as we can run those builds whenever and push test commits. Not sure how to best switch in and out of test mode in a situation like that
• I don't know how changing the launcher will impact existing installations, but we probably don't have enough of those to really worry about yet.

In other news I made the build.gradle under the Core module the template for other modules rather than have a second copy under the templates dir - trades some awkwardness for a slight bit of confusion. Also went back to a less pretty Git pattern to avoid the need for a newer Git version to support "**"

At this point I'm ready to overwrite develop & master, just hoping for some quick Launcher feedback - will work on a nice news post early tomorrow in the meantime

 

[mkalb]

Can you wait some days? I don't have time for a review/release today. Maybe monday or tuesday.
The launcher should be compatible with old versions/jobs.

 

[Cervator]

Can you wait some days? I don't have time for a review/release today. Maybe monday or tuesday.
The launcher should be compatible with old versions/jobs.

Sure, there's no rush, and no lack of other stuff to do in the meantime

 

[mkalb]

I have just a short time. I looked at your launcher changes and I try to do a commit to the develop branch of the launcher the next hour.

 

[mkalb]

The jenkins jobs TerasologyRestructure and NanoRestructure doesn't have the articat "versionInfo.properties" which is need by the launcher.

 

[Cervator]

Hmm, I did check on that at one point, it ends up in the main jar under org/terasology/version right? The TerasologyVersion.class is in there, but I don't see the .properties. I'll look into it, thanks

One trick is that both the version stuff and zip file moved - version stuff is inside the engine.jar now which is in the libs dir of facades/PC/build/distributions, rather than in a plain dist dir with just Terasology.jar. The zip file likewise ends up at that new path, with a near empty Terasology.jar purely holding the PC facade class (plain Terasology.java)

That's where I found one tricky spot in the launcher, which looks to expect the zip to be in the same place in every job option currently

 

[mkalb]

I pushed/merged the first changes ("legacy") to the launcher (based on your changes).

I am not familiar with the "restruct" branch/job.

 

[Cervator]

I think I can just move the dist stuff for PC back into the workspace root build dir. Was aiming to do tonight but it has gotten too late. Might still make sense to leave the version properties file in a new spot there, like right next to the .zip