Suggested Optional or supported modules for a module

Mike Kienenberger

Active Member
Contributor
Architecture
GUI
What do you think about adding a "supports" section to module.txt, to go along with "dependencies"?

As an example, the Crops module provides the support in the prefab to work with MovingBlocks, but if you don't load that module, an alternate system could be provided, such as the new Machines module where time + fertilizer might be required to advance to the next crop stage. As long as there's no conflict, multiple competing systems could be supported in a module.

Another example of this is Spawnable or CraftingRecipe optionally supporting Portals or the crafting module, or Minions supporting Oreons (the resources).

The support section would show up in the select modules description along these lines:

Miniions

[blah blah]

Supports:

ChangingBlocks - for growing crops without intervention
Machines -- for growing crops with fertilizer
Oreons -- as a minion type

Obviously this list would never be exhaustive, but it would give an idea of what is possible.
It could also give a module developer a clue about what other modules are potentially using their module when planning API changes.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I think you should just add this information in the description, since the support section doesn't have any functionality? Basically it is just a list of suggested modules that the user may wish to use with it.

I do think "Supports" may be confusing, because a user may assume anything that isn't listed won't work with it.
 

Mike Kienenberger

Active Member
Contributor
Architecture
GUI
Maybe you're right.

I was thinking supports might check the minimum version values like dependencies do, but maybe that's not really going to add anything since you still might want to use the "unsupported" module for another reason.
 

msteiger

Active Member
Contributor
World
Architecture
Logistics
What about looking at it from the other direction and create a directed, acyclic graph based on all module dependencies. In this graph, you can easily determine which module supports which others. Also, no explicit statement is required. On top of that, it can be computed automatically (no manual edit of module.txt) and it is therefore always up to date.

Sounds like a job for Jenkins! He keeps track of changes anyway and there's even a plugin for it. It maybe needs adaption to extract deps. from module.txt files, but it can even create nice diagrams on the web - easily accessible for everyone (scroll down to see the image):

https://wiki.jenkins-ci.org/display/JENKINS/Dependency+Graph+View+Plugin
 

Mike Kienenberger

Active Member
Contributor
Architecture
GUI
No, there's a difference betwen
What about looking at it from the other direction and create a directed, acyclic graph based on all module dependencies. In this graph, you can easily determine which module supports which others.
There's a difference between the is-dependent-upon (the reverse of depends-on) and supports.

Supports means that the module implements some bare minimum so that that works out of the box with a certain other module, but the other module has no dependency on it.

For example, Crops supports ChangingBlocks by defining the components needed for ChangingBlocks within each prefab. But ChangingBlocks can be used without Crops, and Crops can be used (with an alternate system or just as decoration) without Changing blocks.

In an ideal world, you would have a CropsForChangingBlocks middle module which has prefabs for all the modules in Crops and defines the components needed for ChangingBlocks. This module would depend on both Crops and ChangingBlocks. However, I think that defining these for every module relationship would make the list of modules unmanagable. I could be wrong, though :) And it'd be a lot of double-entry work to define each crop in both modules as well as to maintain them over time.

There's probably no simple solution at this point, which is why I thought adding supports would at least be helpful.
 

Linus

Member
Contributor
The functionality seems useful, but I think it would make more sense to reverse the relation.

So instead of
Miniions supports Machines -> Allows miniions to operate machines -> requires some machine code within Miniions mod​
you'd get
Machines is enriched by Machines Miniions -> Allows miniions to operate machines -> requires some miniions code within Machines mod​

This way the mod that profits from using the API, is responsible for using it correctly and also for adding any additional code that may be required.
Or is this how you envisioned it?
 

msteiger

Active Member
Contributor
World
Architecture
Logistics
I completely agree - that's exactly what I meant (that plus a fancy drawing generator ;)).

Note: I assume the second line should be
Machines is enriched by Miniions
 

Mike Kienenberger

Active Member
Contributor
Architecture
GUI
I think it would make more sense to reverse the relation.

So instead of
Miniions supports Machines -> Allows miniions to operate machines -> requires some machine code within Miniions mod​
you'd get
Machines is enriched by Machines Miniions -> Allows miniions to operate machines -> requires some miniions code within Machines mod​
I think you are missing the point. You are arbitrarily deciding that the Machines author MUST support Minions.

The reality of this example is that either module author could decide to support the other, and there's no right answer as to who should do it.

Let me give you two examples, mostly because I already wrote one out on Crops before I rewrote to work with Machines :)

In an ideal world you would have a joining module like CropsForChangingBlocks when two systems have no dependencies on each other, but could be tied together.

But like I said, it's going to be more practical to avoid a middle module, and add Components to prefabs to optionally allow existing known modules to use the module as is. And where that support is added is probably going to depend on the module authors involved.

Metouto could say "Hey! This is a pure art module of plants! I don't want a dependency on how the blocks are display"

Mike could say "Hey! This is a pure behavior module. I don't want specific stuff in here about plants blocks!"

In that case, you'd probably need to go with a middle module.

But Mike could remain a purist, and Metouto could add the ChangingBlocks components to the plant prefabs. (This is how I did it for Crops).

Or Metouto could remain a purist, and Mike could create child plant prefabs in ChangingBlocks (This is how I did it for Minions based off the Oreons art).

Neither way is "right" ("right" would be creating the middle joining module) as there's no definite right place to put the information in either module.
Rewrote to use Machines instead:

In an ideal world you would have a joining module like MachinesForMinions when two systems have no dependencies on each other, but could be tied together.

But like I said, it's going to be more practical to avoid a middle module. And where that support is added is probably going to depend on the module authors involved.

Josharias of Machines could say "Hey! This is a pure for building machines! I don't want a dependency on how machines are used by minions"

Mike could say "Hey! This is a pure behavior module for controlling minions. I don't want specific stuff in here about operating machines!"

In that case, you'd probably need to go with a middle module.

But Mike could remain a purist, and Josharias could register the behavior-containing components.

Or Josharias could remain a purist, and Mike could register the behavior-containing components.

Neither way is "right" ("right" would be creating the middle joining module) as there's no definite right place to put the information in either module.
This is why there is no "direction" in a supports statement.

The difference between "supports" and "is enriched by" is semantics.

If I say that Minions supports Machines, it means that Minions is enriched by Machines, and that I have added something to Minions to make it work better when Machines is active.

If I say that Machines supports Minions, it means the Machines is enriched by Minions, and that I have added something to Machines to make it work better when Minions is active.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
I was just thinking we needed to have this discussion, and then it popped up while I wasn't able to keep up with the forum for a few days :D

Crops was my example module as well, wondering if we'd put ChangingBlocks definitions into it or if we'd have a farming module (a "middle module") that would depend on both and hold the linking.

Optional modules was the way in which I was thinking about it earlier but I like "enrich" or "supports" as well. Really we can cram extra stuff into prefabs without including a hard dependency on whatever we're referring to - but IMHO that feels "wrong" somehow and I imagine that's what this is trying to address. Which seems to mesh well with the initial post's lack of hard functionality from including "supports" info

Semantics aside what do we put into a convention about what to put where when something can bind functionality from different modules together, yet isn't a solid dependency? Crops also have potential food/drink values to the Hunger module, for instance. To me it feels right to put the definitions into Crops (maybe including some prefabs for items that are harvested from end-stage plants), but it also feels right to formalize that relationship somehow, more than a simple description entry, even if it isn't a hard dependency.

If including an ideal version number where it was tested that would allow some functionality to prevent crashes if optional food/drink data in crops was added before some major changes were made to Hunger making the definitions invalid. The module could attempt to enable but either automatically disable if errors are detected, or if the user checks some "only allow optional functionality with exact version matches" (probably based on SemVer characteristics)

(edited for sleepiness)
 

Linus

Member
Contributor
...
If I say that Minions supports Machines, it means that Minions is enriched by Machines, and that I have added something to Minions to make it work better when Machines is active.

If I say that Machines supports Minions, it means the Machines is enriched by Minions, and that I have added something to Machines to make it work better when Minions is active.
Ah okay, I see were I went wrong now.
I somehow got the idea that the supports tag worked as follows:
  • Machines adds code to make Machines work better with Minions
  • Minions adds supports tag to signify the relation
But you actually meant
  • Machines adds code to make Machines work better with Minions
  • Machines adds supports tag to signify the relation
The reason I wanted to reverse the relation was to have the code and tag at the same end of the relation,
but now I see that this was already the case. :)
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Arise from the grave, O' ancient thread! ARISE!

This has come up again from time to time since this thread was created in the Dev Portal. I just moved it to Suggestions to restart the discussion since we did have a relevant thread already.

We even have the area partially improved, like an "optional" attribute in gestalt-module and some more options via overrides and deltas to emulate optional dependencies. Question is how much further we should aim to improve the area?
Slack said:
cervator [00:38]
@xtariq and @jellysnake could you two weigh in on https://github.com/MovingBlocks/Terasology/pull/2767 please? it is about optional modules which i figure might relate to you two (content focus + inventory fun)

[00:39]
in particular maybe coming up with some use cases that could demonstrate the usefulness of the system

xtariq [01:06]
Intersting...

[01:07]
I'll take a deeper look at it tomrrow.

jellysnake [08:45]
So, I'll add my thoughts onto the pull request (and/or the issue) but whilst it looks good, I'm having issues thinking about when I'd be used. I think it stems from simply, that having the ability to conditionally register a system based on modules isn't enough.
What I mean by this is that if I'm going to have a system that I only want to work when module `foo` is activated, I'm also going to be listening for an event from that module as well most likely. Or I'll be reading from a component defined in it. Either way I need to be able to access the code of that module without requiring it to be there.
Right now if I try that IntelliJ gives me a bunch of errors (and rightly so) about "Cannot resolve symbol" and the compiler is annoyed about "cannot find symbol"
I'm not sure if this is even possible, I'm no java wizard, but for optional dependencies to work that's the level of functionality I'm imagining.

[08:47]
(I think I'm gonna post onto the issue and mention it from the pull or something)

[08:49]
(however I'm gonna do it in the morning when I wake up) (edited)

skysom [08:50]
Yeah I'm not sure how I can see conditional registering systems is a thing.

[08:50]
Like recipes and additional minor content based on certain modules makes sense.

[08:50]
Systems? eh.

jellysnake [09:10]
I'm not 100% sure if conditional systems is the way to go but it seems to work better than alternatives when I think about it.
For instance, say you made a module that adds a bunch of machines to do things. And say I had a module that gave you magic abilities. If I decide that I want to be able to turn on and off your machines with my magic module how would you do that. I don't want to have to always depend on your module. But I want for instance to be able to access a component from your mod to turn it on or off. If you had a module that wired them together then that seems way to superfluous for a simple one off thing.

[09:12]
It may be that my thinking is flawed on some level or that I'm missing an easier way to do this but I'm not java wizard so I'm not sure of an alternate way

[09:13]
(unless of course the terasology way is to split things into small modules as far down as is reasonable.)

skysom [09:15]
I mean for me, adding modules as compile dependencies isn't a huge thing for me...

[09:15]
As I'm used to doing it in MC

skysom [09:15]
looks at his 10 maven dependencies...

jellysnake [09:18]
What do you mean by compile dependencies?

skysom [09:45]
I mean I guess that's what I thought your issue with dependencies was

cervator [10:00]
one "optional" thing that works right now is referring to non-existing components in prefabs

[10:01]
so you could create some prefab with a component in it from an optional module, and that component will just gracefully fail to attach if the module isn't active

[10:02]
yet if it is then the component will be present and whatever systems and events from that module will be live and able to react to the original entity

[10:02]
we've also got occasional "bridge" modules

[10:02]
i'm not sure what the ideal level of support is

[10:04]
once upon a time the `BreathingComponent` was attached in the `player.prefab`, but only triggered if the Breathing module was active. That itself wasn't super clean since as a Breathing dev you'd need your thing attached to the original `player.prefab` ... later on we then got delta prefabs and various on-spawn events and such to where Breathing could be entirely self-contained again

[10:04]
i dunno how much of the interest in the topic actually originate from days when we needed it more

xtariq [12:42]
Alright, I'm awake and ready again.

xtariq [13:40]
Optional components is what I would use optional dependencies for.

cervator [14:48]
one question is whether we have enough hooks now to do without somehow. Can we structure modules in a way where you don't *need* a formal system for optional dependencies?

skysom [14:52]
works in MC

[14:53]
Most of my code is just locked behind classes that only get called if
```Loader.isModLoaded("<modid>");```

[14:53]
returns true

cervator [14:58]
that might be why some coders would expect to find the functionality available. But then MC mods are structured somewhat differently. I wonder if we could sneak around it using some better techniques - may be hard to know early on

skysom [14:59]
I'd be cool with a simple side module system

[14:59]
But yeah.

[14:59]
MC mods are structured oddly indeed.

[15:00]
But yeah that'd be why it's sorta expected

[15:00]
But there are some times (Like I said recipes) where I'd only want to do something if that module was installed

[15:00]
So I'm not sure the best way to handle that.

cervator [15:03]
it seems like using overrides/deltas is a powerful tool

[15:03]
is that at all a thing in MC land?

skysom [15:04]
HAHAHAHAHAH

[15:04]
What little I've heard of them?

[15:04]
Not even close

[15:05]
But yeah there isn't really anything other than what I just said to deal with other mods.

[15:05]
So I'm just gonna assume that we don't have those.

cervator [15:16]
interesting

skysom [15:28]
Mind explaining Overrides/Deltas? I've heard bits and pieces, but never a full explanation

oniatus [15:41]
@SkySom https://github.com/Terasology/TutorialAssetSystem/wiki/Deltas-and-Overrides

skysom [15:41]
Oh hey sweet

[15:43]
Yeeaaahh MC ain't got anything close to that

[15:43]
But it could be more cause we don't have any type of component system

xtariq [15:44]
There's also another nice feature of deltas, where depending on the module load order, you can replace individual components of prefabs. (edited)

skysom [15:53]
But yeah @Cervator we don't have anything like that...

[15:53]
Basically you either replace something direclty with your version

[15:53]
Or you just suck it up

cervator [16:00]
yeah there's a reason we expected a more extensible system could be done from scratch :smile:

[16:00]
now i just wonder if its inherent benefits negate the need for optional modules entirely

skysom [16:02]
I still feel like you could

[16:02]
unless you want to start writing glue modules.

xtariq [16:04]
We'll eventually need branch modules.

jellysnake [17:47]
Okay, so the way In Game Help works at the moment is that modules like Cooking have to depend on it and then listen to an event from it. In Game Help is a perfect candidate for being an optional module.Cooking does not need it at all, but at the moment, it's required.
How would you listen to an event from an optional dependency with in terasology at the moment?

[17:49]
Because adding a component to the prefab containing all the info needed for In Game Help wouldn't work because
a) you'd have to manually update it every time content was added
b) You'd have to reference the component from Cooking, again leading back to the same issue as before in that you can't without all kinds of funky errors

cervator [18:16]
nice example @jellysnake :slightly_smiling_face:

[18:17]
i try to think of it all as a big tree, engine is the trunk, libraries the roots

[18:17]
modules make up the branches and get finer and finer in details the further away you get ("leaf" modules with most the actual content/config)

[18:18]
as @xtariq developed the effects system starting from what was left in TTA we added a bunch of assorted effects to it

cervator [18:24]
as we kept adding some general types and other stuff became more and more visible

[18:24]
and some really generic stuff seemed a candidate for pushing closer to the engine (damage types and registering new ones)

[18:25]
so one outcome there could be pushing some generic stuff closer to the trunk so it is easier for more stuff to get it from the dependency chain naturally (no optionals needed) and more detailed stuff further out to prevent the main relevant module from becoming too bloated

[18:26]
that way keeping everything really slim and minimal might make it overall easier to work off regular dependencies without getting too much fluff you don't want

[18:27]
but yeah if for some reason you wouldn't want a help system enabled (hard core / primitive survival) it should definitely be possible to turn it off, even if it is something closer to the trunk

[18:28]
maybe there would be a way to "push closer" how the help system exposes information, much like how damage type *registration* at least should be pretty close to the engine

[18:28]
make the leaf modules enable more creative damage types tho

jellysnake [18:33]
That works for things that are a natural extension of where they come from, but again the issue is what if you want to go across branches. Ie, an magic mod with damage spells would make sense to depend on say alteration effects or damage types. But if that magic mod also thought it'd be neat to be able to control machines from another mod how would it do that?

[18:33]
A machines mod and a magic mod are separate branches of the tree, it makes no sense to have one depend on the other

cervator [18:33]
yep it doesn't seem difficult to come up with use cases, we just don't have a ton in action right now to play with. InGameHelp is a good one

[18:35]
it is tempting for me to think about sort of reversing direction on the help system and instead of having Cooking depend on InGameHelp instead smarten up IGH to where it will scan for *anything* with valid data of some sort it can display within itself, no dependency needed - but that probably violates the spirit of the ES and not leaving "inactive" extras around (like better decorating prefabs / recipes / components in Cooking)

[18:36]
although in that case there wouldn't be any cross-module logic in action

jellysnake [18:36]
See, there are a bunch of other work arounds but you'd need different ones for each use case, and it's just be so much simpler to have option systems that only work when a module is activate _and can access the code of the optional module_

[18:37]
That last bit is the issue. I can't think of a way to do that. You can't not compile it, because the modules are compiled prior to when they know if the module is going to exist or not.

[18:37]
and you can't compile it anyway because the compiler spits out errors about not being able to find the stuff from the other module

[18:38]
(I've got to go but that outlines my general thought process around optional modules

cervator [18:41]
thanks :slightly_smiling_face:

[18:41]
i wonder if we could draw some inspiration from the `compile` vs `runtime` vs `provided` dependency settings in Gradle

[18:43]
so for building code (and developing) optional dependencies are mandatory, but at runtime they're not. Although how that compares vs. just using feature flags ... eh. Purists might prefer not having the code present if it is optional

[18:44]
and i guess it would still need some clever code to avoid bumping into unavailable code at runtime (much like the little MC example @SkySom posted)

[18:45]
for fetching source and binaries for development there could be an option to get transitive dependency modules as well, all as source, only mandatory as source, or all as binaries beyond the target top level module

[18:45]
(well, any dependencies, regular or 2nd, 3rd, etc level)
 
Top