Future GUI - Reloaded

x3ro

Member
Contributor
GUI
This is the continuation of the "Future GUI" (http://forum.movingblocks.net/threads/future-gui.248/) Thread, which has gotten quite long and mazy, and hasn't been updated for a while.

A rewrite of the current GUI system is our main goal, mainly in order to make the GUI more "moddable", i.e. easier to customize. The consensus in the previous thread has been that this will be done using a JSON-based GUI definition system, which features three main components:

  • Structural Definition
  • Style definition
  • UIElement implementations
Read all about these here (not copying into the thread since the formatting on gist.io is way better :D). Please please read that thing and tell me if I missed stuff or anything sounds fishy, as that will be my main (only?) reference when implementing, and I'd like all involved people to be okay with the draft :)

(I will expand that gist as the discussion continues, so it will contain the most up-to-date information. I intend it to be merged into the project documentation once the GUI rewrite is completed).

My next steps:

  1. Wait for Cervator to restructure the codebase, so that it is safe to start writing code again.
  2. Analyse our code. Understand in detail how does the GUI works (I've already done this once, but since that was quite long ago I'll have to do it again in part). I compiled all existing GUI screens here: http://cl.ly/image/2s162r1p1t17 (if I missed something, please tell me :)
  3. At the same time I will adapt my old prototype to the new structure and changes that have been made since I wrote it.
  4. Profit!


Prototypes made so far


Further inspiration


Pinging all people previously involved

ironchefpython Cervator Adeon Immortius woodspeople Kai Kratz miniME89
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I've been thinking about the UI a bit recently - some rework is going to be required for TGC anyway. Some thoughts:

* Firstly the UI needs to be changed so it doesn't use LWJGL directly. What I have in mind currently is a system where a canvas object is passed through the the UI elements during rendering. This canvas would provide the rendering primitives like drawTexture, drawText and drawMesh. Additionally it could provide transformation and cropping abilities (so a window would crop the canvas so the available draw space is the inside of the window, with 0, 0 as the top-left corner).

* The UI system should be usable and fluent programmatically. It needs a good clean up with better defaults and all the methods that take strings where numbers should be used need to be replaced.

* The UI system needs to be better integrated with the entity system - it needs to be eligible for injection, and probably event receivers too. However, primarily it should use

* Entity data binding. A health bar element for instance would be provided with an entity and a binding string like "engine:health.currentHealth", and should automatically update by reading the health field of the bound entity, if any. This can be supported through the component metadata system.

The reason to use data binding rather than events is many events won't be available client side.

* The json system would then be a system on top of all this. It certainly shouldn't be worked on until the more fundamental aspects are addressed though. I'll post later with some more thoughts on it specifically.
 

x3ro

Member
Contributor
GUI
Firstly the UI needs to be changed so it doesn't use LWJGL directly.
Thanks for your feedback Immortius. I agree that the UI should not use LWJGL directly. If I understand you correctly, what you propose would require a ground-up rewrite of the entire UI though, and I think that the UI customization "framework" can be implemented first. Bottom-line, the only thing said system does is set properties on classes (and perhaps create a few, for the Layouts). If we agree to keep the basic structure, i.e. have a UI element base class from which all UI elements inherit and so on, it should be possible to do your proposed rewrite after the customization system is in place.

The UI system should be usable and fluent programmatically
Very much so. And since the customization system doesn't care about the actual API (the "styles" basically call setters, thats it), a re-design of the current UI API (which would be necessary for the proposed non-LWJGL rewrite anyway) does not conflict with what I want to do, or so I like to think.

The UI system needs to be better integrated with the entity system
Could you expand on these? I am not familiar enough with the entity system, so it'd be great if you could provide some examples of how injection, event receivers and data-binding works in it.

I agree that data-bindings should be used, by the way ;)




The reason I'd like to get the UI customization work done as soon as possible is because I will likely get dragged back into university and work projects at mid- or end of september, and I might get lost in real-life for another year or so :p
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Ok, on the json system.

The approach I would take here is:
* The UI classes defined in engine and modules would be widgets. These would have uris to refer to them, and their properties that can be set through json would be annotated. A simple widget would be a button, a more complex widget might be a radar.
* A layout defines the contents and structure of a container - which may be a window or full screen overlay. These contents would either be widgets or other containers. Contents may have an id, which would allow it to be accessed (for linking an event to a button for instance).
* Styles provide appearance information, like button textures, fonts, colors, borders. These correspond to exposed properties of widgets. Some will be specified directly against the widget types - these are defaults. Others would be a combination of type and hint. In the layout each bit of content would optionally have a style hint, and the overall layout could have a default style.

* Finally, to use the UI the coder can instantiate a layout, optionally overriding the style. This UI can then be programmatically altered, injecting extensions as desired.

That is the sort of thing I would have in mind anyhow.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
If I understand you correctly, what you propose would require a ground-up rewrite of the entire UI though, and I think that the UI customization "framework" can be implemented first. Bottom-line, the only thing said system does is set properties on classes (and perhaps create a few, for the Layouts). If we agree to keep the basic structure, i.e. have a UI element base class from which all UI elements inherit and so on, it should be possible to do your proposed rewrite after the customization system is in place.
I'm not sure I can even agree to that :p. Since I'm presently free of multiplayer rewrites and massive merges, I'll prioritize a review and get back to you on that asap. You are probably correct though.

Very much so. And since the customization system doesn't care about the actual API (the "styles" basically call setters, thats it), a re-design of the current UI API (which would be necessary for the proposed non-LWJGL rewrite anyway) does not conflict with what I want to do, or so I like to think.
That is sensible. This might be an area where the ClassMetadata system used for components and events would be useful, since it provides functionality around enumerating available fields and setting values through field names (going through setters/getters where present). This will also mean when it is optimised to use runtime code generation rather than reflection you would get that optimisation too.

Could you expand on these? I am not familiar enough with the entity system, so it'd be great if you could provide some examples of how injection, event receivers and data-binding works in it.
Injection uses an @In annotation to fill in properties automatically from the CoreRegistry. e.g.

Code:
public class MyGUIWidget implements UIElement {
 
@In
private EntityManager entityManager;
}
Would ideally be automatically populated when the UIElement is initialised by being attacked to a container which is attached to the GUI. (or something like that)

Events use an @ReceiveEvent annotation to receive events sent to entities with a desired combination of components. This probably isn't so important for the UI system, but it goes like this anyway:

Code:
public class MyGUIWidget implements UIElement {
 
@ReceiveEvent
public void onDamaged(OnDamagedEvent event, EntityRef entity, CharacterComponent character) {
  // Is called whenever an entity with a character component is damaged.
}
 
}
Data binding isn't actually a thing at the moment (beyond potential supported by the entity system), but it would work through the previously mentioned ComponentMetadata system - this provides information on what components there are, what fields they have and the accessors for those fields.

I imagine it would go something like this:

Code:
public class MyGUIWidget implements UIElement {
 
    @In
    private DataBindingHelper bindingHelper;
 
    private EntityRef boundEntity = EntityRef.NULL;
    private String binding = "engine:HealthComponent.currentHealth";
 
    // Getters and setters for bound entity and binding (which could be set through json)
 
  public void update(float delta, Canvas canvas) {
      int value = dataBindingHelper.getInt(boundEntity, binding, 0); // The last value is the default if the entity doesn't have the correct component or is otherwise wrong.
      // update appearance for current value
  }
 
}
DataBindingHelper behind the scenes would split the binding string into the component uri ("engine:HealthComponent") and field ("currentHealth"), look up the metadata on engine:HealthComponent which will give it the class of the component, look up the currentHealth field in the metadata which provides access to get the value, and then use this information to get the component and the value of the field of the component. If there are any problems along the way the default value would be returned. In the background this all uses reflection at the moment, with as much work done upfront as possible, but it is well encapsulated so can (and hopefully will) be replaced by runtime code generation during an optimisation pass.

Edit: To clarify, an entity is a container of components - up to one of each class. So a entity may have a character component, health component, location component and light component. An entity cannot have 2 light components though.
 

Skaldarnar

Development Lead
Contributor
Art
World
SpecOps
Hehe, nice to see you again x3ro :)

Just wanted to ping chridal here since he tinkered a bit with crafting/assembling and some rough GUI mock ups.
 

x3ro

Member
Contributor
GUI
So, at last I've reached a "presentable" state for the GUI system modifications. It is currently possible to define UIWindows via JSON defintions, including its child widgets as well as the default styles. It is also possible to specify a controller for a menu, which is in charge of registering all necessary event handlers (I'd like to improve how this currently works, and I will post with more detail in the next few days). I've recreated the main menu as a JSON defintion, which works flawlessly (I've not yet "ported" all the events).

My next steps will be to update the docs to reflect the recent changes, discuss/implement feedback from everyone who can make time to review my changes, think about how I can improve unit test coverage. Big thanks to Immortius and Cervator who've been providing me with on-demand information on some of the more intricate details of the Terasology codebase :)
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Looks good to me just from a superficial look - I'll have to leave the more detailed architectural review to Immortius :)

I will say there'll be some comments about checkstyle violations though, for stuff like this :D

Code:
if(!el.getId().equals(id)) { continue; }
You might want to try getting set up in IntelliJ or what not so you can see the warnings, they're easy to fix. I figure you have more edits to apply, but anyway :geek:

I like how the main menu json + main menu controller look, seem very easy to understand. And the test window is quite festive! So just to confirm: works for me :thumbsup:
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I'll have a detailed look at it shortly (want to get module versioning done first).

Edit: One comment - generally a class like "Tuple2" (often called Pair) is considered bad practice in Java, because it doesn't convey any semantics about the contents or their relationship. You would likely be better off creating a small custom class to return two values if needed. I'm not convinced it is even needed in the one case your using it though - module manager could be a parameter into the method which could produce an asset manager.

Also, the method and variable naming in the Tuple2 class is horrifying. :p
 

x3ro

Member
Contributor
GUI
I basically used Scala's Tuple2 and "ported" it to Java. I find it strange that such a generic class could be considered bad practice, because what you suggest seems to be to create a class "AssetManagerAndGuiManager", and if I'd need to return two values somewhere else, "FooAndBar" (please correct me if I'm wrong). That'd be like creating a class "AssetList" extending ArrayList (or whatever) for when I need a list of assets. That just doesn't make any sense to me, but then again I'm familiar with of the Java best practices and their reasons. It is true that using Tuple2 does not convey any semantics about their relationship, because the relationship is totally irrelevant when simply returning values. Tuple2 shouldn't be used as an instance field or basically anything with a longer lifetime than "return, get contents, discard".

The method naming is inspired by Scala, whereas the variable naming is simply pragmatic (and a little strange, I agree), because Tuple2 is unlikely to be changed ever again (and shouldn't be, actually).

Don't take this the wrong way, I'm not trying to defend my position, I'm just stating the reasons I took for adopting it (that is, I didn't randomly do the stuff I did). :)
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Don't take this the wrong way, I'm not trying to defend my position, I'm just stating the reasons I took for adopting it (that is, I didn't randomly do the stuff I did). :)
I understand. I take an active interest in coding practices, patterns, anti-patterns and so forth - often when I'm researching something I'll bump into an argument about something like this. I cannot remember where I first came across discussion on this subject, but here are a couple of similar ones.

http://groups.google.com/group/comp.lang.java.help/browse_thread/thread/f8b63fc645c1b487/1d94be050cfc249b
http://tech.puredanger.com/2010/03/31/do-we-want-a-java-util-pair/
http://blog.nfnitloop.com/2011/03/scala-vs-java-anti-pattern.html

The last one may be of particular interest because it is talking about how Scala lets you avoid using Tuple in the first place, which makes me wonder why it has Tuple. (Incidentally a good IDE like IntelliJ makes generating a container class fairly simple, since the getters, setters and contructor can all be automatically generated with a few key presses).

Anyway, the principle here is partially around methods having a strong, single purpose and partially around implicitly providing information on the usage of results. If a method is returning two things, they should be naturally related - a method that resolves a DNS entry may return an ip and a port, which naturally go together as an IPAddress. Otherwise it suggests the method is doing two unrelated things, and it should in fact be two methods. Something returning an AssetManager and a GUIManager might actually be two methods, the first setting up an AssetManager and returning it, the second taking an AssetManager and returning a GUIManager. Or perhaps these classes need to be refactored so that they are more independent.

On using the results, if a method returns a tuple it doesn't really tell you what the result is anymore - you have some object with a couple of things in it, you might know what type they are and if you are lucky they are different types. Sure the doco for the method will say, but that doco is off somewhere different from where the method is being used. The best documentation, the best way to make code understandable, is if it reads as naturally as possible just from using it (I'm still working on this). It also can make code less prone to error as well:

Code:
    Tuple<String, Integer> ipAddress = getIpAddress();
    new URL("http://" + ipAddress.getSecond() + ":" + ipAddress.getFirst());
vs

Code:
    IPAddress ipAddress = getIpAddress();
    new URL("http://" + ipAddress.port() + ":" + ipAddress.getIP());
The second form makes it far easier to pick up errors from simple inspection of the code. Better, you are far less likely to make the mistake in the first place.

On a slight tangent, this can also be a problem with method parameters. Something like:

Code:
    public Date(int day, int month, int year);
is susceptible to parameters being provided in the wrong order (e.g. "new Date(9, 30, 2013)"). A rather clever way of dealing with this is through marker classes:

Code:
    public Date(Day day, Month month, Year year);
Where Day, Month and Year may just wrappers around an int (so you would do "new Date(new Day(3), new Month(10), new Year(2013))", though you can certainly do better) - this allows the compiler to prevent misordering. This can be rather cumbersome in general practice though. Still a useful trick in some cases - I used it for Terasology's Rotation class. At any rate you would have to do something for the date constructor - using an enum for Month perhaps. Or at least have a static constructor method like Date.createDayMonthYear() that clarifies the correct order.

I basically used Scala's Tuple2 and "ported" it to Java. I find it strange that such a generic class could be considered bad practice, because what you suggest seems to be to create a class "AssetManagerAndGuiManager", and if I'd need to return two values somewhere else, "FooAndBar" (please correct me if I'm wrong). That'd be like creating a class "AssetList" extending ArrayList (or whatever) for when I need a list of assets.
I would say List gives some semantics around its contents (that is, that it is an ordered collection of objects of a given type) . That ordering might be important, but you wouldn't generally expect the first element to be used for one thing, and the second element a completely different thing - they are generally homogenous.

A better argument could be made around Vector3f - we return this for both position and velocity for instance, two things with very different semantics. Indeed, in more a scientific system you these should probably be different types, and interactions between them would be better controlled - position + velocity would not be allowed, first you would need to convert velocity to a displacement by multiplying it by time. We sacrifice that safety for the benefit of simpler, more generic code, and we reap the cost of the errors that result (we also partly just use Vector3f because that is what the physics engine uses and the interop is convenient). That is the same trade off that Tuple2 represents, but given the low instance of methods returning multiple values it is one where I think we can pick the side of safety.


On the style... methods in standard Java simply don't start with underscores. Even if it only needs to be written once, every time someone wants to use the class they'll have to deal with this. getFirst() and getSecond() would be the more natural java method names.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Some more comments:
  • I see you added a new ignore to the .gitignore file, can you sort it into a category like the others?
  • On the asset factory for layouts - I would have the GUI Manager register it, probably. In theory what should happen is the implementations of core systems like the AudioManager, Renderer and so forth should register the factories that produce the asset subclasses for their implementation of the system. So the OpenGL renderer would register an Texture asset factory that produces OpenGLTextures. Doesn't matter so much for layouts since GUI should ideally be implemented over the renderer in an implementation agnostic manner.
  • No real comment about the changes to GUIManager - the opening/creation of windows needs a complete overhaul anyway. I will point out the the goal isn't necessarily to replace all windows with json assets though. In some ways the json assets aren't actually that much of an improvement over straight code in of themselves - there would need to be a layout editor to create them and test them at different resolutions for to realize their full advantage. Otherwise it is just one text-based system (code) vs another (json) for making a visual display - the weakness of both is you cannot visualize what is being produced without running the game. Pure code as the advantage that it can be changed at runtime to immediately see changes to the layout - json can have this too if written so that it can take advantage of asset reloading.
  • Not sure I like the LayoutDefinition working directly with UI classes - it ties them to a specific implementation when part of the point of the AssetData classes is to be an agnostic object between potentially multiple file formats and multiple implementations. But as I said earlier the UI classes should be agnostic anyway, so probably doesn't matter. And it may well be the easiest way to approach this without changing the UI classes. Don't know why all the fields are public though, and there is no point having final Strings that aren't static unless they can vary during construction.
  • On this code:
Code:
new AssetUri("style:engine:" + clazz)
Firstly, AssetUri has a constructor taking the parts you can use if you don't have the entire string. Secondly, why are you restricting yourself to the engine module? In particular, you can use the new asset resolution support to allow a style to be either a simple uri ("engine:class") or just the name ("class") which can be resolved in the context of the asset (considering its module and dependencies only).
  • I'm not sure of the UIControllers. Specifically I'm not sure they should be specified in layouts, as it limits their reuse and is not a layout concern. Even within the main menu we reuse the level selection screen between Singleplayer and Multiplayer. I would say that instead the controller should be attached by the code instantiating a layout. In fact, there may be no need for UIControllers at all - instead the code instantiating the layout can just registered listeners directly with the relevant widgets, looking them up by id.
  • I think StyleApplicator should be replaced with something using the ClassMetadata system ideally. We don't need a second system for applying properties to classes - they should work the same way as prefabs.
 

x3ro

Member
Contributor
GUI
Some more comments:
  • I see you added a new ignore to the .gitignore file, can you sort it into a category like the others?
  • On the asset factory for layouts - I would have the GUI Manager register it, probably. In theory what should happen is the implementations of core systems like the AudioManager, Renderer and so forth should register the factories that produce the asset subclasses for their implementation of the system. So the OpenGL renderer would register an Texture asset factory that produces OpenGLTextures. Doesn't matter so much for layouts since GUI should ideally be implemented over the renderer in an implementation agnostic manner.
  • Not sure I like the LayoutDefinition working directly with UI classes - it ties them to a specific implementation when part of the point of the AssetData classes is to be an agnostic object between potentially multiple file formats and multiple implementations. But as I said earlier the UI classes should be agnostic anyway, so probably doesn't matter. And it may well be the easiest way to approach this without changing the UI classes. Don't know why all the fields are public though, and there is no point having final Strings that aren't static unless they can vary during construction.
I'll look into that.
  • No real comment about the changes to GUIManager - the opening/creation of windows needs a complete overhaul anyway. I will point out the the goal isn't necessarily to replace all windows with json assets though. In some ways the json assets aren't actually that much of an improvement over straight code in of themselves - there would need to be a layout editor to create them and test them at different resolutions for to realize their full advantage. Otherwise it is just one text-based system (code) vs another (json) for making a visual display - the weakness of both is you cannot visualize what is being produced without running the game. Pure code as the advantage that it can be changed at runtime to immediately see changes to the layout - json can have this too if written so that it can take advantage of asset reloading.
You make it sound like you don't really think that the entire change is worth it in the first place, which I find a little discouraging, considering the work I've already put into it.
  • On this code:
Code:
new AssetUri("style:engine:" + clazz)

Firstly, AssetUri has a constructor taking the parts you can use if you don't have the entire string. Secondly, why are you restricting yourself to the engine module? In particular, you can use the new asset resolution support to allow a style to be either a simple uri ("engine:class") or just the name ("class") which can be resolved in the context of the asset (considering its module and dependencies only).
Probably because I didn't know better. Will fix.



  • I'm not sure of the UIControllers. Specifically I'm not sure they should be specified in layouts, as it limits their reuse and is not a layout concern. Even within the main menu we reuse the level selection screen between Singleplayer and Multiplayer. I would say that instead the controller should be attached by the code instantiating a layout. In fact, there may be no need for UIControllers at all - instead the code instantiating the layout can just registered listeners directly with the relevant widgets, looking them up by id.
This doesn't make sense to me. The code instantiating the layout is generic and lives within the LayoutDefinition. How would you do any event handling there? And the level selection is already implemented using the new system. I thought about being able to specify controllers for separate UI elements in case they're used in separate layouts, but that didn't seem to be the case, so I decided to keep it simple for now.
  • I think StyleApplicator should be replaced with something using the ClassMetadata system ideally. We don't need a second system for applying properties to classes - they should work the same way as prefabs.
If the ClassMetadata system can do something like that, sure. But I'm calling into several thousand lines of Groovy library code, which unpacks stuff like a list of four floats into a Vector4f and stuff like that. I wouldn't want to re-write something like that, nor would I have the time to do so.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
You make it sound like you don't really think that the entire change is worth it in the first place, which I find a little discouraging, considering the work I've already put into it.
I didn't mean it to be that harsh, but you have to admit when it comes to creating a visual result working in text is not ideal. The big advantages of the json formats over code is that it can potentially be hooked into an internal or external visual editor - much harder to do that with code. It can also be overridden through the override system, and it can potentially be reloaded at runtime. There is some potential for linking layouts directly to prefabs and having UI without code at all, but there is a limit to what can be done without at least a little bit of code. Disadvantage is that it isn't subject to compile-time checks. Am I missing some critical advantage?

I do see it as an improvement, but it isn't revolutionary. Once a visual layout tool is provided, then we will have something awesome. :)

This doesn't make sense to me. The code instantiating the layout is generic and lives within the LayoutDefinition. How would you do any event handling there? And the level selection is already implemented using the new system. I thought about being able to specify controllers for separate UI elements in case they're used in separate layouts, but that didn't seem to be the case, so I decided to keep it simple for now.
I would use the layout definition to produce a UI, and then connect the event handling to the UI. The layout definition is a replacement for manually putting together UI widgets. So perhaps something like:

Code:
UIContainer window = guiManager.createUI("engine:menu"); // Layout uri
window.setListener("ok", new Listener() { // The layout contains a button with a property "onClickEvent" : "ok" or something similar?
    onEvent(...) {
        ...
    }
}
This sort of approach is needed anyway, since you will want to bind in entities and programmatically set other things that cannot be handled by simply instantiating a controller. When opening a chest in the world for instance:

Code:
UIContainer chestInventory = guiManager.createUI("engine:inventory");
chestInventory .bind("target", chestEntity);
If the ClassMetadata system can do something like that, sure. But I'm calling into several thousand lines of Groovy library code, which unpacks stuff like a list of four floats into a Vector4f and stuff like that. I wouldn't want to re-write something like that, nor would I have the time to do so.
Basically, you have a system that converts some JSON code into a structure of objects, and applies JSON fields to the fields of those objects. That is exactly what our prefabs do. The way they do this is:

1. Convert the JSON into a protbuf objects. This is only done so that it can leverage the same code for loading serialized entities (from disk or network). You would be able to use something else for most of this, except you would still want to convert your fields into protobuf EntityData.NameValue objects. You can reuse EntityDataJSONFormat.ValueHandler to do this. The outer containers can be whatever you want.
2. Create the components by looking them up in a ClassLibrary. The class library are generated using reflection, so in your case you would have a class library of UIWidgets or whatever, gathered from all the active modules including engine. This means mods can add additional widgets. They are all identified by URIs which avoids all the issues associated with dealing with java packages, and uniquely identifies them by module. Basically, they behave the same as assets.
3. Using the ClassMetadata for the object, together with a TypeSerializationLibrary, to apply the fields to the object. The class metadata provides access to the fields of the objects by name, and the TypeSerializationLibrary knows how to handle the different types, including building up compound types as needed (where such a type is annotated with @MappedContainer). The code for this something like:

Code:
     // Get the serializer for the type. This may be generated if it is the first time working with that type.
     Serializer serializer = typeSerializationLibrary.getSerializerFor(componentMetadata);
 
     // Iterate over all the fields
     for (EntityData.NameValue field : componentData.getFieldList()) {
        // Get the metadata for the field by name
        FieldMetadata<T, ?> fieldInfo = componentMetadata.getField(field.getName());
 
        // Make sure it is an actual field (can log an error if it isn't)
        if (fieldInfo != null) {
            // Apply the field
            serializer.deserializeOnto(targetComponent, fieldInfo, field.getValue());
        }
    }
This... probably could be improved further by adding a method to apply a list of fields to the serializer, now I think about it.

Anyway, the advantage of following the same pattern for Layout Definitions is that we end up with a consistent behavior for how these definition files work, such as things being identified by uri rather than fully qualified java class name, types having the same representation, the same sorts of types supported. Any improvements to this behavior will help all systems at the same time. You would instantly gain access to all the types already supported, such as assets, all the vector types, color4f, blocks...
 

x3ro

Member
Contributor
GUI
It's probably about time I got back to this :)

Firstly, I do think that going from Java-Code to a JSON based layout system is a big advantage. It might not be revolutionary, but I know that I wouldn't like to create website styles using Java code, as opposed to CSS (which I'd argue to be roughly equivalent to what we have here).

I would use the layout definition to produce a UI, and then connect the event handling to the UI.
This is exactly what you do with the controller, except that it happens semi-automatically. In your solution, you'd need an extra file for every menu, one that connects the event handling to the UI. That is what the controller does. If you don't need one, don't specify it.

Regarding the ClassMetadata system: I simply won't have the time to get into that and make it work. The StyleApplicator class is currently 70 lines of code, of which most are comments. If you or anyone else feels confident that the ClassMetadata system can do everything that is needed (or implement it if somethings missing), then the StyleApplicator can be replaced. But I currently cannot do that.

Which brings me to my last problem: I really _need_ my changes to get merged as soon as possible. As I get farther into my semester, I will have even less time to work on side-projects. Last time I had my GUI changes lying around for a couple of months I spent days getting everything to work again (which is fine, of course. I don't expect this project to stand still because of me ;)) But I know that I won't have the patience to do that again, especially with Immortius currently revamping the GUI system, which won't make it easier.

---

TL;DR: I roughly have until the end of this month to get the changes merged, otherwise I won't have time to continue working on them for at least 3 months. If Immortius says this can realistically be done (the merge), without having to rewrite the StyleApplicator and controller mechanism, I'll try to make everything else as smooth around the edges as I can, of course based on your feedback.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
It's probably about time I got back to this :)

Firstly, I do think that going from Java-Code to a JSON based layout system is a big advantage. It might not be revolutionary, but I know that I wouldn't like to create website styles using Java code, as opposed to CSS (which I'd argue to be roughly equivalent to what we have here).
I guess the only thing we disagree on is the scale of the improvement, which is fine.

This is exactly what you do with the controller, except that it happens semi-automatically. In your solution, you'd need an extra file for every menu, one that connects the event handling to the UI. That is what the controller does. If you don't need one, don't specify it.
My solution doesn't require an extra file. Something already has to be invoking the UI, they're not magically coming into existence. My main point is the controller being created by the UI is not in a position to connect event handling to the UI because it knows nothing of the context creating the UI - if I have a UI screen for showing the contents of a container, then the controller cannot know what container to show. Only the context invoking the UI knows. Additionally it would be nice to have simple, reusable UI screens like Yes/No dialogs, without having to have a new layout for each one. I think this might imply that a controller is a separate selection that is made at the same time as creating the UI - perhaps the layout file's controller is just a default that can be overridden.

But this doesn't need to be solved immediately, and there is a continuum of solutions to consider. Granularity may also be worth thinking about too - perhaps rather than a controller for the layout, you would connect simpler controllers to controls - like an exit controller to the exit button. Data binding obviously has a role here too.

One thing that I'm seriously considering is whether the UI system should, rather than being a whole separate system, actually just be a feature of the entity system. This would imply that the UI could just send events back to the entity "hosting" the UI, and ComponentSystems would take the role of controllers.

Regarding the ClassMetadata system: I simply won't have the time to get into that and make it work. The StyleApplicator class is currently 70 lines of code, of which most are comments. If you or anyone else feels confident that the ClassMetadata system can do everything that is needed (or implement it if somethings missing), then the StyleApplicator can be replaced. But I currently cannot do that.
That is fine, but I'm sure you can understand I'm not thrilled with adding architecturally non-conformant changes with the technical debt that implies into the engine. But it is ok. Even if you used the ClassMetadata system now, it would be on the wrong objects (the old UIElements). And you are at least using a library we already have, even if it is one I've been working towards getting rid of :p.

So you know in advance, if/when the NUI system reaches the point where it is ready for use, I will look at adapting the layout system for it. This may just mean switching it to using UIWidgets via ClassMetadata and UIContainers. Or it may mean some fiddling around so it better conforms with the UIWidget/UIContainer system. Worst case, I'll create something similar from scratch. Either way it is likely none of the existing style and layout files will work without modification.
 

x3ro

Member
Contributor
GUI
My main point is the controller being created by the UI is not in a position to connect event handling to the UI because it knows nothing of the context creating the UI - if I have a UI screen for showing the contents of a container, then the controller cannot know what container to show. Only the context invoking the UI knows.
You can always let the controller know about the context after it is created, so that it can respond to context-specific events:

Code:
UIWindow window = guiManager.createUI("engine:inventory");
window.controller.setContext(this);
If you're loading a certain layout, you're bound to having many events that are always the same for every instance of the layout. The intention I had with the controllers is to group said repeating behavior, to make thinks simple and re-usable :) Initially I also proposed to be able to create controllers for a single element (like an "exit button controller"), but people didn't like it so I didn't think it'd be important.

That is fine, but I'm sure you can understand I'm not thrilled with adding architecturally non-conformant changes with the technical debt that implies into the engine.
Of course I can understand that. And I'm not thrilled about the fact that my sole contribution will add technical debt to the project. But I do think that I made a couple of improvements worth having, otherwise I wouldn't ask you to pull it in :)

From your answer I conclude that you'd be willing to pull my changes in the current state, after I've performed some additional polishing, is that correct? That'd leave me with the following stuff I'd like to get done asap
  1. Sort .gitignore into category
  2. Register asset factory for layouts in GUIManager
  3. Adapt asset loading to not use "style:engine:" + class form.
  4. Create getter for the controller, so that one can add context information
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Yes, those changes will be fine for now, along with updating your branch be based off latest multiplayer.

If you're loading a certain layout, you're bound to having many events that are always the same for every instance of the layout. The intention I had with the controllers is to group said repeating behavior, to make thinks simple and re-usable :) Initially I also proposed to be able to create controllers for a single element (like an "exit button controller"), but people didn't like it so I didn't think it'd be important.
I'll do some thinking about this. I think there are two use cases though - elements that do things within the UIWindow (such move list items up and down or show/hide elements) and elements that act outside the window, such as Ok/Cancel buttons. The former would be handled by the controller like you have, and that makes sense. I've been thinking about the latter, and perhaps that can just be handled via events being sent to entities.

Mentally I'm comparing all this to how the layout system worked in UT3, and particularly how I made use of it. Essentially I ended up making a lot of things widgets (such as a scoreboard, or a map) and just using the layouts to control how those widgets were laid out - there was generally little interaction between the elements, rather they interacted with and fed from the same underlying model. The advantage of using the layout system was there was a nice visual editor (that crashed often), and you could preview how it would look at different resolutions (the layouts were adaptive).
 
Top