Render DAG Enhancement - GSoC 2019

dave2s

New Member
Contributor
Initial text:
I'm including my proposal to keep the order in things.

Read only: Overleaf (LaTeX editor) link
Edit/comment rights - requires login (e.g. with google acc, comment by clicking review on top and selecting source code - comment button pops up): Overleaf (LaTeX editor) link

This proposal is centered about continuation of render graph topic. Proposing a way to express dependencies among nodes as drafted by some earlier (than me) issues and maybe deal with some blockers for future dag-module exposé. As a second part I'd like to take on mkienenb's PR for render graph overlay (ingame) and I give some ideas there. As a strech, some nodes might be provided which could leverage these changes.
Edit: This project has been accepted for GSOC 2019. yeet :gooey:
Some basic information:
 
Last edited:

dave2s

New Member
Contributor
Week 1 summary

During the first week I've been drafting an extra layer of connections which are attributed to nodes on the AbstractNode level.
This graph shows which parts of the DAG I've begun to edit, compared to reference dag, during the first week. Everything should be compilable and runnable, although I'm not verifying at this stage.
2536
Some of the basic units in the render graph I've been working with for future reference are

WorldRendererImpl(c); RenderGraph(c)
Node(i)->AbstractNode(ac)->SpecificNode(c);
(new) DependencyConnection(ac)->FboConnection(c)
;
(new) NewAbstractNode(ac, surrogate for AbstractNode)

What is possible at this stage: Connecting FboConnections without using centralized system.

How does this work at the moment: Given nodeA and nodeB you
  1. Add output to nodeA - in its constructor/setDependencies method - addOutputFboConnection(1,nodeA);
  2. After nodeB's creation - explicitly* connect nodeA's output to nodeB's input in WorldRendererImpl -nodeB.connectFbo(1, nodeA.getOutputFboConnection(1));
  3. Inside nodeB's setDependencies() method - use this connection however -
    this.getInputFboData(id) / this.getOutputFboData(id) / this.getInputFboConnection()...
*not happy with this yet
These changes seem to so far create the same output and the game runs.
On the other hand, there's a lot of things to consider and this specific approach is still far from its final shape, if something like this is going to be adopted at all (so far no other direction has been spoken).

Next on:
  • Auto matching in RenderGraph - I'm going try and remove the need to explicitly connect dependency connections by hand trough WorldRendererImpl. Since now you connect dependencies and the nodes order separately, which seems rather redundant so far. This includes setting input connections without data inside nodes. So you basically don't care what you are connected to. This should provide for needed modularity.
  • Still considering every little change that has and hasn't been done. Every change has to be proven needed or it's gonna go away. So far everything is easily upgradable to a different approach.
Further on:
  • When it becomes clearer whether we will continue this way, I need to review legacy code a remove centralized dependency storing alltogether.
PR: DAG DependencyConnection draft #3680

Current problems
:
So far making changes to the basic classes requires spreading the changes troughout majority of classes which makes a messy PR.
Trying to maintain top down perspective is a little problematic while spending time trying to make things work right away. It seems as though it might almost be better to just work with a broken code for a bit.

Blockers: Nothing in particular.
 
Last edited:

dave2s

New Member
Contributor
Week 2 summary

During the second week I had tried to come up with ideas around auto-matching of connections but so far it's a dead end. Other than that I've added some attributes to connections to remember who they belong to and whom they're connected with. I added some error checks to connecting mehods.

Some of the basic units in the render graph I've been working with for future reference are

WorldRendererImpl(c); RenderGraph(c)
Node(i)->AbstractNode(ac)->SpecificNode(c);
(new) DependencyConnection(ac)->FboConnection(c)
;
(new) NewAbstractNode(ac, surrogate for AbstractNode)

Additions:
  1. DependencyConnection:
    Java:
    private SimpleUri parentNode; private SimpleUri connectedNode;
    getParentNode(); setParentNode();
    getConnectedNode(); setConnectedNode();
  2. NewAbstractNode: changed addConection hiearchy of methods to return true on success, false otherwise. connectFbo then checks for existing connections as well as connections which have already been read from, to prevent buffer competition. Next on this-I'm considering adding a connectWithCopy as well as generic types to implement these higher level methods.
  3. Whole dag refactor for class and variable names: ...FBOs... -> ...Fbo...
Next on:
  • API vs Whitelist - I'm going to try to tamper with the dag setup from a module and see where it goes.
  • Discuss whether to deal with lastUpdated/stale gbuffer pair passing the same way as with auxiliar buffer passing.
Further on:
  • Make abstract level methods generic.
  • Cherry-pick FBO.copyToFbo() from a private branch to enable connectWithCopy or something of the sort. Need to discuss whether to add scheduler methods for this.
  • When it becomes clearer whether we will continue this way, I need to review legacy code a remove centralized dependency storing alltogether.
  • Can't forget Test module. Made some changes there to compile during week 1, did not test.
PR: Updated DAG DependencyConnection draft #3680

Current problems
:
Can't figure out what to do with the main buffer pair...do I implicitly pass it? explicitly pass it? do I keep them centralized?

Blockers: Nothing in particular.
 
Last edited:

dave2s

New Member
Contributor
Week 3 summary

During the third week I drafted an API for modules to use after the engine DAG setup and a module based DAG change, which reconnects some dependencies to bypass a node and it works. One setback is that currently there're some redundant state change calls (like binding a different texture to the same variable twice) which is caused by repetitive call of bulky node.setDependencies() call, which happens when a node's input is being reconnected. TODO improve this with some more logical update, or at least postpone these calls (AND therefore RenderGraph construction too) until module based changed have been made too so the calls are made after the dag is set up.

This step depends on whether we want to support mid game DAG. If so, the node.setDependency() method would have to be split and moved under methods which create each dependency. These would have to be split into methods enabling setting connections based on what we want to do with them. Example:
node1.connectFbo[Texture](input id, node2.getOutputFbo(id)); Inside connectFbo we would make a call to add stateChange into queue to send a texture to some local program. This approach however is kind of restricting and it would be hard to cover every option. I don't see a point in enabling this since any mid-game changes are handled inside node's process() and propertyChange(PropertyChangedEvent) methods.

On a related note: So far all dependency calls don't co-operate with the RenderGraph itself. This will have to be added soon.
Why do we need this: Tampering with dependencies does not yet take into account how the nodes are connected in the render graph. But only nodes connected in the RenderGraph are run and only RenderGraph sets the order the nodes are processed.
What does this look like: dependency connecting methods (probably on the level of the abstract nodes) will have to check whether they are the first connection to be added / last to be removed and create/remove RenderGraph connections.
Some of the important files:

dagTestingModule github repo. This module extends BaseComponentSystem and uses initialise() phase to be run. Happens after injection, so we can inject context and use it to obtain instance of renderDagApi, which was created during World creation phase (before init, see StateLoading.java's init[Client/Host] methods).
(new) RenderDagApiInterface(i)->RenderDagApi(c);
(m) DependencyConnection(ac)->FboConnection(c)
;
(m) NewNode(i)->NewAbstractNode(ac);
minor tweaks here and there, WorldRendererImpl(c), RenderGraph(c).


Next on:
  • Create a module demonstrating basic render dag api usage, including module based node which is inserted into the dag. (Due June 24)
  • Postpone setDependency calls after the rendering modules have been initialized. Also render dag construction would have to be postponed.
  • Add some API capability.
  • Integrate with RenderDag operations. (!!!)
Further on:
  • Split rendering nodes into 2 sets - 1.essential features, 2.advanced features. Move the groups into modules. Feature set #1 will be mandatory. Feature set #2 will be optional.
  • To do read only Fbo. Cherry-pick FBO.copyToFbo() from a private branch to enable connectWithCopy or something of the sort. Need to discuss whether to add scheduler methods for this.
PR: Updated DAG DependencyConnection draft #3680

Current problems
:
Redundant calls of setting dependencies for entire node based on only one small change. - Not good
Can't figure out what to do with the main buffer pair...do I implicitly pass it? explicitly pass it? do I keep them centralized?

Blockers: Nothing in particular.
 
Last edited:

dave2s

New Member
Contributor
Week 4 summary

During the fourth week I took out the previously introduced tint node and made it module based. The module now inserts a Tint node between finalPostProcessingNode and OutputToScreenNode effectively turning the final image reddish. Added a areConnected(node, node) query to RenderGraph to see if 2 nodes are connected when sorting dependencies. Added findNode(), resetDesiredStateChanges() overloaded methods to RenderGraphApi and Nodes, fixed toString() for connections, added isDependentOn() method to query a node for dependency on another node - used for automatic connecting/disconnecting nodes in the RenderGraph upon reconnection of dependencies. Updated connectFbo() and reconnectInputFboToOutput() renderDagApi/Connection methods to use this. Had to implement somewhat a @API wrapper for java.beans.PropertyChangeListener so that the module-based nodes are not denied access during runtime. Did not test the events' functionality.

Further I updated ShaderManagerLwjgl to allow for adding shaders from within different modules than engine. Just overloaded the addShader() method with another argument. Updated RenderDagApi interface with addShader() and shaderManager injection.

Added new ModuleRenderingSystem extending BaseComponentSystem, which is now extended by the rendering module instead to hide the rendering module's initialization.

dagTestingModule github repo. This module's main class now extends ModuleRenderingSystem which extends BaseComponentSystem. Adds a tint shader as well as TintNode.
(new) ModuleRenderingSystem(ac)<-BaseComponentSystem(ac)
(m) RenderDagApiInterface(i)->RenderDagApi(c);
(m) DependencyConnection(ac)->FboConnection(c)
;
(m)NewAbstractNode(ac);
(m) RenderGraph(c).


Next on:
  • Filling Missing/Incomplete Javadocs
  • removing redundant or unused things from ; getting back to node's setup - getting rid of previous FBO storing, updating all nodes to this system.
Further on:
  • Split rendering nodes into 2 sets - 1.essential features, 2.advanced features. Move the groups into modules. Feature set #1 will be mandatory. Feature set #2 will be optional.
  • To do read only Fbo. Cherry-pick FBO.copyToFbo() from a private branch to enable connectWithCopy or something of the sort. Need to discuss whether to add scheduler methods for this.
PR: Updated DAG DependencyConnection draft #3680
DagTestingModule:
dagTestingModule github repo

Current problems:
  • Java beans property change listener/event inaccessible in modules..workaround not tested, compiles though.
  • So far a few workarounds like storing RenderGraph and Context instances in nodes themselves.
  • Rendering module needs to pass its class to its super so we can store Name of the module for use in creating Uris. I only tried getting classes name which returns name of the class implementing the abstract module rendering class, but that is not sufficient because we need the full path including the module's package to find the class this way. There might be a better way, though it's not that important now. At least I could assert non-null on the ModuleRenderingSystem's protected providingModule attribute somewhere so it's forced for now. In addShader at least, that's where it's needed. We don't need to do that when creating Nodes because they are created within the module, unlike calling addShader().
Blockers: Nothing in particular.
 
Last edited:
Top