Javascript Modding API

ironchefpython

Member
Contributor
Architecture
So I know this is going to sound a little crazy, but hear me out. I think for the purposes of modding, an event-driven, asynchronous Javascript API would make it extremely easy for modders to develop and test mods, and open up the pool of potential modders beyond Java developers. Technically this is quite feasible, Rhino is an embeddable Javascript interpreter written entirely in Java.

To give an idea of what I’m talking about, here’s a “mod” that would add torches that “burn out” into the game.

Code:
// Game.addBlock accepts an object, reads the properties and creates a new
// block type and adds it to the game world.  It returns a reference to the
// newly created block type.

Torch = Game.addBlock({
   name: "Torch",
   placedAs: "wallItem",
   model: "litTorch"
});

UnlitTorch = Game.addBlock({
   name: "Burnt Out Torch",
   placedAs: "wallItem",
   model: "unlitTorch"
});

// Blocks have a number of event handlers.  Here, we are referencing the
// "onPlace" event handler, called when a player (or NPC) places this
// block

Torch.addEvent("onPlace", function(){
    // Note that "this" refers to the current torch being placed
    this.luminescence = 15;
    this.fuel = 256;
});

// BlockSet is a Rhino Java Host Object that can contain arbitrary sets of
// blocks.  It exists for the convenience of the modder for easy membership
// testing and iteration.
torches = new BlockSet();


// The Game object also has a number of event handlers, "loadChunk" is called
// every time a chunk is loaded into the game world (either by being newly
// created, or by being loaded from network or disk)
Game.addEvent("loadChunk", function() {
    // Note that "this" refers to the current chunk being loaded
    
    // this.blocks returns a read-only BlockSet
    blocks = this.blocks;
    for (var i = 0; i < blocks.length; i++) {
        if (blocks.typeId == Torch.typeId) {
            torches.add(block);
        }
    }
});

Game.addEvent("unloadChunk", function() {
    // BlockSet.removeAll removes all blocks from the blockset that are also
    // present in the collection passed as a parameter.
    torches.removeAll(this.blocks);
});

// setTimeout and setInterval work exactly as expected
setInterval(function() {
    for (var i = 0; i < torches.length; i++) {
        torch = torches[i];
        torch.fuel--;
        torch.luminescence = Math.min(torch.fuel, 15);
        if(torch.fuel == 0) {
            torches.remove(torch);
            torch.replaceWith(UnlitTorch);
        }
    }
}, 100000);
The ultimate goal is to allow anything to be built with the engine, from a tower defense game to a harvest moon farming simulator without needing to know Java. The target user would be one that might not know how to use Eclipse or build systems, but who has done a little HTML/Javascript, and who can edit a text file and tell the game to reload. (or better yet, use the in-game editor) :)

If this is a direction people would like to consider, we would need to add the Rhino jar to the build, and create a number of Host Objects (Java objects that can be interacted with from Javascript), and start defining the game events that can be attached to.

== EDIT ==
My Groovy is a little rusty, but would look something like this:

Code:
def Torch = Game.addBlock [
   name: "Torch",
   placedAs: "wallItem",
   model: "litTorch"
]


def UnlitTorch = Game.addBlock [
   name: "Burnt Out Torch",
   placedAs: "wallItem",
   model: "unlitTorch"
]

Torch.addEvent "onPlace", {
    it.luminescence = 15
    it.fuel = 256
}

torches = new BlockSet()

Game.addEvent "loadChunk" {
    it.blocks.each {
        if (it.typeId == Torch.typeId) {
            torches.add block
        }
    }
}

Game.addEvent "unloadChunk", {
    torches.removeAll it.blocks
}

Game.setInterval 100000 {
    torches.each {
        it.fuel--
        it.luminescence = min(it.fuel, 15)
        if(it.fuel == 0) {
            torches.remove it 
            it.replaceWith UnlitTorch 
        }
    }
}
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Well, we've already got Groovy for the light-weight scripting language to help ease modding with wonderful access to everything in the running game via very simple object definition. Lots of shortcuts available there to make writing code easy, don't necessarily need full-fledged Java knowledge, yet can immediately apply full Java power if needed :)

Would using Javascript instead / in addition to really be that different? It seems like a version of your example in Groovy would be extremely similar, even possibly a bit snazzier with the added Groovy convenience methods and closures :geek:

Your example is a great example either way (thanks!). I can't wait till we have the pending Entity System ready to the point of being able to write event handlers like that / do an initial modding API :)
 

ironchefpython

Member
Contributor
Architecture
I like Groovy, but honestly I think it's an evolutionary dead end. I would argue that Javascript would allow far more potential developers, and will have far more longevity than Groovy.

That being said, you can absolutely do everything with Groovy that you can do with Javascript. I've never done any event-driven asynchronous development with Groovy (it's an uncommon idiom in the space where Groovy is often used). But I'm pretty sure you can do it.

So I guess there's are two decisions:

one: should there be an asynchronous event-driven modding API? (as opposed to a threaded/promises api, or (god forbid) the Minecraft "modding" methodology of replacing a bunch of Java classes)

two: what language (or languages) should be supported for the modding API?
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
I think the event-driven modding you're talking about would slide right in with the Entity System design, as registering an event would probably create a relevant entity with the appropriate components attached that'll then be iterated over at the appropriate time by paired systems. But I could be wrong, especially since I'm not 100% on top of that design yet, maybe Immortius can chime in with an opinion on that :)

As for languages, I do like Groovy and think it can be very useful within the game, like in our GroovyConsole where we can dynamically interact with game state using code we write right there in the game. Could you do something similar with Rhino, you think, independent from the modding angle? Maybe Scala has some potential too. Maybe we could support both Groovy and Javascript modding APIs? I like opening options that may reveal potential down the road, although I feel daggers being stared at me each time I pick at the corners of our (not really) neatly defined toolset :D
 

ironchefpython

Member
Contributor
Architecture
I wouldn't recommend running multiple languages in your console, or supporting multiple languages for modding. (Modding in Scala counts as modding in Java for the purposes of this discussion). I would pick a single idiom, and a single language for modding.

But let me break down what I'm trying to ask in a little more detail. Here are some modding idioms

Adding Classes: this is the idiom used by Minecraft modders. You write Java (or Scala or Clojure or whatever), and you compile it to .class files, and add them to the application.
  • You need to be a Java developer
  • Lot of conflicts between mods
  • Not limited to an API
Scriptable Engine: This is where your "mod" runs a continuous loop, executing some code, and then executes the game engine to update (and display) the world. This is how many game engines work
  • Limited to a single mod
  • Limited to the API exposed by the game engine
  • Would work with any scripting language
  • Simple conceptual model for developers
Event Model: This is where your mod registers new objects and event handlers at runtime, and as the game engine executes it calls the appropriate callbacks to handle events.
  • Limited to the API and events exposed by the game engine
  • Would support multiple mods without conflict
  • Would work with any scripting language
  • Very similar idiom to that used for web application development
My opinion (and you may disagree), but your application will live or die based on the ease and power of your modding API, as we've seen with Minecraft, there's no way a single development group can equal the creativity and drive of a modding community. And if you present an API that allows you to make a mod in a few dozen lines of Javascript, rather than decompiling Java classes and modifying them, that is a compelling reason for people to play with, and develop for your game. (and like I said before, Groovy is just as capable as Javascript, if not quite as compelling)

As far as the Entity System, that's an implementation detail. Implementation details are cool and interesting, but at the end of the day, it's the API that your users interact with that is going to matter. Get your API right, and you'll get users.

So I think it's important to pick an modding idiom. I'm biased in favor of an event model, but a decision should be made; the decision on how to support mods will be critical to the application design. If an idiom such as "Event Model/Javascript" or "Event Model/Groovy" is chosen, I can help flesh out the API that could be supported, as well as creating sample mods such as for crafting and cooking, planting and growing, mechanisms such as switches, levers, lights, doors, etc.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
ironchefpython said:
My opinion (and you may disagree), but your application will live or die based on the ease and power of your modding API, as we've seen with Minecraft, there's no way a single development group can equal the creativity and drive of a modding community.
I absolutely and wholeheartedly agree. Part of my personal drive for this project is to focus on the super easy power user contribution angle, as we've already seen with the ability for otherwise not-so-tech-savvy contributors being able to contribute to the project, even this early :)

ironchefpython said:
So I think it's important to pick an modding idiom. I'm biased in favor of an event model, but a decision should be made; the decision on how to support mods will be critical to the application design. If an idiom such as "Event Model/Javascript" or "Event Model/Groovy" is chosen, I can help flesh out the API that could be supported, as well as creating sample mods such as for crafting and cooking, planting and growing, mechanisms such as switches, levers, lights, doors, etc.
While I'd still like to hear some of the other core folks chime in here, the event models sounds like it would be my choice too. And now that you've outed yourself as knowledgeable on the topic, you up for helping with the very core of said design? :D

I'm not at all married to Groovy either, by the way. I know Kai for instance would love nothing more than throw it out the window into the path of an oncoming bus :D I do like it and can use it myself so hey. If something better pops up and can be justified I'm game.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
ironchefpython said:
Code:
you up for helping with the very core of said design?
I'll start putting some designs up here: https://docs.google.com/document/d/13Bk ... n36op7FCNg

I'd put it on the wiki, but it's closed to contributions (someone doesn't really get the wiki way :p )
Sorry about that - we closed registrations just a day or two ago as the wiki was being slammed by spambots :)

PM me a username / password / email and I'll make you an account?
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
I've been assuming an event driven model anyhow, and it is core to the current entity system design (both in the form of the application hardcoded events like initialise() and update() calls to component systems, and events like collision events, damage events and so forth which can be sent to entities. Mods would be able to add new events of the latter type).

I've been thinking of the entity system in terms of it being a core part of the mod API. Components are the way mods can add new data to entities, component systems (perhaps need a better name?) are how mods can add new behaviour by responding to events, the entity manager is how entities can be discovered for interaction. Obviously this isn't everything that is needed, but it provides a fundamentally extendable core.

As far as scripting language - I don't really mind what language is used if any, as long as Java mods are also supported. That is proper Java mods, not class replacement or arbitrary rubbish. Creating classes against an exposed API, jar them up, plop it in a folder (perhaps with some asset files), maybe zip the whole thing up, and then when the game is run it can be detected and activated through a proper mod mechanism. Any extra language support beyond that is a bonus to me.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Nice! Cool to see stuff appearing. Very exactly so since I can see you editing thanks to the whole real-time thing of GDocs :)

One question - blocks currently are saved in a file, sometimes more than one when also involving new images. Wouldn't it make more sense to be passing in stuff loaded from one or more files rather than a mod "building" a block via code?

Current example could just be an example, I admit :geek:

Also, you include "BlockSet" - I wonder if that'll class with the existing "BlockGroup" which is actually a single block, but involves multiple possible orientations (IIRC)
 

ironchefpython

Member
Contributor
Architecture
Cervator said:
One question - blocks currently are saved in a file, sometimes more than one when also involving new images. Wouldn't it make more sense to be passing in stuff loaded from one or more files rather than a mod "building" a block via code?
Pretend you're a brand new modder. You open Terasology, and you open the in-game text editor, and start typing. You add a new block type in one line of code, and then you add a behavior to it in a couple more lines. You press "test", and BAM! You have a new block type in the game.

Being able to add block types with code is crucial to the immediacy that I'd love to provide to the modder. That doesn't preclude being able to have separate files containing block definitions. The simplest implementation of the separate file could be multiple addBlock function calls. Or there might be an API method that reads block definitions from CSV or XML or JSON data formats. Or the modder might write their own import function that reads block data from their own custom format. But at the end of the day, it's crucial to allow the runtime creation of blocks via an API, rather than relying on a file with fixed IDs, because the API call allows runtime ID assignment, to protect against ID conflict.

Cervator said:
Also, you include "BlockSet" - I wonder if that'll class with the existing "BlockGroup" which is actually a single block, but involves multiple possible orientations (IIRC)
I don't care about any of the names I've provided. BlockSet is a java object that is exposed to the modding api for convenience and performance over the native arrays in the scripting language. It can be called anything that indicates it's a set of block instances that exist in the world. If you can think of a better name to express it's blockness, and it's abstract setness, I'm fine with that.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Okies on naming, just wanted to make sure I pointed out the existence of BlockGroup :)

Sounds good with the either/or option for defining blocks, I'm fine with that. One note - the current block definition files actually do not hardcode any IDs at all, that's one of the strengths in the system. The code more or less parses all the blocks at world create time and builds a manifest at that time that ties whichever blocks to ID values specific to that world. That's then also handily available for tweaks if needed.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Ok, feedback.

Seems like a solid start. I was going to suggest the addBinding() method, but you beat me to it. :)

The suggestions I have are around entities (again :p). While it may be it may be worth having some specific interfaces for items, in general items, mobs, players, non-block world objects, and meta-block objects would all be entities, just with different components on them. So an addEntityPrefab( /* Prefab definition */ ) method would cover defining all types of new entities.

For entity events, the way these can be listened to by Component Systems in Java at the moment is by creating a method like

Code:
    @ReceiveEvent(components={PlaySoundActionComponent.class})
    public void onActivate(ActivateEvent event, EntityRef entity) {
        /* Play a sound */
    }
Which I imagine would translate into JS something like
Code:
EntityEventsSystem.listen("Activate", ["PlaySoundActionComponent"], function(entity, eventData) {
   // Do whatever.
});
That is you register for when a specific type of event (Activate) is sent to an entity with the set of components you are interested in (in this case, just the PlaySoundActionComponent) and receive the entity the event is occuring on and whatever relevent event data is sent. There may also need to be a priority parameter when registering a subscriber (to determine the order they are invoked), and yeah, cancellation stuff.

Main events that I think are missing is hooking into a timer to get a call every X seconds/minutes. And maybe hooking into the update loop, although given that should be avoided whenever possible I can understand leaving that only for Java mods rather than scripted mods.
 

ironchefpython

Member
Contributor
Architecture
I would envision a system where events were sent not to a whole group of listeners at once, but would "bubble up" through a defined hierarchy of potential listeners, and fire a handler with each object that registered a listener.

Let's say for the sake of argument, that a user clicks on an (in-world) button. A mousedown event would be created, and sent to the window object (if it registered a listener), and then the game object (again, if a listener was registered). If no listeners were registered, no events fire. Then a mouseup event would be created, and go through the same process. And finally a click event would be generated.

For the sake of argument, let's say that an event handler has been added for the "click" event. The event might check to see what the player was looking at, and if it is a solid block, and the player isn't holding a placeable item, the code could send an "interact" message to the targeted block.

This interact message would bubble up the same way (but with a different set of potential listeners. The mod that added the switch to the game would have added an event handler to the interact event to the block prototype.

So for mouse events (e.g. click, dblclick, mouseover, mouseout, mousedown, mouseup, mousescrolldown, mousescrollup), the hierarchy would be:
window -> game

And for block events (e.g. place, create, added, remove, interact) the hierarchy would be:
block instance- > block prototype -> region -> chunk -> world -> game



EntityEventsSystem.listen("Activate", ["PlaySoundActionComponent"], function(entity, eventData) {
// Do whatever.
});
I would express this like so:

Code:
entity.addEventListener("PlaySoundActionComponent", function(eventData) {
   // Do whatever.
});
But it's a style preference. The addEventListener pattern ( https://developer.mozilla.org/en/DOM/el ... ntListener ) is well known to Javascript programmers.
 

Immortius

Lead Software Architect
Contributor
Architecture
GUI
Hmm, I think you are missing part of the advantage of the entity system approach. You don't register for individual entities, you register on families of entities that contain a set of components that are of interest. This vastly reduces the amount of subscriptions floating around. The example had a single component type, but you might be interested in entities with an Item and Light components for instance (i.e. a torch).

Given that, it isn't clear how to use a hierarchy to determine event ordering. If you have one system that introduces doors and registers for the Use event on entities with a Door component, and a second system that introduces locks and registers for Use events on entites with a Lock component for the purpose of cancelling the event when the entity is locked, how do you determine which should be invoked first? It certainly shouldn't be the script that introduced doors, if such a thing even has meaning. A priority number would work, but makes things a bit complicated because you need to know the priority of other handlers in the environment. Better might be for events to have a number of phases, starting with a check cancellation phase that runs first, followed by the do stuff phase (or a pre-event, event, post-event phase style).

Keep in mind that a Lock can be applied to any Usable entity, so once you set up the simple script to cancel Use, you could apply the component to doors, chests, books, whatever.

I think this would be also true for registering for events on a hierarchy of "contexts" like you describe though. If there are two listeners on the Game->Interact event, one which plays a sound and the other which cancels the event, how do you determine correct ordering?
 

rapodaca

New Member
Contributor
I like Groovy, but honestly I think it's an evolutionary dead end. I would argue that Javascript would allow far more potential developers, and will have far more longevity than Groovy.
I'll second that one.

There's also performance to consider and there's a great deal of activity on that front with JavaScript on Java for server-side uses.

I wouldn't recommend running multiple languages in your console, or supporting multiple languages for modding.
+1 again.

Rhino performance is apparently not that good, but much faster JavaScript engines running on the JVM are on the horizon - for example:

Nashorn http://bit.ly/H8aRdS

However we decide to get modding to work, there should only be one language to do it in. This would concentrate modder development effort into a single (widely-used and well understood) language, thereby increasing the pool of open code to draw from. Positive network effect, in other words.

Having said that, JavaScript presents its own technical issues - not the least of which being a radically different object model compared to Java. But I think the problem is solvable given a well-thought API. Unfortunately, the only way we'll get there is with a lot of trial and error (and broken mods along the way).

One way to make this process faster is to eat our own dogfood. By that I mean peel off a chunk of critical functionality we were thinking of having in the core of the game, and implementing it as a (JavaScript) mod instead. This would force us to use our own API and would help uncover design flaws early.

We may actually get to a point where a good deal of the game is actually written in JavaScript - not Java.
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
Javascript - madness, but I suppose we're all a little mad :D

I do like Groovy, and bet I'd like Scala too. Wouldn't Java-Scala-Groovy more or less count as one language anyway? They all compile to the same bytecode.

I'm not closing any doors tho. Were we to indulge the madness would that open up utility worth from Node.js?
 

ironchefpython

Member
Contributor
Architecture
rapodaca said:
Rhino performance is apparently not that good, but much faster JavaScript engines running on the JVM are on the horizon
If you're putting more than a simple event handler or two in the main UI loop, performance isn't going to be an issue. If you have a mod that calculates tree growth, or fluid dynamics or something, you run it in a separate thread, and weak performance isn't going to impact your gameplay.

Cervator said:
I do like Groovy, and bet I'd like Scala too. Wouldn't Java-Scala-Groovy more or less count as one language anyway? They all compile to the same bytecode.
Java and Scala and Groovy all compile to bytecode, but you lose the immediacy of a scripting language for modding if you're going to bytecode, and if you're talking about using those languages to write the engine... well, some of the worst codebases I've ever had the misfortune of working on were heterogeneous amalgamations of whatever language and idiom each individual developer chose to use that day. There's a lot of value in structure and consistency when it comes to maintainability.

Cervator said:
I'm not closing any doors tho. Were we to indulge the madness would that open up utility worth from Node.js?
Node.js is an interesting server, I use it for a lot of experimental stuff, but no. You'll get little to no value out of trying to match a Java client with a Javascript server. When you go to write the server, find yourself a nice asynchronous nio-based java transport framework (like netty http://www.jboss.org/netty )and build from there, so you'll have as much code reuse as possible from the client, rather than having to port your persistence and serialization and event handling framework to a new language.
 

brian

New Member
By coincidence i'm working on a YA/A Dune inspired historical novel, and curious about game adaptations, if for nothing else it forces a certain structure on my fictional writing, i looked at minecraft, and from that to this which seems very interesting. I coming from the java world, now do everything in clojure, which would be my preference. Clojure has really opened my eyes up, I truly believe it is the great next step, groovy, ruby, etc were great, but only brought us half way.
 
Top