Maintenance Signalling

[Marcin Sciesinski]

Name: Signalling
Summary: Allows to emit and transfer signals (on/off) over network of blocks
Scope: Engine / Core Content (I hope) / Mod
Current Goal: Provide some basic functionality for emitting (i.e. switches, pressure plate), conducting/transmitting (i.e. cables, AND / OR gates) and consuming (i.e. lamp) signal, as well as a framework for other mods to use to introduce new signal emitters (quite likely), transmitters (not very likely) and consumers (most likely) of the signal.
Phase: Design / Implementation
Curator: @Marcin Sciesinski
GitHub Repository: https://github.com/Terasology/Signalling
Related: blockNetwork

Details:
Introduces a concept of a signal - boolean flag that can be acted upon by a mod/system. Uses producer/consumer model with a conductor in the middle (if any).

Producer can emit a signal with either an infinite strength (range) or with a limited strength (range). A signal of range 0 signifies that the emitter does not emit signal. Producer has to have a SignalProducer component defined.

Consumer defines in what way does it interpret signals from networks it is connected to. Consumer is also notified when it receives or stops receiving a signal. Consumer has to have two components defined: SignalConsumer (input component) and SignalConsumerState (output component). The latter is modified by the Signalling system whenever signal state is changed (on/off).

Use instructions:
If you want your block to receive a signal and act upon, make sure it has two components - SignalConsumer and SignalConsumerState. Register an event listener on ComponentChangedEvent for the SignalConsumerStateComponent and act upon changes done to the signal - yes that is ALL you have to do to integrate with the Signalling mod.

Need help with:
1. Suggestions on functionality blocks - have an idea for a cool emitter, conductor/transmitter or consumer that fits in a generic mod category, please let use know.
2. Would you like to help us with textures, please post here.

 

[Cervator]

Good stuff

Suggestion: Block that receives signal input and as a result physically ejects output (so of a different kind). So like a dispenser, or turret (with combat!). Though I'm not sure if that's framework-level or actual implementation.

 

[Marcin Sciesinski]

Just to make it clear, I'd like to move away from the concept Minecraft has of "redstone pulse", where certain blocks require to go from not receiving signal to receiving signal to execute an operation. This kind of approach requires a lot of updates across multiple blocks and changes on the system to perform something very simple. So for little or no value (I guess, could be disputed) introduces a lot of computation overhead and network traffic.

So, if a dispenser/turret like that would be needed, I'd rather it have a setting (in UI) like "dispense every X seconds while having a signal", rather than "dispense on each signal change from off to on" as it's done in Minecraft. Optionally, actually have this "dispense on signal change" setting, as I can see some applications of this approach that couldn't be achieved otherwise, but have this as a not recommended setting, as it introduces performance issues/network lag.

 

[Cervator]

Oh, yeah, I wasn't thinking about pulses at all. If turret powered then it shoots as per its design

There'll be some TurretComponent with the info like RoF, which could be upgraded by in-game actions to improve your defenses and fun stuff like that.

With your wireless suggestion you could simply use a controller approach where pulses are needed and said controller object simply pulses the target directly, assuming there's a network connection between the two (or whatever is required for it to be valid)

 

[UberWaffe]

On a related note: Has Terasology (or one of the modules for it) ever considered the use of a master/slave entity system?
I.e. each of the blocks on a special group has an associated entity that points at another entity (tied to the first/main block of the group). The entity on the main block stores all data related to the block group, and does processing as required.
This approach I mainly though-up for doing block families (or 'multi-block-structures' as they are known in Minecraft).
You can have only one block processing, storing data, checking structural integrity (on ticks, etc.) instead of multiple blocks. The others all just process on block updates.

In this signalling module, each of your signal blocks could point at the master block of the network. Then, if you need to change the network from "Off" to "On" you only need to update the entity data on the main block. All other blocks would simply grab the data from it as required.
Also, other blocks need not ever process. Their entity is created and setup upon placement, and only ever changed/destroyed upon blockUpdates.

This would likely make more sense in your underlying network module.
Essentially the following events/code forms the base of the concept:

Spoiler

Upon placement of a network block:

• When a block is placed down, it checks to see if it can connect to a network. (Likely by checking for adjacent blocks, but might be a different check. Say for wireless networks.)
• If it cannot find a network to 'join', it instead defines itself as a master, and create an associated entity with all the data needed to track the state of the network. (Data stored depends on network type. But things like: signalState or totalPower or lineVoltage, etc.)
• If it can find a network to join, then it creates an associated entity that points at the 'master' of the network. I.e. the entity storing the state of the network. And also a pointer to the block that it had 'connected to' in order to join the network. (For triggering blockUpdates that potentially drops this node from the network.)

Upon breaking a network block:

• If the broken block is the block that you point at in order to 'connect to' the network, then run code for checking if you disconnect from the network or reconnect to a new block.
• If you cannot reconnect, then run the appropriate code. (I.e. become the master of a new network, or just shut down, etc.)
• If a block is broken or changes networks, then it issues a block update, so that other blocks using it as the connection to the network can update.

Points that need to be carefully programmed & tested:

• When two separate networks are connected, how they pick and switch to one master, and combine/change relevant data on the picked master.
• When a network is broken into two networks (broken in the middle), how it decides where the new master is created (if at all), and how it distributes/changes the data on the masters.
• When the master of a network is broken, how it creates (if at all) a new master on the network, and transfers the data.

The worse-case in terms of data/entity updates is when the master gets broken. It should, however, not be any more processing that would be required anyway if a signal is passed from block to block.
But overall, you should save a lot of block/entity data updates by not having to 'pass-along' the signal or value from block to block.
Overall, unless you plan to store data using only the extra-data-bytes per block, you should end up with less data in RAM. (I.e. no duplication of information across multiple entities, beyond the pointers to the master & upstream connector.)

 

[Cervator]

Good notes, UberWaffe

And good news, I do believe we support master-slave type setups already. In some of the portals / holdings work I've done with Stuthulhu we play with the idea of GelCubes, Oreons, or whatever being bound to the entity that spawned it and making decisions based on that. In other words the Spawner is the master entity and the Spawnables the slaves. When a Spawner has made enough Spawnables a "Queen" is picked that itself becomes a Spawnable, that itself tracks its offspring and so on till the world is all gray (or green) goo you can still track in a giant entity tree

Reference:

HoldingSystem.java
SpawnerSystem.java

// Finally create the Spawnable. Assign parentage so we can tie Spawnables to their Spawner if needed
EntityRef newSpawnableRef = factory.generate(spawnPos, chosenPrefab);
SpawnableComponent newSpawnable = newSpawnableRef.getComponent(SpawnableComponent.class);
newSpawnable.parent = entity;

A network of blocks could work much the same way, although there you have an added layer of possible optimization. Since you're interacting with primitive blocks you only really need to track their locations, you don't need an entity per block. You end up with a single network entity that as one of its Component variables could have a BlockSelection for the block positions.

Block updates that touch one of the positions tracked there could then trigger an event against the network entity and you go from there.

Admittedly a little bit of the old multi-block stuff is my awful code that might need some substantial refactoring before it could reliably be used in all-world block update catching schemes

As for per-block data that's trickier, but doesn't relate to signalling (yet) as Marcin Sciesinski points out the network and its components act more as a whole than as individual blocks already. It is more the liquid system that's tinkering in that area.

 

[UberWaffe]

Since you're interacting with primitive blocks you only really need to track their locations, you don't need an entity per block. You end up with a single network entity that as one of its Component variables could have a BlockSelection for the block positions.

That is brilliant in its simplicity. I wonder why I hadn't thought of using just a single entity for the whole network...
I'm a big fan of K.I.S.S

Anyway, thanks for the links. Those are golden.

 

[Immortius]

Doors are also an example using a single entity to control both blocks.

 

[Marcin Sciesinski]

I don't think creating an entity to store all the network data, or use one of the blocks in a network to store it is an optimum solution. The biggest problems with that approach are:
- you are limited to what kind of information you can store in entities (types),
- when the entity information is stored in world/chunk save, when world is reloaded, different chunks may be loaded, and the entity will contain invalid information or not be loaded at all.

If I store all those data separately in the System, I have access to complete suite of Java classes and optimization techniques (Maps, Multimaps, Sets, and caching). This approach also allows me to limit the amount of information stored per block to the absolute minimum, often times - just a "transient" prefab.

While I can see this is a good approach when working with multi-block structures (like in Minecraft), where either you have multi-block structure, or you don't. I don't think it applies to structures that can grow, shrink, split or merge and have to be able to track its shape, distances between their blocks and possibly other network topology information.

 

[Cervator]

I don't think entities would necessarily be broken up by chunk state if you wouldn't want that? If you have a few networks in a world and each have a controlling entity (that would get loaded in the System and hooked up to appropriate stuff) with EntityRefs to special parts of the network that may or may not be loaded (power supplies, consumers, etc) just do a test to see what state they should be loaded in, if at all, on startup/login ?

We've talked in the past about "ghost" objects that exist in the world yet do not rely on chunks being loaded. Stuff like cities that may or may not have generated at the block-level yet, but could slowly emulate the passage of time and grow virtually - just adjusting the stats that would be needed to do block generation when some cause makes it time to load the city. Could be the same for stuff like rivers. A state somewhere between fully saved on disk + purged from memory and fully loaded in-world.

Admittedly there I'm talking out of place when it comes to current state of the ES, both of you are far more familiar with that than I am. Maybe the saving to disk part is what risks breaking stuff

 

[UberWaffe]

Reasons for not using a single entity

Ah, I had not considered that. I was purely thinking of the "propagating a network change through a very large network" problem.

[Off Topic]

Spoiler

Yeah, I can see a lot of merit in ghost entities (i.e. entities unbound by in-world location.)
Especially in (perhaps not perfect, but 'good enough') simulation of systems in unloaded chunks.
Ex: Civilizations, remote power generators, etc.

 

[Marcin Sciesinski]

Cervator there is no reason to keep a network in memory, and process updates for it, if none of its blocks are loaded and can actually be affected. This is the reason I'm opposed to actually storing the network state in an entity. You either have to store it as an entity in a world (loaded always) or with a block (stored when block is stored). In first case you end up with unnecessary data in memory. In the second case, you have to face shuffling the data around when the chunk with the block is unloaded - move the data to an active block, etc. All of this on top of being unable to have efficient caching code due to entity properties limitations.

UberWaffe It seems you misunderstand how the blockNetwork mod works. It does not propagate a network change through a network. None of the blocks know about any network change, and if a network changes, there is actually very little processing done, which is why I think it should be really fast.

 

[Cervator]

I certainly agree that we don't need the network loaded if none of the stuff it is useful for is needed. If no need, unload

I suspect we'll eventually approach some sort of region approach where we don't keep the whole world loaded in the same block of memory. Lots of stuff could be left unloaded, even "ghosty" stuff that isn't strongly tied to a single location, although it may cover a few chunks and be unsuitable for chunk-level storage.

Still, I most certainly leave this in the hands of you and Immortius - especially on the performance magic to make it all run decently!

 

[UberWaffe]

UberWaffe It seems you misunderstand how the blockNetwork mod works. It does not propagate a network change through a network. None of the blocks know about any network change, and if a network changes, there is actually very little processing done, which is why I think it should be really fast.

Allright then, for the intellectually challenged me.
Where is the "Signal=true/false" value stored?
If a switch is flipped on the network, how does the signal values for the various network blocks get changed?

 

[Cervator]

Where is the "Signal=true/false" value stored?

I suspect in the individual "special" blocks that get updates via events

Say there's a consumer at the end of a cable network and a disconnected producer at the other. On placing the last piece of cable (a place block event) the system code traverses the cable network once to find out what if anything changed, sees the producer is now connected, and powers the consumer nodes (toggles their component data to "on")

That saves us from having to propagate "power" state from block to block every tick. Until the next event (maybe another consumer is added) is triggered that is relevant to the network nothing changes.

At least that's my impression of it, details probably aren't quite exact

 

[Marcin Sciesinski]

SignalProducerComponent component has signal strength - this is the "switch", "torch", "lever" or whatever that "produces" the signal. The value is the range the signal can reach on the network, or -1 for infinite.

Each block that wants to receive signal (be notified of changes to its status) has to have two components: SignalConsumerComponent and SignalConsumerStatusComponent. The first one defines how it evaluates its state (input into the system), the second one has a boolean flag that specifies if the block is receiving the signal (output form the system).

Cervator is quire right. When a block is added/changed its settings/removed, I check, which network it is going to connect to/merge/split and makes the necessary changes to the grouping of blocks by network, then it marks affected networks as "dirty". During a tick, I get through all the dirty networks (only there if anything changed since last tick), I traverse through all consumers in the affected networks and calculate their status in the network. Then based on how the consumer wants to evaluate its input, the system sets the flag (if changed) on the SignalConsumerStatusComponent on affected consumers.

This explanation is not 100% accurate, but should give you an idea. There is a few optimizations done on the algorithm, and if you are really interested, you can take a look at the code in my branch:
https://github.com/MarcinSc/Terasology/blob/multiplayer/mods/signalling/src/main/java/org/terasology/signalling/componentSystem/SignalSystem.java

The benefit of this approach is, that the most expensive (processing-wise) operation is changing the topology of the network (adding/removing block - requiring user interaction), rather than changing the signal of the producer (i.e. "lever switch" - manual, but "timer", "mob proximity detector", "crop growth detector", etc. - automatic). My assumption is, that the latter will happen more often, than the former.

Minecraft (as an example) with redstone networks, to its demise, uses the opposite approach - modifying network is cheap, but changing the signal is expensive, as it causes many block updates over time as the signal traverses the network one block every 2 ticks. I believe that this approach doesn't work very well for Minecraft, especially considering that a lot of blocks use the "redstone pulse" approach, where the change in signal is needed to power some behavior.

 

[Marcin Sciesinski]

I've created a short video showcasing the cables (shapes thanks to Janred), switch and lamp.

 

[UberWaffe]

Marcin Sciesinski: Really awesome! Looking good!

Sidenote: I'm trying to draw up MoSCoW requirements for all incubator threads (First test release, first playable release, full release). Would you like me to do so for this thread?

 

[Marcin Sciesinski]

Sure, I'm not really sure, if there is anything left to be done, apart maybe some changes to the block shapes/textures.

 

[UberWaffe]

Sidenote on MoSCoW:

Spoiler

[+] Means a new requirement.
[-] Means a requirement was removed.
[~] Means the requirement was slightly changed (or just reworded).
[?] Means the requirement is undecided, and requires debate/revision.
An requirement changing priorities would be shown as a [-] on the old priority, and a [+] in the new priority.

--Full release Boolean Signal version requirements--

Spoiler

MUST:

• Have low performance impact
• Allow signal network framework that:
  • Can transmit boolean (on/off) signals
  • Can define emitters of the network signal
  • Emitters have a range component (defining range of emitted signal)
  • Can define consumers of the network signal
  • Can define transmitters/conductors of the network signal
• Implements basic components:
  • Emitters:
    • Switch
  • Consumers:
    • Light
  • Transmitters:
    • Cable

SHOULD:

• [?]Implements basic components:
  • [?]Emitters:
    • [?] Button (Timed high-pulse signal)
    • [?] Pressure plate (Timed high-pulse signal)
  • [?]Consumers:
    • [?] Pistons
    • [?] Dispenser (Ejects an item)
    • [?] Collector (Picks up adjacent item)

COULD:

• [?]Allow signal networks that:
  • [?] Can define sided emitting/consuming/transmission of signals
• [?]Implements basic components:
  • [?]Mixed functionality:
    • [?] Logic block: AND
    • [?] Logic block: OR
    • [?] Logic block: NOT (Inverter)
    • [?] Logic block: Off Delay Timer
    • [?] Logic block: On Delay Timer
    • [?] Logic block: Set-Reset
    • [?] Logic block: Pulse Width Modulator

WOULD LIKE TO:

• N/A

Notes:
In order to do these logic blocks, there would need to be some way of defining multiple inputs (probably sides) to consumers.
As well as being able to define a block as both an emitter and consumer.
Which is why I defined them all as could.


Pulse Width Modulator is a simple definition of how long the output signal is on, and how long the output signal is off. Repeats indefinitely as long as input signal is ON. Given the movement away from edge triggered (OFF -> ON) block activation, this might still be useful in rare occasions. Maybe.

On Delay Timer simply means the input must be ON for at least X ticks before output goes ON.

Off Delay Timer is the same as the Timed high-pulse signal and simply means that even after the input has gone OFF, the output stays ON for X ticks.

Set-Reset simply means that it has a SET input that sets its output to ON. The output remains ON even if the set input drops away. The output only goes to OFF if the RESET input goes to ON.

I think with these basic logic functions (AND; OR; NOT; On Delay Timer; Off Delay Timer; Set-Reset) you can do a massive amount of automation.

In fact, these are the most often used components when I do industrial automation in real life. (On boolean operations at least. I admit to also using edge triggering [negative: ON->OFF] and [positive: OFF->ON] quite often).

So I would vote for those. (Hence why I threw them in as [?] requirements.)

Totally up to you, though.

Also, feel free to use/hack/mutilate the MoSCoW as required.

[EDIT]
As you stated, all the MUST requirements are already done. And even the SHOULD are not that many, and are all up for debate anyway (just suggestions from me at the moment).

Again, I would love to see multiple inputs allowed (in whatever way) and logic blocks implemented. But this is your masterpiece.