Suggested Client identity cloud storage service

[gianluca_n]

Hi everyone!

I'm opening this thread to discuss a feature I have in mind; if there is interest, I could try to implement it in the next weeks (this is separate from my GSoC project, while there are points where the things are connected).

At the moment (correct me if I'm wrong), what authenticates a player to a server is the Client Identity, a key different for each server (identified by a public key) composed of two parts, clientPublic and clientPrivate.
In my understanding this system provides a very good security level, however could be uncomfortable for a player that wants to:

• play with the same account on a server from multiple computers (or maybe multiple OSs in dual boot); this is not something everyone does, but I don't think it's that uncommon either.
• connect to a server using something that isn't the primary game client, like the web interface I'd like to make as GSoC project. As you can see in the proposal draft as a simple workaround I'd implement a console command that can be used in-game to set an username/password pair to tell the server that you'll later want to authenticate with said credentials from the web interface; still, I'm not sure how much this is user-friendly (you'd have to run the command on each new server you join and want to access via web).

What I'm suggesting is to set up a way to allow the users to synchronize their identity certificates across their computers using a cloud-based service. Think of the following scenario:

• A player enters his cloud storage service access credentials (email/password) in client configuration (via GUI)
• The player joins a server he/she never joined before
• During the connection handshake the keys are generated and stored in the client configuration
• The client acknowledges this and uploads the new keys to the cloud (if the user is OK with this, obviously)
• When the user later launches the client on another computer, he's asked if he wants to synchronize the keys; if the answer is yes, he enters the cloud storage credentials and the client downloads the keys, and when he joins the same server he joined on the other machine, he will be recognized as the same player
• If he wants to access the web interface I will implement if my GSoC proposal gets accepted, this could have an option to get the keys from the cloud after the user logged in with the cloud credentials.

Getting to the technical point, I can think of two ways to realize this:

• Custom cloud service: we set up a small web service where users can register and then enter the credentials in the game client to perform the synchronization. The keys could also be encrypted client-side with some symmetrical algorithm with the password chosen by the user and entered in the game client. The problem is providing hosting resources - I'd be OK to host this at my expenses for the initial phase (I have a DigitalOcean 50$ coupon from Github Student Pack yet to use), but I'm not sure how well it would scale.
• Using the APIs of services like DropBox, Google Drive, etc and put a JSON file with the keys - or the whole configuration - there. Problem, however, is that they use OAuth which doesn't play well with open source projects, so I'm not sure if this is feasible.

What do you think? Any idea/critique is welcome!

 

[Skaldarnar]

I really like the idea you are presenting, and imho we have to go this way sooner or later. Ideally, we could rely on existing systems and users are free to choose their cloud storage of trust. As you say, this might lead to problems from the implementation perspective. Maybe we have to investigate in this direction further?
If we introduce a simple service where users can register we should think about it in the long term. This should be their Terasology (or MovingBlocks) account they can use to to log-in (if they want) and share their credentials over different OS or devices. Do we have to take anything special into account when going this way? How "much data" is produced for each client when logging into different servers?

 

[gianluca_n]

Ideally, we could rely on existing systems and users are free to choose their cloud storage of trust. As you say, this might lead to problems from the implementation perspective. Maybe we have to investigate in this direction further?

I found some discussions and blog articles around the Internet about this topic but not a real solution - the API providers want developers not to divulgate private OAuth keys. For example, Google (which could be used for Drive in our case) explicitly states that developers should "make reasonable efforts to keep their private keys private and not embed them in open source projects"... an "effort" could be embedding them in a closed-source binary blob but this won't be a real solution since it would be quite easy to extract them anyway (a Java .class can be decompiled, on a native binary the strings utility can be used); as far as I know this is however the way closed source applications handle it.

Do we have to take anything special into account when going this way? How "much data" is produced for each client when logging into different servers?

Would be interesting to hear from people who wrote the authentication code; anyway, what I personally found out (please correct me if I'm wrong) is that every peer (server or client) is identified by a pair of public and private certificates; the relevant part of the client's configuration maps every server's public certificate to the certificate pair used to connect to that server. I think that a database could store the server's public certificates "globally" and then associate them to the keys of the users which decide to use the service. So we would have to store, for each server, it's public certificate and a pair of public-private certificates for each registered user which has connected to it at least once. A single certificate is made of two BigIntegers, with a public one being slightly bigger than a private one due to including an ID too (a short string); in the JSON configuration a certificate is about 1.5k characters including the labels, so it could probably take about 1KB or less in a DBMS with proper support for large integers.

So in my opinion it should not be much data at all; however, I'm a bit afraid there could be flaws in my reasoning, so I kindly request anyone which notices something not OK to point it out. Thanks!

 

[gianluca_n]

If anyone is interested, I'm working on a basic implementation of a client identity storage server here (WIP, at the moment it's unstable; needs tests, and needs work for more meaningful error codes).

This is the original Entity-Relationship diagram I figured out for the database:

I then merged the Client Identity and Private Certificate entities; this is the version I'm using in the implementation:

Relevant files in the repository are:

• API specification (still very high level, needs more detailed explanation of the JSON-based data interchange format)
• Database structure

The bigintegers at the moment are stored as PostgreSQL byte arrays and for transport over JSON they are encoded to (and decoded from) base64 strings. For the client it should be quite simple to interact with the server thanks to the BigInteger toByteArray() method, constructor from byte array, and Java's Base64 utilities.

The HTTP server implementation is based on a simple Node module I made which maps HTTP endpoints to Postgres stored procedures, where the business logic is implemented. I think it can be a good fit here since this is a quite simple CRUD application, but as always any sort of feedback is appreciated.

 

[gianluca_n]

Hey everyone!

I finally reached a working version of this feature.

If you are interested, here is the branch diff for the client (going to open a pull request soon) and here my server implementation in Node.JS + PostgreSQL (from the API specification it's possible to develop alternative implementations if someone doesn't like the stack used).

The registration happens on a web page (maybe I could later add the possibility to register in-game if someone wants this feature), here is a screenshot:

Once you register, you can login in-game:

As you can see, the whole thing is meant to offer alternatives to user, e.g. not a single server, but you can enter the URL of the server you want to use and, since it's open source, you could host one yourself.
As soon as you log in, and every time you run the game, the client identities are synchronized with the service. Then, every time you connect for the first time to a new game server, the generated identity is uploaded to the service.
All the requests made to the service are asynchronous (made on their own dedicate thread) to avoid locking the UI.

Note that the system currently doesn't manage conflicts (e.g. if there are two different client identities for the same game server on the client and on the service). This scenario is somewhat unlikely to happen but definitely possible, for example if the user connects to a server with a client where he's logged into the service, then connects to the same server using another client (maybe on another computer in a real scenario, but can be simulated using a different data directory) without logging in - so the game server generates a new identity - and then logs in from the second client.

Now I'm going to:

• Set up a public service instance on a VPS so you can try it without having to install locally yourself Done: http://165.227.140.7
• Improve the user experience by reporting the success/failure messages to the game console and maybe show popups (at the moment they are only logged to stdout) Done: The MiniChatOverlay is used to show notifications.
• Implement conflict handling - this one is a bit tricky but should be feasible, maybe a good option is to ask the user which identity should be kept, the local or the remote one Done. The user can solve the conflicts (whether to keep the local identity or the one stored on the service) when opening the Join Game screen.

As you can see actually to register you only choose a nick and a password (no way to retrieve it if you forget!). Maybe it should be switched to email, to allow a "forgot password" feature? Also, I'm considering to synchronize the player name to the service too, since at the moment if you connect, in sequence, with two different clients using the same account, you will be the same player on a server but appear with different names (according to each client configuration) unless you manually set them. Probably switching to email improves this detail too, if it is implemented, since having two different names - one for logging in and one which will be shown in-game - might be a little confusing to new users.

Any opinions?

 

[Cervator]

Tested it out - works great!

I made an account on the test server you established, then started a headless server locally + logged in with a first client. Placed myself somewhere notable and left the game.

Started a second client (both entirely clean - deleted config first), logged in, joined the same headless server and found myself exactly where I had been with the first client! Excellent

Suggestion: Move the identity button to the Player Settings and include a notice on the "First time loading multiplayer" popup that to maintain your identity consider logging in to an identity server under the player settings. Maybe even put a shortcut button there or something.

 

[gianluca_n]

Hi there,
recently I made various improvements (conflict management, notifications, password recovery) both to the server and to the client, which is now on this pull request.
Also, a running server instance is available here: http://165.227.140.7

As suggested by @msteiger about a week ago, I made a short video to show the implemented functionality:

I'm mainly focusing on FacadeServer at the moment since it's my GSoC project, but obviously I'll still improve this if necessary (e.g. bugs, code improvements to the pull request).

 

[Cervator]

The PR for this functionality is now merged so it is available in-game!

We still need to figure out how (and where) to document this well, plus where to actually host the service long term. Since it runs in Docker maybe we can add it to the Weblate droplet? Since it already has Docker set up. Any idea how much more resources it would consuming?

Poking @qwc and @msteiger for the server side ideas.

 

[gianluca_n]

Thank you for merging!

Not sure I understand the "since it runs in Docker" - the server I written doesn't, at the moment, as far as I know (unless someone already set up the image and I can't find it). If you are interested, I'll set up an image so that you can easily run it on your servers (for example the same which runs Weblate as you suggest) without the manual setup which is a bit tedious. I have to learn a bit about Docker before, I know what it is but never used it. Doesn't look anything too difficult, anyway.

Unfortunately, I can't give meaningful estimates of the resource usage; my instance currently running at http://165.227.140.7 is on a small (5$) DigitalOcean droplet which is almost idle all the time, but this isn't a very useful information since there is almost no traffic, and I don't know how I could estimate how it would scale for a large number of users. I can try to provide some disk space usage estimate but I'm not sure it's the correct way of calculating it:

• A public certificate should be about 3.3KiB. This is obtained by dividing the table size of the running instance, which has some records in it, by the number of records. In my understanding this should be accurate enough since while there are variable sized columns (BYTEA data type for the big integers), they will have the same size for each record since the numbers (modulus, exponent, etc) generated when generating the certificates in the engine are always the same length.
• There is a public certificate for each server, and one for each client identity. For each client identity, there is the private certificate too; using the same method described before, a record in the client identity table (which includes the private certificate and the references to the client's and server's public ones) is about 3.4KiB.
• Then there are the user and sessions records; averaged from the various test accounts I made, an user record uses about 7.1KiB, but in this case the measurement probably isn't very meaningful since the usernames and emails lengths are variable. A session is around 2.7KiB.

But probably this is useless without an estimate of the number of users, the number of servers and the number of identities for each users (which is the number of servers a user visited).

 

[Cervator]

Oh, I'm sorry about the Docker misunderstanding. Maybe I'm thinking of the telemetry project instead

On resources I should have narrowed it down to memory usage - yeah I can imagine CPU and disk space will be minimal for the near future. And if it runs on a $5 droplet then that's not a lot of memory in any case

 

[AlteOgre]

I've tried to register here, but there is no effective mail service providing me a token. Hence I cannot enter the token supposedly sent to me, to complete the registration. Some temporary malfunction?

 

[gianluca_n]

Hi @AlteOgre, thank you for reporting the problem.

We are aware that there are some problems with the outgoing mail from the server. This is related to spam filters and mostly happens with larger providers, for example I can register using my university mail address (I get the confirmation token).

We discussed this in chat some time ago, but haven't found/decided what's the proper solution yet (there are roughly two options - trying to use a gmail address as sender, or researching how to get past spam filters with the current domain).

 

[AlteOgre]

Okay. I had been trying to register using a gmail address. So, if I understand it correctly, if I'd use another mail address domain, chances may increase registration will run as planned? Well, I just tried that, and registration now worked. Not ideal, but effective anyway. But ... logging in hasn't been succesful. Five attempts, also after client restarting, all lead to connection failed, timed out. Other issue at hand?

 

[gianluca_n]

I tried but I haven't been able to reproduce this issue. The fact that you mention connection failures leads me to think the service URL is not correct - what you should enter is exactly https://utility.terasology.org. If from the screen shown in the attached screenshot I click Ok, it works for me (well if exactly as in the screenshot I get Invalid username or password, but this is a response from the server so the connection is working).

 

[AlteOgre]

That was the cause indeed. I mistakenly assumed another url was required there. Works fine now. Thx.