Server infrastructure and supporting services

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
I put together a nice sketch of our current server setup + ideas for improvements (in blue). Am really liking sketchboard.io, it lives on there (account needed - I can add yours to the team, let me know), and I exported a .png to TeraMisc for reference - really should start using that repo more for infrastructure stuff.

infrastructure-overview.png

This might be mainly of interest to @msteiger and is the thing I wanted to put together today to get an easier overview of what we have and where we might want to put stuff with Heroku not being a good option for tiny services anymore.

For good measure and completeness I added everything I could think of, including some ideas for improvement (in blue). Did I miss anything? Didn't include IRC/Slack or draw lines for Gooey as that'd probably get too messy.

It is amazing how much easier it can to be think about stuff when you have a good chart!
  • Suggests using a single 1/2 GB droplet to host a dedicated MySQL DB for the other servers to connect to. Might help shave off a little memory use on the tight droplets, although I could be wrong and the DB process memory usage there minimal
  • Suggests Gooey and the master server list could use said new dedicated DB instance, although really they could probably just use one of the existing DBs if we keep them - likely less work. Either actual process is minimal - just throw on an existing droplet / into an existing web container. Maybe Gooey would benefit from running out of Jenkins itself.
  • Shows how we could introduce the use of Bintray to host both release binaries from Jenkins (game zip) and jar files from Artifactory (libs and modules)
  • Show where the Module Index & Manager concepts fits in - the manager would read from the index + Bintray and in turn be used by the game/launcher/module site
  • Includes other services that seem related for various integration ideas and other stuff we could do like provide hosting for light-weight stuff like IRC bouncers or email forwards for Contributors
I hooked the 4 main servers into New Relic for monitoring, which is suggesting Artifactory is only at 61.7% memory usage, yet when I tried to install the monitoring agent I had to stop Artifactory first or enough memory couldn't be allocated for the install, heh ..

My personal ultimate goal is still to manage everything server related via Chef or so, designing cookbooks/roles for our different components. With that in place we should be able to move stuff around fairly easily. For instance if we come across some cheap hosting with a big beefy single dedicated server we could just apply all the roles to it and everything would install together on there (taking some care to avoid port conflicts and reuse DBs/containers)
 

Cervator

Org Co-Founder & Project Lead
Contributor
Design
Logistics
SpecOps
@msteiger: @PrivateAlpha dug up http://scalingo.com which looks like a Heroku competitor that offers three tiny free instances per account before they start changing. Maybe something to look into for the master server list?

We could make a shared org account if it shows promise, then host from there.
 

msteiger

Active Member
Contributor
World
Architecture
Logistics
That's indeed quite interesting - I heard that Gooey signed up already!
 
Top