After many, many months of working crazy hard to get server scaling working, I have a solution! But first, lets go over the things that didn't work...
I initially started with Amazon's Game Lift service built on the AWS platform. I spent literally days reading through their cryptic and often out-dated documentation, compiling libraries, etc. This went on for weeks until I finally realized that the version of Unity that I had built the game on (5.5) doesn't work with the AWS libraries. Dead end.
Then I went back to Azure which I've used for other projects, and my day-job. I saw that they had scalable server sets, and was eager to use those. Scaling server sets are a series of servers that can be increased in size, or decreased in size, depending on requirements.
These servers were provisioned by running a shell script on each VM. However, here's where the trouble began. I could NOT get these scripts to function correctly. There were so many things that needed to be configured for each VM, and updates to the script did not correctly propagate through to the VMs.
This was such a headache, and really a hard time for the project. So much time and effort put towards progress, and so little progress actually made. Then... one late night, at 11:30 PM... a revelation.
I had been writing an API for my day-job, and I thought, what the heck, an API is just a way to call server-side stuff from the web. So I wrote an API for my game..... in 30 minutes. This was made possible by realizing that all I needed to ask (for now) were three simple commands:
So I wrote my own API, created my own servers (on azure) with minimal provisioning, and was able to run the game. I even have my own development and production environments!
The diagram in this post shows the general flow of things in Salt the Earth. Players connect to the master server (seen on the right). The master server starts a game server (seen on the top right). Players connect to the game server and play their match (seen on the left side), and then the whole process starts over from the beginning.
Anyway, after many months of struggling, this is finally done. Now I have a ton of old code to fix. We'll get there, a day at a time.