One of the first tools that we developed in-house was used to distribute test builds of DRG before release. We called it Beam, which naturally comes from the U.F.O. tractor beams which are used to move the build from the servers to the desktop. The tool was initially built in around two weeks over the Christmas period in 2016.
I used a plethora of other open technologies for the distribution process, at the core was rsync which delivered the incremental build updates and was pretty essential, since, believe it or not, the internet wasn’t all that quick in a number of our working locations. This also made the server and security aspects easy, we simply deployed an EC2 Linux box with OpenSSH and I say “simply” because it was indeed simple using our already established automation pipeline powered by the open source Puppet server.
You might have spotted in the screenshot that we keep an archive of builds, which means that we can always switch back and forth between various versions and branches. As you might imagine though game builds can get pretty large, ours is actually only 7GB or so, but when you’re paying EBS by the gigabyte-month, you might think twice about how many builds to store. This is where our secret ingredient comes in, we used another tool “bup” to store the builds more efficiently. It’s actually designed as a backup system, but it provides an effective block level deduplication system which uses a rolling checksum for variable block sizes. The whole tool is based on the git packfile format (but not git itself so you can even store huge files if you like). Naturally it provides excellent versioning features. The final part of the puzzle was the ability to mount any build version to a FUSE filesystem which allowed it to be accessed transparently by rsync. Before long we had 100+ builds across multiple branches all stored in less than 30GB space, saving well over $70/month we could now even afford a few beers to celebrate.
The look and feel of the tool hasn’t actually changed all that much since version 1, we added some DRG branding, a game launcher and some metadata for the builds (stored in DynamoDB). Here’s what it looks like today:
The front-end definitely delivered, with minimal development time a tool was born. The original prototype was implemented mostly in a single HTML file and that was driven by jquery. The simplicity of it all was quite elegant but the engineering certainly wasn’t; it wouldn’t scale to more complicated tools and for a moment it was as though I’d forgotten all about MVVM, MVC, MVC-VM patterns, let alone something trendy like immutable application state or stateless components. In fairness, this was quite deliberate, I wanted to prove the concept without becoming bogged down with engineering but with the next tool, I decided to get serious… join me in part 3 where I’ll be talking about TypeScript and React and possibly trying to hold back my opinion of gulp.