Google Cloud Storage vs. Amazon S3
3 min read

Google Cloud Storage vs. Amazon S3

A quick review in serving static content.

TL;DR: Google Cloud Storage didn’t work.

Here’s the problem: I have a web service and I want to serve “important” static content as quickly as possible. That means Javascript and CSS.

The solution is quite simple and well-known:

  1. Minimize the size and number of JS and CSS files for the initial download. This effectively means running the Javascript through a minifying process (UglifyJS or Closure Compiler) and combining all files into one, and doing the same with CSS.
  2. Serve them as no-expiration, publicly cacheable files. Publicly cacheable means that proxies and browsers are free to cache the files anywhere or any time. Contrast this with private caching which only caches on the end user’s browser. Of course, the trick here is that since the files are going to be cached, they cannot be easily changed. To solve this, you simply create a unique filename for each JS and CSS file that is initially downloaded. An MD5-fingerprint suffix is sufficient here.

So far so good. The only problem remaining to be solved is where to host the files. It’s best to use some form of edge cache (location closest to your users as possible). This day and age, we can stand on the shoulders of giants and develop on both Google and Amazon’s infrastructure as a service. Notably, Google Cloud Platform and Amazon Web Services.

My goal was to integrate the upload of my JS and CSS binaries to the storage system as part of my release script (which is in Gradle btw).

Given I knew exactly what I needed to do, I decided this would be a great opportunity to try both and document my experience.

First, I worked at Google for five years. I know many of the individuals that work on this product. They’re all smart and awesome. This is in no way a reflection of them but instead what I believe is a lack of leadership in place to tie all of the new and legacy components together into a cohesive product.

I didn’t really care about pricing as they’re both comparable and I’m serving small files so storage and network costs are negligible.

But oh dear god, the flow for setting up Google Cloud Storage was so utterly and completely broken. Even though my application is on AppEngine, the two simply do not play nice together. I couldn’t use the app ID from my AppEngine application as my project ID for GCS and so needed to setup an entirely different account with its own billing. Of course, that didn’t work. Once I signed up for billing, I naturally tried to setup a storage bucket but was greeted by the following opaque error message:

The account for this specified project is disabled.

What? There was nothing left for me to do here. No amount of searching around helped either. I was out of luck.

Naturally I tried this several times, each time greeted with the following beautiful UI.

Who felt this was good enough? 4 hours later, I gave up.

Amazon S3

I setup a new AWS account and signed up for S3 in under 5 minutes. 15 minutes later, I had created a storage bucket, decided on my directory structure and had uploaded test files via a command line script. 30 minutes later, it was completely integrated into my build environment thanks to this Gradle plugin.

And that’s it. Next step is to put CloudFront in front of S3 and I can move on with life. And while I’m certain GCS would have suited my needs perfectly, the ease of setup and simplicity in AWS far outweighed any marginal price or performance gains I may see from GCS, plus I can always try and switch to it later. There’s not much else I have to say here.

Love or hate this article? Let me know @davidbyttow.

Enjoying these posts? Subscribe for more