Continuous Deployments. An inside look.

An important part of the customer experience with software is how and when software updates are delivered. In days of old, we’d wait patiently for years while Microsoft toiled in isolation on their latest and greatest creation. Today, however, people are delivered updates in software on a more continuous basis.

At Apple, the operating systems update each year. At Facebook, new updates are delivered once per day. At Etsy, though it may not be obvious, updates are delivered sometimes 50 times per day! The speed of improvement is astounding, but it leaves me wondering, “how do they do it?” I explored this question a little and here’s what I found.


The following is taken from a talk given by Chuck Rossi, lead of the Facebook Release Engineering team. With a release engineering team of only 3, it’s no surprise that they have invested heavily in tooling. Also interesting, Mark and the other cofounders built much of the continuous build and deployment system that is still being used today. This approach has always been part of Facebook’s DNA.


There was a great article in Wired recently which detailed the story of how LinkedIn took a significant business risk to modernize their development and deployment process.

Shifting from feature-branch-based development to the new continuous deployment system required halting all development for two months as LinkedIn trained staff, migrated old code, and built out the automated tools it needed to make the new system work.

“It was a pretty big risk the business took,” says Scott, “to look at its engineering team and say, ‘we’re going to completely change the way we do software… and somewhere in the middle of this two-month process you’re going to run across a bridge and burn it behind you.”


Here are some high level comments regarding the Netflix approach to releasing software. This won’t surprise anyone who also read the Netflix culture document.

There’s virtually no process at Netflix. They don’t believe in it. They don’t like to enforce anything. It slows progress and stunts innovation. They want high velocity development. Each team can do what they want and release whenever they want, how often they want. Teams release software all the time, independent of each other. They call this an “optimistic” approach to development.


I didn’t find a ton of detail written about the Google release process, but here are a few little bits about their approach, taken from their engineering blog (2011)


There is much written on the Etsy approach to continuous deployment, and I’d consider them one of the more extreme examples with their 50-60 prod pushes per day… wow.

I think one of the more interesting things that I learned while conducting this research is that although these companies push real code to production daily, it doesn’t mean that each and every push results in an active change for the customer. A big part of this approach is the ability to push changes that are turned off in production which allows the functionality to be enabled when the formal software “release” happens, and can be aligned with go to market activities. For example, Facebook said that in some cases, the next 6 months of features are already in production today, just waiting to be enabled.

I certainly didn’t seek out examples of more manual deployment approaches. No doubt there are many good companies still doing things a little more manually and less frequently. However, the examples above are of companies with quality products, operating at high velocity and incredible scale. I think the ideas they’re using to make continuous deployments work should be inspiring to all as we try to reduce the pain of deploying software and start releasing value to customers on a more continuous basis.


Now read this

Inside Facebook: iOS Development

This is an absolutely fantastic inside look at how Facebook builds their iOS app, and the path they took from web to native. I have summarized some of the standout info, but encourage everyone who’s interested in native iOS development... Continue →