Moving Fast, Without Breaking Things: Modern Software Delivery with Feature Flags

This article is part of Scaling Feature Flags: A Roadmap for Safer Releases & Faster Development.

Where we stand today

No matter what industry you're in, the pace of change keeps ratcheting up. Organisations see increased competition from both established incumbents and nimble upstarts, while at the same time the external landscape these businesses operate in seems increasingly turbulent—regulations change, supply chains are shocked, revolutionary technology appears seemingly overnight…

To respond to all this change, enterprises need to become more nimble—they need to be able to make quick decisions, experiment, and adapt. And given the increasingly digital nature of every enterprise, this naturally leads to higher and higher expectations around the pace of software delivery. 

But while expectations around the pace of software delivery are increasing, expectations around quality, security, and compliance remain the same, particularly for larger enterprises operating in higher regulatory environments. 

‍

Speed vs. safety: DORA research in Accelerate

On the face of it, this appears to be an unresolvable conflict. Surely accelerating the pace of change means accepting a trade-off in terms of risk—higher risk for bugs and errors slipping past QA, higher risk for deployment incidents, higher risk for security vulnerabilities. "Move fast and break things", as the saying goes.

‍

Happily, that's not necessarily the case. Over the last couple of decades, leading software organisations have demonstrated that accelerating the rate of change can actually reduce risk. It turns out that, when you use the right practices, you can "move fast, with safety". 

‍

This claim—that a higher pace of software throughput can lead to greater stability—may provoke some healthy scepticism from yourself or your peers. It certainly sounds a little counterintuitive. However, this idea is backed up by solid empirical research, as detailed by Dr Nicole Forsgren and her DORA team in their book Accelerate. This research group applied statistical modelling techniques to data from over 23,000 survey responses, collected over the course of multiple years, to try and understand the practices that organisations around the world use to build software, and how well those practices work. 

One of the most interesting findings from this wide-ranging group was the correlation between a higher pace of software throughput and higher levels of stability and quality. It's also worth noting that their findings didn't seem to be strongly affected by the size or type of organisation—big or small, regulated or not, the same correlations were evident.

The most exciting part is that the DORA team didn't stop at just looking at these performance metrics; they also looked at the practices these different organisations were using. They identified a specific group of "high performer" organisations who were differentiated on a key set of software delivery metrics, and most importantly they were able to lay out a set of software delivery practices that these high-performing organisations used to achieve speed with safety. 

The obvious question is, what are these organisations doing that allows them to accelerate their pace of software delivery while still maintaining quality and stability? In large part, we can say that their success lies in a set of practices that accelerate the feedback loop between code being written and code being deployed— a set of practices often referred to as Continuous Delivery. This cluster of techniques works together to achieve an interconnected set of principles:

• reduce manual work in testing and deployment activities
• work in small batches
• keep codebase in a deployable state
• architect systems for decoupled deployments
• make frequent deployments to production

What the DORA research showed us is that organisations that focus on these Continuous Delivery practices are able to move faster than their competitors, while still maintaining safety and quality. 

‍

Where feature flags come in

One practice that is near-ubiquitous within these high-performing organisations is feature flagging.

Feature flagging facilitates a central tenet of Continuous Delivery: separating deployment from release. If a team wants to work in small batches and make frequent deployments to production, they are confronted with a challenge: what to do with work that doesn't fit into one of those small batches. Let's say the team doesn't want to accumulate more than a week's worth of code changes without a production deployment. What do they do if they have a feature which is going to take two weeks to implement? The answer is that they incrementally merge and deploy their changes all the way to production, despite it still being a work-in-progress. That's ok though, because the changes are deployed as latent code, inactivated behind a feature flag. Then, once the feature is fully implemented (and tested) the feature flag is flipped on and the feature is released. This is what we mean when we say deployment is separated from release—the implementation of the feature is deployed incrementally, but the feature is only released when ready.

We shouldn’t stop here.  While Continuous Delivery can transform a team's way of working, feature flagging can do a lot more. It’s also a central part of practices like A/B testing and experimentation, helping to level up the entire product delivery process. On the operational side of things, feature flagging unlocks advanced techniques like canary releasing and dark launching; another way for organisations to move faster, with safety.

‍

The evolution I've seen

I first started talking to engineering leaders about feature-flagging in 2016, and have served on the OpenFeature governance board for the last three years. Over the past 10 years,  I’ve had the opportunity to see first-hand how, as companies navigate an increasingly challenging and dynamic landscape, feature flags keep on showing up to enable a practical balance of speed and safety. It’s been gratifying to watch the adoption of these practices grow from a quirky idea evangelised by some upstarts in Silicon Valley to a very mainstream technique suitable for even the most mature engineering organisations. 

‍

Proceed with care

I’ve also seen that these benefits don’t come for free—feature flags do have a carrying cost, and can contribute their own technical burden if not managed intentionally. This is especially true at scale and for large organisations. If, however, your feature flag adoption is done strategically, with practitioners learning the correct technical approaches along the way, then the agility and speed that can be gained is undeniable. For a roadmap of how to safely and effectively do this at scale, read the full eBook.