DevOps and Agile: Understanding the Relationship

Agile development used to be front and center in the conversation about software development. Now, DevOps has taken over the conversation. How do agile and DevOps relate? Both ideas began as ways to improve different aspects of software development. Agile embraced the changing nature of requirements and prioritized working software over rigid processes. DevOps collapsed the development and operations silos to improve both development and production operations. Each shares some fundamental ideas, but each target different stakeholders and set different business goals. Given the overlap, people ask is DevOps agile or if we do DevOps do we also need to do agile? Or, is agile even still relevant? Or, what’s the point of either of these things anyway? This post answers those questions and also demonstrates how DevOps is different than agile and that both are still useful, and required for successful software development. (You can also watch our Recipe for DevOps Success webinar and read our post on The Four Tactics for Cultural Change in DevOps Adoption to learn more about what can be achieved in the enterprise with DevOps.)

Context and Background

Agile development traces back to the 1990s with practices like scrum, extreme programming (XP), feature-driven development, and user stories. The agile manifesto was written in 2001 by prominent software developers like Martin Fowler, Kent Beck, Ward Cunningham, and Bob Martin. The primary section of the manifesto reads with bold highlights:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

Agile was a response to constraining software development processes like waterfall. The idea was to be flexible, iterate, work with stakeholders to produce software that met their requirements while also increasing predictability. Generally speaking, agile development is a set of practices for developing software in teams and largely focused on planning and requirements.

DevOps traces back to the late 2000s and early 2010s. Organizations tended to have dedicated development and operations teams where development threw their compiled code over the wall to operations for them to run it. This created problems since operations was disconnected from the code they’re supposed to support in production, and development was also disconnected from the operational aspect of their software. Each group had conflicting goals. Operations wanted stable environments and to minimize changes. Development needed to ship features as quickly as possible to build new product features. These two groups were at odds in fundamental ways. The DevOps idea was to break the silo between operations and development (hence “dev” and “ops”) and replace it with a shared feedback loop.

Today, DevOps is an umbrella for technical practices like continuous integration, continuous delivery (or deployment), high levels of automation, and cross-functional teams.

There’s no official manifesto, but the DevOps Handbook is the best reference for the ideas and practices behind DevOps. The DevOps Handbook documents the three ways of DevOps:

  1. The Principle of Flow: decrease the time from commit to code running in production. This metric is called lead time.
  2. The Principle of Feedback: increase the feedback from production back to development (through practices like real-time monitoring and automated alerting)
  3. The Principle of Continuous Learning and Experimentation: continuously improve and evolve processes

Each principle mandates technical practices such as building a continuous delivery pipeline, adding required production telemetry, and conducting outage post-mortems.

Simply put, Agile tackles the problem of building software and DevOps focuses on getting code into production and improving that process. Each addresses a different—but equally valuable—part of software development. Agile set the stage for DevOps. Agile provides teams a way to build software faster and thus deploy more often. DevOps provides teams a formula to deploy more often and increase quality.

How Agile and DevOps Differ

Agile and DevOps each target different actors in the SDLC. Agile covers project management and requires strongly defined roles like Product Owner. DevOps targets more technical work and requires engineers to accept shared responsibility for building and deploying software while requiring management and product owners to think of their software in a certain way. Here’s how this plays out in practice.

There are plenty of different methodologies such kanban or scrum. Both focus on the procedure for accomplishing tasks—regardless of the technical details. Kanban has a set of principles to limit work in progress, track bottlenecks, and promote visibility. Scrum aims to make software development more predictable by setting a fixed time and scope for a sprint, then letting the team self organize and work through the issues as they arise. This is where many people encounter agile development and it is often driven by whatever software the team uses to track work.

DevOps comes in when code needs to be written. The first principle of DevOps focuses on getting code into source control and then into production. It makes no assumptions about the process used to spec out customer requirements or how to prioritize a backlog. The premise is simple: whatever work is done should ship to production as quickly as possible. However, a successful DevOps implementation requires back-pressure from development that agile methodologies supply. This is the opposite of waterfall development, which is incompatible with DevOps and for with is the antithesis of Agile.

Agile calls for fixed roles and responsibilities. Product Owner (or PO) is arguably the most important role since the PO is responsible for setting requirements and clarifying how the product should work for engineers. Scrum uses a dedicated scrum master to manage the process. There are even certifications for scum masters. DevOps makes no proclamations about dedicated roles. DevOps is closer to a set of behaviors than a defined role. In fact, there’s a trend in the industry to post job “DevOps Engineer” roles. This goes against the DevOps ideas of shared responsibilities and technical practices. DevOps cannot be a dedicated role.

Agile-Backed DevOps is the Way Forward

It should be clearer now how Agile and DevOps are both ideas and apply to different areas in the SDLC. Agile is a framework for building products and DevOps is a set of technical practices for deploying and running production systems. Given these two ideas are different and equally powerful, you should adopt both in your SDLC.

Agile practices such as pair programming is a fantastic way to distribute knowledge and improve code quality, regardless of what the code is for. Such practices can produce new features, fix bugs with internal tooling, or lead to infrastructure as code in your deployment pipeline. DevOps calls for shared responsibility and understanding of the deployment pipeline and other associated infrastructure. Pair programming is a great way to expose engineers to this environment and grow their knowledge.

Again, agile development practices apply to most software projects be it working through the product backlog or building out internal tooling. All software projects benefit from faster cycle times and quicker feedback from production. In fact, that’s key to a successful agile process. The agile process focuses on stakeholder feedback from interacting with working software. The faster working software is released to stakeholders, the faster changes happen, thus the overall process accelerates. Adopting agile without bringing in DevOps principles will inhibit your organization’s velocity and slow your software release cadence.

Accelerate – The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations researches the traits behind the best performing teams. The authors find that implementing DevOps practices such as trunk-based development and continuous delivery are clear indicators of high performance. Their 2017 research found high performers compared to low performers have1:

  • 46 times more frequent code deployments
  • 440 times faster lead time from commit to deploy
  • 170 times faster mean time to recover from downtime
  • 5 times lower change failure rate (1/5 as likely for a change to fail)

The first two points stand out since they demonstrated just how fast an agile process backed by DevOps may be. These factors build on each other to create more successful businesses. In fact, Accelerate also found that high performers had 50% higher growth in market capitalization over three years compared to low performers.2 Again, bear in mind that these achievements are possible when the agile process provides adequate back-pressure to DevOps behaviors.

Conclusion

This post covered the relationship between Agile and DevOps along with their historical context and target area. Just by looking at the history, we find that DevOps is a technical continuation of Agile’s ideas. Both philosophies favor fast iterations and quick reactions to changing conditions. They’re similar ideas but target different phases of the SDLC. The most successful teams apply agile development practices and DevOps deployment practices. The odds are that your organization is already working with agile development in some way. Improving your DevOps practices is the next step to boosting velocity and quality in your organization. Reach out to us to request a demo and learn how we can guide you in your digital transformation journey.

  1. Forsgren PhD, Nicole. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations (Kindle Locations 434-436). IT Revolution Press. Kindle Edition. ↩︎
  2. Forsgren PhD, Nicole. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations (Kindle Locations 2734-2735). IT Revolution Press. Kindle Edition. ↩︎
Avatar

Written by

Adam Hawkins

Passionate traveler (currently in Bangalore, India), Trance addict, Devops, Continuous Deployment advocate. I lead the SRE team at Saltside where we manage ~400 containers in production. I also manage Slashdeploy.

Related Posts

Alisha Reyes
Alisha Reyes
— July 22, 2019

Cloud Academy’s Blog Digest: July 2019

July has been a very exciting month for us at Cloud Academy. On July 10, we officially joined forces with QA, the UK’s largest B2B skills provider (read the announcement). Over the coming weeks, you will see additions from QA’s massive catalog of 500+ certification courses and 1500+ ins...

Read more
  • AWS
  • Azure
  • Cloud Academy
  • Cybersecurity
  • DevOps
  • Kubernetes
Avatar
Adam Hawkins
— July 17, 2019

How to Become a DevOps Engineer

The DevOps Handbook introduces DevOps as a framework for improving the process for converting a business hypothesis into a technology-enabled service that delivers value to the customer. This process is called the value stream. Accelerate finds that applying DevOps principles of flow, f...

Read more
  • AWS
  • AWS Certifications
  • DevOps
  • DevOps Foundation Certification
  • Engineer
  • Kubernetes
Avatar
Adam Hawkins
— July 9, 2019

Top 20 Open Source Tools for DevOps Success

Open source tools perform a very specific task, and the source code is openly published for use or modification free of charge. I've written about DevOps multiple times on this blog. I reiterate the point that DevOps is not about specific tools. It's a philosophy for building and improv...

Read more
  • Ansible
  • Chef
  • configuration management
  • DevOps
  • devops tools
  • Docker
  • infrastructure-as-code
  • Kubernetes
  • telemetry
Avatar
Adam Hawkins
— July 2, 2019

DevOps: Scaling Velocity and Increasing Quality

All software teams strive to build better software and ship it faster. That's a competitive edge required to survive in the Age of Software. DevOps is the best methodology to leverage that competitive advantage, ultimately allowing practitioners to accelerate software delivery and raise...

Read more
  • continuous delivery
  • DevOps
  • software
Avatar
Adam Hawkins
— June 13, 2019

Continuous Deployment: What’s the Point?

Continuous Deployment is the pinnacle of high-performance software development. Continuous deployment teams deploy every commit that passes tests to production, and there's nothing faster than that. Even though you'll see the "CD" term thrown around the internet, continuous deployment a...

Read more
  • Development & Deploy
  • DevOps
Avatar
Adam Hawkins
— May 31, 2019

DevOps Telemetry: Open Source vs Cloud vs Third Party

The DevOps principle of feedback calls for business, application, and infrastructure telemetry. While telemetry is important for engineers when debugging production issues or setting base operational conditions, it is also important to product owners and business stakeholders because it...

Read more
  • Analytics
  • DevOps
Avatar
Adam Hawkins
— April 16, 2019

The Convergence of DevOps

IT has changed over the past 10 years with the adoption of cloud computing, continuous delivery, and significantly better telemetry tools. These technologies have spawned an entirely new container ecosystem, demonstrated the importance of strong security practices, and have been a catal...

Read more
  • DevOps
  • Security
Avatar
Adam Hawkins
— March 21, 2019

How DevOps Increases System Security

The perception of DevOps and its role in the IT industry has changed over the last five years due to research, adoption, and experimentation. Accelerate: The Science of Lean Software and DevOps by Gene Kim, Jez Humble, and Nicole Forsgren makes data-backed predictions about how DevOps p...

Read more
  • DevOps
  • Security
Avatar
Adam Hawkins
— February 7, 2019

Measuring DevOps Success: What, Where, and How

The DevOps methodology relates technical and organization practices so it's difficult to simply ascribe a number and say "our organization is a B+ on DevOps!" Things don't work that way. A better approach identifies intended outcomes and measurable characteristics for each outcome. Let'...

Read more
  • DevOps
Avatar
Adam Hawkins
— February 5, 2019

2019 DevOps and Automation Predictions

2019 DevOps and Automation Predictions We recently released our 2019 predictions for cloud computing and are doing the same here for DevOps and automation predictions. 2018 was a great year for software, and DevOps falls somewhere on the slope of enlightenment on the Gartner Hype Cy...

Read more
  • Cloud Predictions
  • DevOps
Avatar
Adam Hawkins
— January 17, 2019

Testing Through the Deployment Pipeline

Automated deployment pipelines empower teams to ship better software faster. The best pipelines do more than deploy software; they also ensure the entire system is regression-free. Our deployment pipelines must keep up with the shifting realities in software architecture. Applications a...

Read more
  • DevOps
Avatar
Adam Hawkins
— December 12, 2018

Getting Started With Site Reliability Engineering

Much has been written and discussed about SRE (Site Reliability Engineering) from what it is, how to do it, and how it's the same (or different) as DevOps. Google coined the term, defined the profession, and wrote the book on it. Their "Site Reliability Engineering" book covers the idea...

Read more
  • DevOps
  • SRE