At Capital One, DevOps is about delivering high quality, working software, faster. This means software that is reliable, secure, usable, and performant while providing value and accomplishing those important end user goals. Everything is about speed of delivery and getting that feedback. Mark Andersen is the Director of DevOps for Capital One’s Auto Finance group and the Cloud Account Lead for Capital One Auto Lending in Plano, Texas. He recently joined Cloud Academy for the webinar, Recipe for DevOps Success, to talk about some of the lessons learned from their DevOps transformation. In this post, we’ll be sharing Mark’s 10 essential ingredients for making your own move to DevOps a success.
Capital One Before DevOps
Prior to DevOps, Capital One was on-prem. Releases were inconsistent and even the smallest change required many handoffs between many teams. Manual steps were documented in spreadsheets. Setting up a new environment could take several months and, even then, there were lots of inconsistencies.
Capital One initiated their DevOps transformation over several phases. They started out with some configuration management automation tools like Chef, which got them part of the way there.
In phase two, Capital One went all-in on cloud, looking at full infrastructure automation and growing into the delivery pipelines. They started with two legacy applications and assembled a cross-functional SWAT team of developers, infrastructure, production, and support personnel who worked closely together for two months. A few professional services resources were added to address the knowledge gaps. This team focused on automating the build deployment of the infrastructure and then the application deployment. Here, they used tools like Jenkins, AWS CloudFormation, and Chef in the process.
They moved on to address applications of increasing complexity and brought on more applications over time. Eventually, the application teams were able to drive themselves, and they disbanded the SWAT team. Today, Capital One is continuing their DevOps journey by implementing robust pipelines with increased quality checks and automation.
Here is what they learned along the way.
10 ingredients for DevOps Transformation
Empower your developers.
The goal of DevOps is to enable developers to be self-sufficient so they can move quickly. It’s not enough that your DevOps engineers know the automation. They need to teach the agile pod and the developers how things work and how things get fixed. Developers who know more will be able to resolve issues on their own. They won’t have to wait for help or create a ticket, etc.
Mark says, “I like the analogy of treating automation like a car. You can teach the application team how to drive and how to change the oil and do some of the general maintenance, but they don’t need to know how to build the engine.” Is the app team going to understand every little nuance around cloud formation and deep networking? That’s probably a little too much for them. However, it is useful for them to understand the pipeline in how things get deployed. This way, when things break, they know where to look and understand how everything works together.
Be prepared to get creative.
Automating on-premises and legacy software will be challenging, and you’re going to have to employ some creative solutions along the way. Capital One has a lot of the same infrastructure that other Fortune 500 companies have, and it’s not easily automated. On-premises software usually isn’t built for the dynamic nature of how often you deploy or replace things, and it isn’t necessarily built for the cloud. You will have to make some choices in terms of how far you can get in automating and deploying some of these applications, and it won’t be perfect. That’s OK.
Don’t try to do everything at once.
Start out by focusing on just one or two things. You’ll experience many ups and downs on the change curve. By breaking things down into smaller batches, you can enjoy more incremental successes and keep your team motivated along the way. And, when you celebrate, do it like Dude Perfect.
Make sure the tools you select or build have APIs.
APIs are the enabler for the DevOps and automation journey and also for the cloud. Tools without an API will be a major barrier to automating everything and delivering faster. Even a bad API is better than no API. If you are API-first, your applications and your users can orchestrate these tools together. You can always build the UI after that.
Long feedback loops are bad feedback loops. If you automate everything the first time, it’s going to take a really long time to get that feedback. You’re going to have to look at parallelizing everything and especially your functional tests. Functional tests that take hours do not make for a good feedback loop.
Don’t forget to unit test.
Don’t just focus on functional tests, which tend to be slower and more brittle. With automation and faster delivery times, there is a trend of building more functional tests at the expense of unit tests. Unit tests are fast and stable. They give your developers feedback, and you should be able to run them on your laptop or anywhere.
It’s not that functional tests are bad—just don’t favor them over unit tests. You don’t want to have the aggregate of all your dependencies affect your build and your feedback loops. Consider mocking out some of those, especially the ones that are less stable. This speeds up the process of getting feedback in your tests.
Translate what you’re doing into developer language.
Networking and security are difficult, and it’s even more so for developers. If you get into doing DevOps in the cloud, you’re going to get into networking. Your developers and agile pods will need training.
Take centralizing your log collection for example. Let everyone see the information and create things in the developer language so they have the information they need. Developers don’t usually understand CIDR blocks or complex networking, but they can identify, for example, that a given IP address is really the LDAP service versus the business service the app is tied to, Make information available in a language that developers can understand.
Don’t create a new silo.
Don’t replace your infrastructure silos with a DevOps silo. If you’ve siloed your DevOps team to just working with tickets, you’re not getting the actual flow of delivery from the agile pods if they’re waiting on tickets.
Focus on removing and reducing the handoffs.
How can you create self-service for the things that you can’t give developers direct access to? If you can’t give them access to create LDAP groups or security groups, give them a tool to do it the right way or solve the 80% problem. Putting everything behind a manual process will really slow developers down. Give them the tools and the access they need.
Don’t let great get in the way of being very good.
Deliver something, get feedback, and do more. If you’re waiting for something to be absolutely perfect, you’ll never deliver. Unicorns don’t exist (if they do, they’re really tough to find). Strive to be a horse. Accept that you will run into lots of challenges along the way. Work on getting better at first. Don’t let perfection get in the way of that.
Learn more about Capital One’s DevOps transformation in the on-demand Cloud Academy webinar, Recipe for DevOps Success.
To get started on DevOps, you can find a full collection of DevOps content in Cloud Academy Library.
When To Use A Multi-Cloud Strategy
There are both advantages and disadvantages of multi-cloud environments, and knowing when to use a multi-cloud strategy could depend on how minimizing your dependency on a single provider could affect costs, performance, and security. Before discussing these advantages and disadvantages...
What are the Best Practices for a Large-Scale Cloud Migration?
For organizations of any size, migrating to the cloud presents a challenge. Leaders have many decisions to make, including which applications to migrate and how to connect such activities to business use cases. Large-scale migrations are the specialty of David Turretini, Worldwide Lead...
New on Cloud Academy, March ’18: Machine Learning on AWS and Azure, Docker in Depth, and more
Introduction to Machine Learning on AWS This is your quick-start guide for building and deploying with Amazon Machine Learning. By the end of this learning path, you will be able to apply supervised and unsupervised learning, ML algorithms, deep learning, and deep neural networks on AW...
What are the Benefits of Migrating to the Cloud?
If you’re like most businesses, you already have at least one workload running in the cloud. However, that doesn’t mean that cloud migration is right for everyone. While cloud environments are generally scalable, reliable, and highly available, those won’t be the only considerations dri...
4 Trends That Will Change How Companies Invest in Cloud in 2018
The cloud is forever changing how we look at IT. Over the past years, we’ve had a front seat view of how the cloud has evolved and how large companies and industries are changing practices internally toward a response that looks more and more like the innovation leaders have read about...
7 Elements For a Successful Cloud Migration Plan
When it comes to cloud, the good news is that we’re past the period of fear, uncertainty, and doubt. Everyone now agrees that the cloud is a key part of any firm’s IT investment. The not-so-good news is that there is still confusion about what to move, how to move it, and the best pract...
Moving Your Business to the Cloud. 5 Reasons Why.
Whether the destination is a private, public, or hybrid cloud, it’s clear that companies are continuing to migrate their workloads and applications to the cloud. The latest RightScale report shows that 95% of the respondents for its 2017 State of the Cloud survey are currently using ...
Migrating the Smart Way, with Azure
The cloud era is gaining greater speed, and market projections look very healthy for years to come. This growth and the enthusiasm it’s creating for cloud technology is pushing organizations who, until very recently, were resistant to start their own cloud initiatives. This post is abou...
7 Things Every CEO Needs to Know Before Moving to AWS
For companies, the question of moving to the cloud is not just a technology decision, it’s a business decision. While IT will have to manage the choice of services, setup, recovery, and many more details, the CEO is tasked with thinking about the impact of the cloud on the overall organ...
5 Good Reasons to Migrate on Azure Public Cloud
Azure public cloud: For companies interested in moving to the cloud, Microsoft Azure is one of the most complete public cloud platforms available. With so many providers and features to choose from in this growing, competitive market, there are many good reasons to choose the Azure publ...
Cloud Migration: Getting it Right the First Time
As the transition from local hosting to cloud-based applications continues, getting cloud migration right is hugely important. While I've never migrated any large infrastructure projects to AWS, I have moved a few small WordPress sites. Those experiences were enough to teach me about th...