2019 DevOps and Automation Predictions
We recently released our 2019 predictions for cloud computing and are doing the same here for DevOps and automation predictions.
2018 was a great year for software, and DevOps falls somewhere on the slope of enlightenment on the Gartner Hype Cycle, gaining traction in the industry and continuing to demonstrate itself as the best methodology to build, deploy, and operate software. To date, one of the most important books published on DevOps, which I’ve cited numerous times is Accelerate: The Science of Lean Software and DevOps. This publication asserts through data that DevOps practices create high performers. Together with the DevOps Handbook, both publications provide a window into what to expect in 2019. In a nutshell, there’s no surprise that the industry will continue to adopt DevOps and automate more of their procedures to remove toil and increase efficiency, but there are also new opportunities to see DevOps applied in surprising new contexts. Before we dive into the predictions, if you’re looking to start your DevOps training, have a look at all of the content offered by Cloud Academy in its DevOps training library. I also advise you to check out the DevOps Playbook – CI/CD Tools and Services Learning Path.
Prediction #1: Kubernetes will grow substantially in 2019
We’ll start with a safe prediction. The IT industry has converged on Kubernetes as the leading technology for container orchestration. Kubernetes has gained widespread adoption, and is now more accessible with cloud providers such as AWS, Google, and Microsoft offering Kubernetes-as-a-Service products. This means that the industry has standardized on an open source stack. However, it’s more accurate to say the stack is a Cloud Native Computing Foundation (CNCF) project. CNCF provides stewardship, fostering the continued growth and evolution of Kubernetes, together with many other cloud-related projects. This is not a bad thing but it’s curious to wonder how many projects will be under the CNCF umbrella by the end of 2019.
Kubernetes will grow substantially in 2019 with the industry firmly behind it and will grow more powerful features, but the wider ecosystem will be the bigger beneficiary. AWS, Google, and Microsoft now have a vested interested in improving the Kubernetes experience on their platforms. AWS’s IAM and VPC integrations come to mind as important integration points. At the other end of the spectrum, the industry may focus on improving downstream uses instead of pushing for adoption. So in other words, it will get easier and more convenient to deploy applications to Kubernetes.
Prediction #2: Kubernetes API will Improve Experience for 3rd Parties
Now the entire supporting cast like service meshes, databases, metrics, and logging tools can improve platforms integrations. More applications will be deployed as containers in 2019, so improving the full stack developer UX is important for open source and paid software vendors. Companies such as DataDog, a telemetry SaaS, will tighten their integration with the Kubernetes platform. Kubernetes is in a great position, backed by the supporting projects in CNCF, to define new APIs to automation production operations.
Specifically, I’m interested to see how Kubernetes exposes API’s for telemetry (and even alerts), so that teams and individual developers don’t have to integrate with specific application stacks, but instead use Kubernetes APIs. I’m generally supportive of this approach since every production application requires telemetry and it should be possible to standardize a harness to collect and report data. Plus once some interface is created, it’s easier for subsequent iterations and improvements. Any progress in this area will reduce toil across the industry and promote better production practices. On the other hand, it pushes more systems to use infrastructure-as-code.
Prediction #3: Infrastructure-as-Code for Alerts & Dashboard (Finally!)
I expect more areas to adopt infrastructure-as-code and for developers to come to expect automation support in more areas as well. Consider the various hosted CI systems like CircleCI, SemaphoreCI, and TravisCI. This isn’t an exhaustive list, but the sample illustrates the point. Both CircleCI and SemaphoreCI released a big new version of their products in 2018, with configuration-as-code being the key selling point. Developers have come to expect their pipeline and build steps committed to code and not managed in a GUI. While I don’t think that’s surprising at all, I do expect more products and services to follow suit, including telemetry systems. It’s also common to configure dashboard, charts, and alerts all through a GUI, an activity I myself have gone through too many times and I don’t see it as valuable time spent. That’s a definite opportunity to remove toil and increase repeatability, especially when reused across similar applications and when new technologies exist.
Hopefully, developers across the industry can take a step back and examine the larger context their applications operate in and automate large swaths of that problem space. It’s possible that we may see scaffolding technology that can spin up a fully functional deployment pipeline (based on the assumption that the application is containerized and deployed to Kubernetes) with appropriate test steps, stage promotions, and even security checks. I expect tools such as Spinnaker will get increasingly powerful by focusing on container and info-sec driven integrations.
Prediction #4: Automated GDPR Compliance and Verification
Info-sec is also the gateway to enterprise applications dealing with existing compliance rules and government regulations. For better or worse, the EU companies are on the compliance forefront due to the EU GDPR. GDPR establishes regulations for how companies must manage and dispose of customer data and came into effect this year but will see more tests in the coming year. The United States does not have a similar regulation and is generally not fond of this kind of regulation, but public opinion may be shifting due to constant data leaks, manipulation of the 2016 USA elections, and general mistrust in large tech companies. I don’t think that the implementation of a law will happen in 2019, but it’s likely in the future. That kind of law would drastically change IT’s requirements for data management across the deployment pipeline. GDPR compliant organizations will look to a new crop of automation and compliance testing tools to stay up to date and while companies may not be altruistic about their compliance, consumers are the end benefactors here and that translates into revenue. Automating compliance and handling ensures that more companies become compliant and stay compliant. Unfortunately, the industry will likely see bombshell cases in 2019 which will reinforce the need for compliance.
Machine learning and algorithms will likely be used alongside other systems to increase efficiency. Detecting GDPR violations is one such application area and more applications with machine learning means more data engineering since those algorithms need training data sets. The data sets and their test environments require deployment pipelines just like any other piece of software. More data application deployment pipelines also means exposing DevOps principles to more engineers in different technical areas. We may see a platform-as-a-service option for data pipelines and it may be containerized or serverless. Regardless, the industry will definitely see more focus on applying DevOps automation principles to their data based teams.
Prediction #5: Python and Golang Eat The world
There’s already writing on the wall for some of this. Cloud Academy’s upcoming courses include 3 courses on data science and machine learning. There are another 6 plus courses on different aspects of the deployment pipeline, which include automation and optimization, Kubernetes based automation, and managing container security. The first course covers advanced automation uses cases, exemplifying how DevOps is used to remove toil from the process. The course on Kubernetes based automation prepares engineers to the upcoming Kubernetes-first world. Cloud Academy also has a dedicated learning path to GDPR Compliance on AWS, with skills and expectations heading in that direction.
Engineers, as always, will have to keep their skills up to date as both technologies and the industry change. Kubernetes’ increasing prevalence will expose Golang to more engineering teams. They’ll likely use Go when integrating with Kubernetes since Kubernetes (and many supporting projects in the ecosystem) are written in Go. Don’t be surprised if you end up writing Go in 2019. Python will continue in its popularity and usage through data science and machine learning, so it may be time to bust out the old Python books again.
Also, don’t be surprised if you end up encountering a serverless application in 2019. Personally, serverless (or Functions-as-a-Service) is the most exciting aspect of our industry, offering a new architecture and execution model. It also provides some back pressure and competition with containers for infrastructure and developer UX. Finally, it’s easier to deploy a serverless application as compared to a containerized one. Container orchestration systems have a lot to learn from serverless applications in this regard. Hopefully, improvements continue in this area due to Kubernetes standardization.
2019 will be a great year of DevOps-minded teams. The technology is improving and the war for ideas is over and now the industry can onboard more teams to the DevOps way. DevOps is far from being the secret sauce used only by the elite. As Gene Kim, author and contributor to both Accelerate and The DevOps Handbook puts it:
It also indicates that a high rate of DevOps performance is more easily attainable than it has been in the past and is no longer something reserved for an exclusive set of teams.
In short, 2019 looks great for those already in high-performance teams and those who want to become one!