Automation build tools are extremely powerful but can be complex. Putting forth the effort in learning to use Azure DevOps and developing the ability to design and implement effective and efficient pipelines to automate tasks will pay dividends.

Let's do a brief review of what was covered and pull the individual pieces together.

A good place to start is with build tools. There are many pipeline tools available today. Each has associated characteristics that may or may not be desirable. Some can be run on-premises. Others may have professional support. Whatever criteria must be met, due diligence will need to be done to find one that fits your use case the best.

Azure DevOps is a robust, end-to-end planning, testing, and pipeline tool. Azure DevOps Pipelines subscribes to the configuration as code convention by using an Azure Pipelines YAML file to declare the build criteria and implementation.

Let's take a look at an Azure Pipelines YAML file that has been prepared.

At the top of the file, we can find the trigger declaration. The pipeline is configured to trigger on the master and release branches. Specifying branches to include overrides the default of including all branches.

Triggers allow builds to be automatically started or excluded from starting based on matching criteria.

The branches syntax is not required if simply including branches to trigger on, however, it is required when options are desired such as exclude for branches and the use of paths and tags. 

Wildcards can be used for pattern matching. The asterisk matches zero or more characters and the question mark, which is not used in this example, matches a single character.

By using include, implicitly all other branches not listed or matched are excluded. The same goes for exclude when used. All other branches not listed are implicitly included.

Paths allow file paths to be included or excluded. Any specified path also encompasses children of the specified directory, if specifying a directory. The default is to include root but this declaration has excluded development. By reaching deeper into the file structure, the directory main under development can be included. Everything else under development is excluded. 

Tags can also be specified either under branches using git syntax or under the tags object. All tags are included here, which is converse to the default.

Keep in mind that branch conditions must be a match for the build to run and if tags or paths are specified, one of those must have criteria that matches for the build to run.

Pull request triggers allow branches and paths to be specified. This only applies to builds when pull requests are involved. Branches that are the target of the pull request, meaning the branch being merged into, can be specified to include or exclude. Although wildcards are not used here, they are very much applicable.

Paths are specified to include or exclude as well. The pull request paths are not required to be the same as the ones specified under normal trigger paths. It just happens to be the case in this example. The default is to include all pull request branches and the root path. Tags are not allowed in the pull request trigger specification.

Further down the file are the declared steps to be run. These steps can be logically grouped under jobs. Those jobs can be further grouped into stages. By default, every pipeline has one stage and one job. Although not the case in this example, if only one stage and job are needed then the syntax to explicitly define those can be omitted.

Here, there are two stages: validate_and_test and build_and_archive. Since there are multiple jobs, this would run in parallel if more than one agent is available and the dependsOn syntax wasn't used to force the builds_and_archive job to wait for the validate_and_test job to successfully complete.

The dependsOn syntax can be used for jobs or stages. It is useful for not only establishing dependencies but to also optimize the build. For example, there may be a case where a job could be run in parallel but the overhead of doing so is a detriment to the build. In that case, forcing the build to be sequential would conserve resources.

Chain of result is also a consideration. Build steps typically use the results from prior steps to operate upon. Using more than one build agent means that jobs would either need to be completely independent or some other mechanism must be used to provide context from one job to another.

For example, the initial job for a pipeline tests and builds code, the next step may be to run integration tests on the built code. That could be done within the same job or it could be done in a separate job. If done in a separate job then the built application artifact would need to be made available to the downstream jobs. This could be easily done by storing the artifact at the end of the initial build job and then retrieving the artifact for each successive build.

In fact, often, not only does the code artifact need to be stored, configuration details and metadata need to be promoted. Artifact storage is certainly applicable to these details also. This data can be written to one or more files and then stored as one or more artifacts for later consumption.

These principles also apply to hybrid builds since hybrid builds are essentially build jobs performed across multiple platforms.

Hybrid builds can be started on one platform and then finished on another platform, such as using Jenkins to build the artifact and then using Azure DevOps to thoroughly test the artifact. 

The build process needs to be broken into distinct jobs, while keeping in mind the criteria that needs to be met that prompted the utilization of hybrid builds.

There are several reasons for hybrid builds. This may be due to security requirements, proprietary software, resource allocation, tooling availability, compliance requirements, optimization strategy, or any multitude of other reasons.

One of the most important design items for hybrid builds is how the builds will be triggered. Most commonly this is done with hooks, Git triggers, or API calls.

Careful consideration must be given to what criteria defines a trigger, when it should trigger, how it should trigger, and what actions, if any, need to be performed upon failure of the build at critical points. This will ensure that the build process runs smoothly between platforms.

By understanding the advantages and limitations of build tools and applying the principles and techniques covered with build triggers, parallel builds, and hybrid builds, you will be well on your way to providing a foundation for a secure, repeatable, auditable, and complete continuous integration solutions for your projects.

Lectures

• Introduction
• Build Tools
• Build Triggers
• Parallel Builds
• Hybrid Builds