Build Agents and Pools
Configuring API Access
The course is part of this learning path
Azure DevOps has many great tools for implementing and managing your build infrastructure, and this course walks you through how to use them. With a mix of theory and real-life demonstrations from the Azure portal, you will learn how to create Azure pipelines, use them to integrate 3rd party build systems, utilize agent pools, and learn how to put it all together to set up an automated workflow that can potentially save you hundreds of man-hours. So, whether you’re here to learn more about DevOps, improve your development process, or learn more about Azure DevOps in particular, this course will help get you further along your path.
For any feedback, queries, or suggestions relating to this course, please contact us at email@example.com.
- Learn how to build an Azure DevOps pipeline and add automation tasks
- Use the visual designer for adding build tasks to a pipeline
- Create a pipeline that closely matches an existing build process
- Determine a pipeline strategy based on the information given to you during a discovery process
This course is intended for:
- IT professionals looking to become certified DevOps professionals
- IT professionals experienced with other cloud providers who want to gain a better understanding of Azure
- Anyone who wants a better understanding of Azure build infrastructure processes
To get the most from this course, you should have some understanding of the software development process, at least at a high level, as well as an understanding of what DevOps is and the terminology related to it.
In this demo, I'm gonna show you how to create organization and project level agent pools, and how to install and register build agents into our organization pool and how to reference organization pools from our projects.
Some things to know when using self-hosted windows agents is that it must be Windows 7 or higher, if you are using a client operating system, and if you are using Windows Server, then it must be Windows Server 2008 R2 SP1 or later, and you must have PowerShell 3.0 installed. It's also recommended that you have installed Visual Studio 2015 or later, and the .net framework 4.6.2 or later.
One of the main reasons for using a self-hosted agent is machine hardware. For example, the code for Azure DevOps itself uses a pipeline with a 24-core server and four self-hosted agents on a single machine.
To prepare for this demo, I have created an Azure virtual machine with Windows Server 2019 and Visual Studio Enterprise 2019. This is a Standard_D4_v3 image, it has four CPUs, and 16 gigs of RAM.
When creating agent pools, there are two levels of scope for an agent pool: One is at the organization level, and one is at that project level. When you create an agent pool at the project level, a matching agent pool is created at the organization level, so you'll have two identical pools, one at the project level and one at the organization level. This is a one-to-one mapping.
The first thing we're gonna do is sign into our organization page. Then, we'll click the organization settings in the bottom left-hand corner. This will open our settings menu. Then, we will click on the agent pool menu option. This will open our agent pools page. This already has a pool named Azure Pipelines and one named Default. The Azure Pipelines is for Microsoft Hosted agents, and the Default is for self-hosted agents. If you do not mind the name of Default or Azure Pipelines, then you don't need to change anything here. You can simply add Microsoft hosted agents or self-hosted agents to the various pools. We are going to add a new pool here and show how to reference the organizational level pool from the project level pools. There are two configuration options when creating an organizational level pool. The first, grant access permissions to all pipelines, if checked, will give all existing pipelines access to the newly created agent pool. The second is Auto-provision this agent pool in all projects. This option, if checked, will automatically add a matching project level pool to any new projects we create. I'm gonna leave these unchecked for now.
Next, we will create our project-level agent pool. We bring up our project, click on the project settings. Then we'll click on the agent pool's option, and that will bring up our agent pool's page. They're matching Azure Pipelines and Default pools on this page. This is at the project level. We'll go ahead and click the Add button, and now we have the option for selecting a new or existing agent pool. Selecting the existing agent pool will give us a list of the organizational agent pools to choose from. We will select the existing option and select our Org1 Agent Pool. I'm gonna uncheck the Grant access to all pipelines, and click Create. This will create a project-level agent pool, that has a direct reference to our Organizational agent pool of the same name.
The next thing we need to do is to create a Personal Access Token, or PAT. This is how we are gonna authenticate the agent machines with our DevOps organization, and project. We will click on the person icon here in the top right, and bring up the profile menu. Then, we will select Personal Access Tokens. This will bring us to the page where we can add an additional token. Go ahead and click the New Token Button. And we'll give our token a descriptive name. I'm gonna use Build Agent. Then, we need to set the permissions for the token here. We have the option of giving full access to the token or we can give very specific permissions. We can also set the expiration length here. It has the option to never expire. But I'm just gonna leave it for the default expiration of 30 days, and we only need to give Build, Read and Execute, and Agent Pools, Read and Manage, permissions to this token. To find Agent Pools, we're gonna need to click the Show All Scopes. Once we have set everything set, click create, this will create a new Personal Access Token, and give us the option to copy the token. It's very important to remember that this is the only time Azure will show you this token value. There is no way to retrieve the token value once you leave this page. You can edit the permissions later and you can regenerate the token, but you can never get this particular value back. So it's a good idea to copy it and store it with your other secure passwords.
Now, we need to login to our build agent machine and open our browser to our organization DevOps page. After we sign into our organization, we click the organization settings. Then, we'll click on the agent pools options and then we'll click the new Org1 Agent Pool that we created and click on the Agents tab at the top. This will bring us to our Agents page with a button for adding a new agent. We will go ahead and click this button, and a dialog will pop-up with instructions on how to install an agent. It has the option to download the zip file for the agent software, or you can copy and paste a script, we are gonna download this file and run the installer. If you are comfortable with PowerShell and fully understand the script that is being run, then feel free to use that option.
Once we've downloaded the file, we will unzip it to its own directory. Once this is done, I am going to create a second build agent directory and copy all of the files to the new directory, so that I can create a second build agent on this machine. In the address bar of the Windows Explorer, we can just type CMD and that will open a Windows Command Prompt in our current directory. We will run the config.cmd file and this will ask us a series of questions about our organization and agent pools and authenticate our build machine using the Personal Access Token we created earlier. This will configure our agent, and send a list of capabilities to our organization. The first thing it needs to know is the access URL to our organization. I'm just gonna copy mine from the browser address bar. Then, it's gonna ask us how we want to authenticate with our organization. We can choose PAT, Personal Access Token, by default by typing PAT or hitting enter. You can also use a windows username in the format of domain slash username, or firstname.lastname@example.org. If you're using this format, it will then ask you for a password for your account. We, however, will be using a Personal Access Token. Then, I will just paste my Personal Access Token here. It then authorizes the build agent machine with our organization and connects. Now, it'll ask us which agent pool we'd like to use. You can just hit enter if you want to use the Default, a self-hosted agents pool, but in our case, we want to use the one we created, so I'll just copy the name from our organization agent pool page.
Now, we enter our build agent name. Hitting enter will use the machine name of the computer. I'm gonna use Build Agent A. It then scans for all the relevant software installed on the build agent machine and adds the machine to our organizational agent pool. It's then going to ask us for a name for our working directory. This is the directory that all our tasks and working files are saved to when building our project and using our build agent. I'm gonna hit enter and use the default entry. It's now asking us if we want to run the agent as a service or interactively. Running interactively requires additional configuration and can involve domain policies, which are not within the scope of this course. We are gonna type Y for yes, and then it's gonna ask us if we want to use the NT AUTHORITY\NETWORK SERVICE as the user account for the service. We're gonna hit enter and accept the default. It will then finish setting up our build agent and let us know if it was successful or not.
Some of the reasons that it may fail are failure to authorize our PAT, no network connectivity to the server, or not having write permissions on the directory the build agent is installed in. We have now installed our first build agent, and I'm going to quickly install a second build agent. It's an identical process, it lends itself well to scripted and unattended agent installations from multiple agents.
Now that we have our agent pools configured and our build agents installed, we can now update our pipeline to use our pool of agents to build our project. I will navigate to our project pipeline and click edit. This will bring up the YAML file. Now, I've already configured a pipeline to use the new project agent pool, which delegates to our matching organization pool. All that I've changed here was to replace the vmImage attribute with the Name attribute under the pool heading in the YAML file. I've also set the name attribute to our new project agent pool Org1 Agent Pool. And I'll just click run. But wait, what happened? Remember when I unchecked the option to grant access to all pipelines? Unchecking this box means that we must give permissions to the pipeline to access our agent pool. So we can just click the Authorize Resources button to the right of this message.
Now if we happen to forget to create a project-level agent pool that maps to our organizational agent pool, then when we run this pipeline, we would get an error that says it cannot find the pipeline named Org1 Agent Pool. So, there must be an agent pool at the project level that references an organizational level agent pool that contains build agents in order to be able to use the agents to build our project.
Now, let's try and run our pipeline a second time and see if it uses both of our build agents. This first one here, it's using Build Agent B. Let's check the other running job and see which build agent it's using. And as you can see here, Build Agent A is being used. So, both the agents in our agent pool are being used.
So, in this demo, we created an agent pool at the organization and project level and mapped the project agent pool to the organization agent pool. We installed and configured two separate build agents on a virtual machine that has Windows Server and Visual Studio installed, and we've configured our pipeline to grab an available agent from the build agent pool to build our project. Before Azure pipelines came along, this process involved knowing and running complex PowerShell scripts that could be hard to understand and fix when things did not go quite as planned or unique environments came into play. But as you can see in this demo, Azure DevOps has really made an effort to simplify this process as much as they can.
I hope you enjoyed this demo, and I look forward to seeing you in our next module!
As well being the owner and CTO of Sharp Solutions Group, a software development and IT staffing company based in the Philippines, Kelso is a Microsoft Certified professional and an avid knowledge seeker. His belief is that you need to learn something new each day to stay on top of the constantly changing IT world. He is an avid gamer (both video games and board games) and lives in the Philippines with his wife and soon-to-be-delivered son.