- [Instructor] Welcome back! In this lecture, we'll provide you with an understanding of how to build, scale-out, and setup a multi node or master agent Jenkins infrastructure environment. We'll discuss the various motives as to why you would consider deploying Jenkins in a multi server setup. Okay, let's begin. For starters, Jenkins can quite easily be installed and operated on a single server setup, assuming that the server in question is adequately spec'd in terms of both RAM and raw disk space. However, as your CICD requirements start to grow, potentially in terms of build job numbers, frequency of builds, complexity of builds, software frameworks and/or operating system build requirements, you will likely find that you will end up running Jenkins in a multi-node configuration. 

In a multi-node configuration, a Jenkins master can be configured to distribute build jobs amongst multiple Jenkins build agents. Motivating reason to scale-out and introduce multiple build agents vary per environment. Some of the more common reasons may include Jenkins master server security. This requirement is needed when you don't want to compromise the security of a master server. Building on the master server can be dangerous if unvetted build jobs perform unintentional or intentional manipulation of the master server setup. 

Build job security. This requirement is sometimes needed when a build job requires some form of access to sensitive material that you want to protect. Building on a dedicated access-controlled build server can help isolate and maintain security over the sensitive material. Tooling and required software frameworks. For example, you maybe building an iPhone application that has a compilation dependency on Apple's Xcode. Therefore, this build job would be farmed out to a Jenkins agent installed on a macOS server. Or, alternatively, you maybe building a Microsoft .NET Windows application that must be built on a Windows server. Hence, this build job would be farmed out to a Windows-based build agent. A floating master server utilization. Building on the master server can impact overall use if server utilization peaks and performance bottlenecks. Offloading builds onto dedicated agents can ensure that the master server has full utilization for performing administration only tasks. Faster build job times. By having dedicated build agents, build jobs can be optimized to complete as quickly as possible. The count of build agents and/or the build agent hardware specs can be increased and scaled out when more build performance is required. 

And finally, geography of builds. You may have a requirement to perform build jobs in different geographic locations, maybe in different office locations or different time zones. In this case, build agents can be created and installed in the required location, with build jobs distributed amongst them. Communication between the master and each of its build agents is performed using TCP/IP-based network connections. The connectivity can be initiated in either direction, but is typically done as best practice from the master. Initiating connections from the master to the build agents has several advantages over the opposing direction, mainly the master maintains scheduling and dispatching control over where and when build jobs take place. A typical setup involves installing an SSH service on the build agent side. The master then initiates an SSH connection to the build agent using a pre-generated public/private keypair for authentication. Having initiated the SSH connection, the master will transfer a JAR file, namely the remoting.jar file, to the build agent server. The remoting.jar file contains a compacted version of Jenkins, containing just the required components to perform a distributed build. 

Let's walk through the Jenkins administration requirements to setup a distributed build agent. In this scenario, we'll create a Linux-based Jenkins agent running Ubuntu 18.04. Before configuring the setup within the Jenkins administration console, we would run the following bash script on the Ubuntu server. This script performs the following actions in order. One, updates the available package lists. Two, installs the OpenSSH package to provide an SSH daemon which the Jenkins master server SSH client will connect to. Three, installs a JRE which is required to run the agent JAR remoting file, which is pushed down from the master server when an SSH connection is successfully established. Four, creates a new user named jenkins and sets the home directory for the new user as /var/lib/jenkins. Inside this directory a .ssh directory is created. Five, ssh-keygen is then used to generate the SSH public private keypair used to establish SSH authentication between the master and agent server. Six, the public key is then stored in the authorized_keys file. Seven, and finally, we reset the folder permissions recursively on the /var/lib/jenkins directory, giving exclusive access to the jenkins user. This completes the agent side of the server setup. 

Next, we return to Jenkins administration console and set a new SSH username with private key credential, copying in the private key portion copied over from the agent server. We set the username to jenkins. We now have all the credential management in place and we are ready to configure a new build agent node. To do so, we navigate to the Manage Jenkins section and click on the Manage Nodes option. You are then prompted to supply a node name and whether to make it a permanent agent or not. Permanency dictates whether the agent is going to stick around or not. A non-permanent agent may, for example, be spawn as part of an elastic cloud hosted architecture involving auto-scaling, for example. For most requirements, we can simply go with a permanent agent. 

Next, we need to set the specifics regarding the new node. In particular, we will specify the number of executors. In this case, we'll raise the number of executors to four. The remote root directory. Here, we set it to be the home directory we allocated to the jenkins user, namely /var/lib/jenkins. A label is specified, which we can use later within our build job and/or build pipeline configuration to dictate where that particular build job or pipeline execution will take place. Launch method. This one is important and here, we've chosen to select the Launch agents via SSH option. With this option selected, we then need to specify the following; the agent host address, the credentials that we previously configured within Jenkins, and we need to pick an appropriate strategy for the host key verification. In this case, we're going with the known hosts file strategy. 

This now completes the configuration phase for both the master and build agent nodes. With all the configuration in place, all that is left is to do is activate and launch the agent by clicking the Launch agent button. The Jenkins master will then attempt to form an SSH connection to the new Jenkins agent, presenting and authenticating with the SSH private key that it has been configured with. A successful connection concludes with the message, Agent successfully connected and online. With the new agent online and connected, we are now presented with a view of the stats associated with the agent, including the number of build executors available. In this case, we can see that the new build agent supports four build executors as expected. When troubleshooting connecting and launching new build agents, review the launch console log carefully. Typical problems often encountered when launching agents are inbound SSH connections on port 22 are blocked on the agent side, the SSH daemon has not been started on the agent side, and/or SSH key mismanagement. 

Okay, that completes this lecture on how to build, scale-out, and setup a multi-node or master agent Jenkins infrastructure environment. Go ahead and close this lecture and we'll see you shortly in the next one.