Here we look at configuring security for HDInsight with Enterprise Security Package. This means enabling users to authenticate with their domain credentials. 

Setting this up involves a few steps. Firstly, we need to enable Azure Active Directory Domain Services. Then we must create and authorize a managed identity. Once these preliminary steps have been completed, we can create an HDInsight cluster with Enterprise Security Package. Just note that ESP is only available with Hadoop, Spark, and Interactive Query cluster types.

To enable Azure Active Directory Domain Services, create a new Azure AD Domain Services resource. Assign your resource group and your location. Next, if you do not have a virtual network, you will need to create one. You need to create a dedicated subnet before you deploy Azure Active Directory Domain Services. You will need to assign at least one user to a special administrative group called AAD DC Administrators, which is used for managing the Azure Active Directory Domain Services domain. Members of this group are granted administrative permissions on virtual machines that are joined to the managed domain. Next, we turn on synchronization and click OK in the summary.

Next, we need to create an authorized assigned managed identity by creating a new User assigned managed identity resource. A user assigned managed identity simplifies securing domain services operations. After creating the managed identity, we need to assign the HDInsight Domain Services contributor role to it. We do this by going into our Active Directory Domain Services resource. Select Access Control, and then select User managed identity from the find drop-down, and select your managed identity. Click Add role assignment and select the HDInsight Domain Services Contributor role. Next, we want to add users to the Managed Identity Operator role of our user assigned managed identity.

Now we create the HDInsight cluster. Give it a name and select the region. I'm going to use the Hadoop cluster type, as this supports Enterprise Security Package. You'll need to supply a cluster login username and password. I'll just stick with the default storage settings. If you chose a cluster type that did not support ESP, the Enterprise Security Package checkbox would be missing from the Security + Networking page.