In an earlier lesson, you learned how to make external connections from Azure, especially using ExpressRoute or a VPN. Those solutions are great, but if you only need external connectivity for a single application, then they’re probably overkill, because you need to modify your network to use them. Fortunately, there are much simpler, more focused solutions that you can use instead.

Azure provides many services to help you build hybrid applications that combine Azure and on-premises resources. Which one you use depends on what you’re trying to do. Azure WCF Relay lets you expose Windows Communication Foundation services in on-premises applications to Azure. Azure Relay Hybrid Connections is a newer service that does the same thing for WebSockets and HTTP rather than WCF. You can also use Hybrid Connections from within Azure App Service. The Azure Data Management Gateway, now known as the Self-hosted Integration Runtime, lets you copy data between Azure and on-premises data stores. Finally, the Azure On-Premises Data Gateway connects on-premises data sources with Azure Analysis Services.

I have to admit that I get the Data Management Gateway and the On-Premises Data Gateway confused sometimes, because their names sound very similar, even though they’re totally different. Maybe that’s why Microsoft renamed Data Management Gateway to Self-hosted Integration Runtime. Of course, the new name doesn’t really help you remember what it does either. Oh well, after you’ve worked with them for a while, you’ll remember which one is which.

Azure Relay service allows you to expose on-premises applications to Azure without having to open a port on your firewall. The way this works is the server and the client communicate through an Azure Service Bus namespace. The on-premises application communicates with Service Bus over HTTP or HTTPS. Since these ports are already open on most firewalls, no changes are needed. Of course, both the server and the client need to include code that connects to Azure Relay.

Originally, Azure Relay was called Service Bus Relay and it only worked with Windows Communication Foundation services. Microsoft renamed it WCF Relay, because they created an alternative feature that doesn’t use WCF. It’s called Hybrid Connections and it works with WebSockets and HTTP. WCF Relay and Hybrid Connections exist as side-by-side alternatives in the Azure Relay service.

Microsoft considers WCF Relay to be a legacy service, though. If you have existing WCF-based applications, then WCF Relay is the way to go, but if you’re developing new applications, you should write them with WebSockets or HTTP instead and use Hybrid Connections. The biggest advantage of using WebSockets or HTTP is that they’re open standards, so you can use them in any language on any platform that supports them.

Azure App Service has a Hybrid Connections feature as well, but it works slightly differently. Suppose you have an Azure app that needs to access an on-premises database. You’d have to install a Hybrid Connection Manager (or HCM) in the network where the database resides. It runs on Windows Server 2012 and higher. Both sides would still connect to Service Bus, but the on-premises side would connect through the HCM to get to Service Bus. This would set up a TCP tunnel between the app and the database. The connection would be over HTTPS. Authentication and authorization would be done using a shared access signature.

By the way, if you already have a VPN connection between an Azure virtual network and an on-premises location, then App Service can use it through its VNet Integration feature. OK, on to the Self-hosted Integration Runtime (formerly known as the Data Management Gateway). This is an on-premises client that works with Azure Data Factory, so let’s go over how Data Factory works before diving into the Integration Runtime client.

Data Factory is used to move and transform data. For example, suppose that every night you want to copy records from SQL Database to Data Lake Store, then run a Spark job on an HDInsight cluster to process that data, and finally, store the processed data in SQL Data Warehouse. You could automate this process by creating a Data Factory pipeline that consists of three activities, one for each step.

Before creating the pipeline and activities, you’d need to define the data sources and sinks. First, you’d create a linked service for each data source. A linked service contains the connection information Data Factory will use to access the data. For example, the linked service for the input data would give the connection string for the SQL Database instance. Then you’d create a dataset that tells Data Factory which data to access. In this case, it would be the name of the table. You’d also need to create linked services and datasets for Data Lake Store and SQL Data Warehouse.

Now you could create a pipeline with 3 activities in it, one to copy the data from SQL Database to Data Lake Store, one to spin up an HDInsight cluster and run a Spark job, and one to copy the output data to SQL Data Warehouse. Once the pipeline is ready, you could run it manually, but it would be much easier to schedule it to run every night automatically, which you could do by setting a schedule trigger.

OK, so that’s how Data Factory normally works, but how do you get it to work with external data? That’s where the Self-hosted Integration Runtime comes in. You install it in your on-premises environment, register it with Azure, and connect it to your SQL Server instance. Then, when you create the linked service for the data source, you give it the connection details for the SQL Server instance, but you tell it to connect using the integration runtime.

Data Factory supports two other integration runtimes as well. The Azure Integration Runtime is the default, so you don’t normally need to explicitly create it. The Azure-SSIS Integration Runtime lets you run SQL Server Integration Services packages on Azure. An SSIS package is actually kind of like a Data Factory pipeline. It contains a series of tasks to transform data. If you already have SSIS packages that you’ve been running locally, then this is a great way to lift and shift them to Azure.

When you create an SSIS Integration Runtime, you are provisioning a cluster that can run SSIS packages. Once it’s provisioned, you can even use SQL Server Data Tools or SQL Server Management Studio to deploy packages to it. However, the SSIS integration runtime incurs charges while it’s running, so to save money, you can use Data Factory to start it, execute an SSIS package, and stop it. This is similar to how Data Factory spins up an HDInsight cluster to process data. If your SSIS packages need to access on-premises data, then you have to connect the integration runtime to a virtual network that’s connected to your on-premises environment.

Alright, now we’ll move on to the Azure On-Premises Data Gateway. This is something you install onsite that will let Azure Analysis Services retrieve on-premises data. First, you install it as a Windows service. Then you register it with the Gateway Cloud Service. Next, you create a gateway resource in Azure. Finally, you connect your Azure Analysis Services servers to your gateway resource.

Once you have it set up, your data models can access your on-premises data sources through the gateway resource, which goes through Azure Service Bus to reach the On-Premises Data Gateway, which connects to your data sources.

As you’ve seen, there are usually better ways to connect specific on-premises resources to Azure than using a VPN or ExpressRoute. Another example of this is with domain services. Most Azure customers already have a locally-installed Active Directory Domain, so if you have a direct connection to Azure, then it might seem like a good idea to just connect your Azure VMs to your on-premises domain servers. Although that would work, it could potentially be unreliable or slow, especially over a VPN.

A better solution is to use Azure Active Directory Domain Services. I won’t go into the details of synchronizing it with your on-premises domain, because that’s covered in our Active Directory course. I do want to point out a couple of things to watch out for when you’re joining an Azure VM to a domain, though. First, the VM needs to be in a virtual network that’s connected to the one where your managed domain resides. Otherwise, it won’t be able to join. Second, the user whose credentials you specify for joining the VM to the domain has to be a member of the Azure AD domain controller administrators group or it won’t work.

And that’s it for hybrid application connectivity.