Data Transfers with AWS DataSync
Running Operations with the Snow Family
Amazon CloudFront Design Patterns
Inter-Regional and Intra-Regional Communication Patterns
Understanding Direct Connect
The course is part of this learning path
Instructor: Mike Brown
VPC Endpoints and Endpoint Services
There will be many occasions when resources deployed to a VPC will want to connect to other AWS services such as S3 or DynamoDB. Services like S3 and DynamoDB have public endpoints, most offering regional endpoints that you can connect to programmatically.
This means that if you have an application running on an EC2 instance in a VPC in eu-west-2, it can use the S3 regional endpoint to connect to a bucket and read or write content.
The problem is these connections are using public endpoints so the traffic from your EC2 instances will travel on public networks to get to the S3 bucket.
Using VPC Endpoints, you can connect to services provided by AWS and 3rd parties through either a gateway endpoint or an interface endpoint. These endpoints keep your traffic on the AWS backbone keeping your data away from public networks so therefore more secure. Also as your data is not being routed to public networks and instead being routed directly to the service your application should also see reduced latency.
Only two AWS services provide Gateway Endpoints, S3 and DynamoDB.
When you configure a Gateway Endpoint a new route is added to your VPC route tables, you get to choose which VPCs and which route tables the routes are added to. Here is an example of an S3 endpoint route in a VPC route table.
It appears in the route table as a destination and a target, in other words any traffic addressed to destination Pl-7ca54015 will be sent to the VPC endpoint target.
Pl-7ca54015 is a prefix list, you can see more details of the prefix list in this screenshot.
The prefix list entries are the CIDR blocks that our traffic will be matched against. If our traffic is addressed to any IP address matched by these CIDRs it will be routed through the VPC endpoint, in this example the endpoint to S3 in the eu-west-2 region.
Instead of adding routes to route tables, Interface Endpoints create Elastic Network Interfaces (ENIs) and attach them to the subnets that you choose. The Interface Endpoint ENIs obtain IP addresses from the subnets they attach to. To route traffic to a service you send that traffic to the appropriate ENI IP address.
All other AWS services that support VPC endpoints are configured as Interface Endpoints as are all VPC Endpoints that connect to 3rd party services. In addition VPC Endpoints to S3 and DynamoDB can also be configured as Interface Endpoints.
Differences between Interface and Gateway Endpoints
- Gateway Endpoints are free whereas when using Interface Endpoint you are charged an hourly fee for the endpoint and a per GB fee for data sent through the endpoint.
- Gateway Endpoints can not be accessed from outside of the VPC, Interface Endpoints can be accessed from any source so long as that source can route traffic to the VPC subnets that endpoints ENIs are connected to.
It is worth remembering that VPC Endpoints are regional, the only exception to this is S3 that offers a global S3 interface endpoint. Because VPC Endpoints are regional, if you want to create an Endpoint that connects to a DynamDB database in eu-west-2 then you must create and attach the endpoint to subnets in eu-west-2.
But what if you want to connect from an EC2 instance in eu-west-1 to a DynamoDB database in eu-west-2, can you use Interface Endpoints?
The answer is yes.
- In eu-west-2 create a VPC and Subnets.
- In eu-west-2 create an Interface VPC Endpoint that connect to DynamoDB.
- Then either create a VPC Peering connection between the VPC that hosts the EC2 instance in eu-west-1 with the VPC in eu-west-2 that hosts the endpoint, or attach both VPCs to Transit Gateways.
As well as creating VPC Endpoints that connect you to AWS services and 3rd party services through the AWS backbone, you can also advertise your services through the AWS backbone so that your customers or other business units can connect to your service without having to send traffic out onto the internet.
To create an Endpoint Service:
- Create the compute services to host your application e.g an EC2 instance.
- Create a network load balancer and a target group.
- Add the compute services that host your application to the target group.
- Through the VPC console or CLI create your Endpoint Service.
Depending on the service you are trying to advertise, you can also use Gateway Load Balancers with Endpoint Services.
Here you can see the properties of a newly created Endpoint Service. The arrow is pointing at the Allow Principals tab. Through this tab you can identify entities that are allowed to access your Endpoint Services.
The entities that are allowed to access your endpoint service could be:
- An AWS account
- A specific IAM user
- A specific IAM role
Once the Endpoint Service is created, your customers or other business units can connect to your Endpoint Service by creating a VPC endpoint and searching for your service.
This course covers the core learning objective to meet the requirements of the 'Designing Network & Data Transfer solutions in AWS - Level 2' skill
- Understand the most appropriate AWS connectivity options to meet performance demands
- Understand the appropriate features and services to enhance and optimize connectivity to AWS public services such as Amazon S3 or Amazon DynamoDB.
- Understand the appropriate AWS data transfer service for migration and/or ingestion
- Apply an edge caching strategy to provide performance benefits for AWS solutions
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.