Building High Availability into your environment
Understanding SLAs in AWS
Which services should I use to build a decoupled architecture?
Managing RTO and RPO for AWS Disaster Recovery
The course is part of this learning path
The simple notification service also called SNS implements a “one to many” message distribution model. A single message is published to a topic and that message gets published to one or more subscribers to the topic.
In this case, the interaction point similar to a mailbox is called a topic, applications generating messages are called publishers and systems receiving messages are called subscribers. A topic does not store messages like queues do. There is no message preservation in a topic. Once a message is published to a topic the message is pushed to all existing subscribers. Once a message is published there is no mechanism to recall the message back.
SNS can send push notification messages to mobile devices and desktops using one of the many supported push notification services including:
- Apple push Notification Service for both iOS and Mac OS X
- Windows Push notification Services.
- Amazon Device Messaging
- Baidu Cloud Push (Baidu)
- Firebase Cloud Messaging
Subscriber types can also be HTTPS URL endpoints, email addresses, AWS Lambda Functions, or even text messages to phone numbers via Short Message Service (SMS) for clients in over 200 countries and for an expanded set of use cases like Multi Factor Authentication (MFA) and One time passwords (OTP). You can also mark SMS messages as Transactional or Promotional based on the content of your messages.
The AWS documentation for Amazon SNS is updated regularly to reflect the details of using SMS depending on the destination country. Please, check it for details before using SNS for sending text messages.
Amazon SNS can also publish messages to Amazon SQS queues to implement reliable and asynchronous “fan-out” message distribution patterns.
The idea of notifications is to maximize the message distribution by permitting the same message to be pushed to multiple subscribers to the same topic. SNS is the go to service in order to implement notifications using AWS services including events.
You can seamlessly scale from a handful of messages per day to millions of messages or more. A single topic can handle over 12 million subscriptions and you can have up to a hundred thousand topics if needed and publishing at a rate of 30,000 messages per second in the US East 1 region. The message processing rate per second changes between regions. Please verify the documentation for details in the region you use.
The maximum message size is 256KB and SNS breaks messages into 64KB chunks and deal with them as individual requests. Similar to SQS Amazon SNS producers can use the SNS Extended Client for Java in order to publish messages that are larger than 256KB and up to 2GB in size. The library uses Amazon S3 to store the message payload and publishes the reference to the object to the the topic. Subscribers from SQS can use the SQS Extended client Library for Java to retrieve the payloads from S3. Lambda functions and other endpoints will need to use the Payload Offloading Java common Library for AWS in order to access the objects stored in S3.
Message filtering enables topic subscribers to selectively receive only a subset of the messages that interest them as opposed to receiving all messages published to a topic. This is done using a subscription filter policy in JSON format. The JSON policy contains attributes that define which messages to deliver to the subscriber. SNS compares the attributes on each message published to the attributes in the filter for each subscriber and sends the message only if any of the attributes in the message match the subscriber’s filter policy attributes. This filtering mechanism decouples any message routing logic from message publishers and eliminates the need to use different topics for different conditions. It allows you to use a single topic for publishers and differentiate your messages using attributes that each corresponding subscriber will match in it’s filter policy.
An Amazon SNS Delivery Policy can be used to control the retry pattern for message delivery as linear, geometric, exponential backoff, maximum and minimum retry delays, and other parameters during message delivery by the SNS topic. When the message delivery retry policy is exhausted, Amazon SNS can move the message to a dead-letter queue (DLQ).
A dead-letter queue as discussed before is an Amazon SQS queue that an Amazon SNS subscription can target for messages that can't be delivered to subscribers successfully. Messages that can't be delivered due to client errors or server errors are held in the dead-letter queue for further analysis or reprocessing.
Let’s compare these two messaging services side by side.
Amazon SNS allows applications to send time-critical messages to multiple subscribers through a push mechanism.
Amazon SQS exchanges messages through a polling model: sending and receiving components are decoupled.
Amazon SQS provides flexibility for distributed components of applications to send and receive messages without requiring each component to be concurrently available. This is due to the message persistency feature and decoupling mechanism.
In order to fanout identical messages to multiple SQS queues the process is to Create a topic first using SNS. Then create and subscribe multiple SQS queues to the SNS topic. With that defined, whenever a message is sent to the SNS topic, the message will be fanned out to the SQS queues. SNS will deliver the message to all the SQS queues that are subscribed to the topic and once the message is put on each queue it will trigger any polling application as needed.
The process in diagram form would look as shown. This is your basic fan-out implementation using SNS and SQS combined.
In this basic fragment all three queues are subscribed to the topic for notifications and the same message will be published to all three in order to dispatch the image processing required.
We can expand on this scenario and include the fan out portion into an existing application for image processing. The user uploads the image and then manually triggers a function that sends the necessary message to a designated SNS topic. This will start the rest of the workflow.
We can improve on the prior diagram because Amazon S3 buckets are event aware and they can trigger a notification to SNS when an object is uploaded. In this alternative scenario, uploading the image to Amazon S3, triggers an event notification, which sends the message to the SNS topic automatically.
This course covers the core learning objective to meet the requirements of the 'Designing for disaster recovery & high availability in AWS - Level 2' skill
- Analyze the amount of resources required to implement a fault-tolerant architecture across multiple AWS availability Zones
- Evaluate an effective AWS disaster recovery strategy to meet specific business requirements
- Understand SLA for AWS services to ensure the high availability of a given AWS solution
- Analyze which AWS services can be leveraged to implement a decoupled solution
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.