Utilizing Managed Services and Serverless Architectures to Minimize Cost
Amazon API Gateway
Amazon Elastic Map Reduce
Design a Multi-Tier Solution
When To Go Serverless
Which services should I use to build a decoupled architecture?
The course is part of this learning path
The simple notification service also called SNS implements a “one to many” message distribution model. A single message is published to a topic and that message gets published to one or more subscribers to the topic.
In this case, the interaction point similar to a mailbox is called a topic, applications generating messages are called publishers and systems receiving messages are called subscribers. A topic does not store messages like queues do. There is no message preservation in a topic. Once a message is published to a topic the message is pushed to all existing subscribers. Once a message is published there is no mechanism to recall the message back.
SNS can send push notification messages to mobile devices and desktops using one of the many supported push notification services including:
- Apple push Notification Service for both iOS and Mac OS X
- Windows Push notification Services.
- Amazon Device Messaging
- Baidu Cloud Push (Baidu)
- Firebase Cloud Messaging
Subscriber types can also be HTTPS URL endpoints, email addresses, AWS Lambda Functions, or even text messages to phone numbers via Short Message Service (SMS) for clients in over 200 countries and for an expanded set of use cases like Multi Factor Authentication (MFA) and One time passwords (OTP). You can also mark SMS messages as Transactional or Promotional based on the content of your messages.
The AWS documentation for Amazon SNS is updated regularly to reflect the details of using SMS depending on the destination country. Please, check it for details before using SNS for sending text messages.
Amazon SNS can also publish messages to Amazon SQS queues to implement reliable and asynchronous “fan-out” message distribution patterns.
The idea of notifications is to maximize the message distribution by permitting the same message to be pushed to multiple subscribers to the same topic. SNS is the go to service in order to implement notifications using AWS services including events.
You can seamlessly scale from a handful of messages per day to millions of messages or more. A single topic can handle over 12 million subscriptions and you can have up to a hundred thousand topics if needed and publishing at a rate of 30,000 messages per second in the US East 1 region. The message processing rate per second changes between regions. Please verify the documentation for details in the region you use.
The maximum message size is 256KB and SNS breaks messages into 64KB chunks and deal with them as individual requests. Similar to SQS Amazon SNS producers can use the SNS Extended Client for Java in order to publish messages that are larger than 256KB and up to 2GB in size. The library uses Amazon S3 to store the message payload and publishes the reference to the object to the the topic. Subscribers from SQS can use the SQS Extended client Library for Java to retrieve the payloads from S3. Lambda functions and other endpoints will need to use the Payload Offloading Java common Library for AWS in order to access the objects stored in S3.
Message filtering enables topic subscribers to selectively receive only a subset of the messages that interest them as opposed to receiving all messages published to a topic. This is done using a subscription filter policy in JSON format. The JSON policy contains attributes that define which messages to deliver to the subscriber. SNS compares the attributes on each message published to the attributes in the filter for each subscriber and sends the message only if any of the attributes in the message match the subscriber’s filter policy attributes. This filtering mechanism decouples any message routing logic from message publishers and eliminates the need to use different topics for different conditions. It allows you to use a single topic for publishers and differentiate your messages using attributes that each corresponding subscriber will match in it’s filter policy.
An Amazon SNS Delivery Policy can be used to control the retry pattern for message delivery as linear, geometric, exponential backoff, maximum and minimum retry delays, and other parameters during message delivery by the SNS topic. When the message delivery retry policy is exhausted, Amazon SNS can move the message to a dead-letter queue (DLQ).
A dead-letter queue as discussed before is an Amazon SQS queue that an Amazon SNS subscription can target for messages that can't be delivered to subscribers successfully. Messages that can't be delivered due to client errors or server errors are held in the dead-letter queue for further analysis or reprocessing.
Let’s compare these two messaging services side by side.
Amazon SNS allows applications to send time-critical messages to multiple subscribers through a push mechanism.
Amazon SQS exchanges messages through a polling model: sending and receiving components are decoupled.
Amazon SQS provides flexibility for distributed components of applications to send and receive messages without requiring each component to be concurrently available. This is due to the message persistency feature and decoupling mechanism.
In order to fanout identical messages to multiple SQS queues the process is to Create a topic first using SNS. Then create and subscribe multiple SQS queues to the SNS topic. With that defined, whenever a message is sent to the SNS topic, the message will be fanned out to the SQS queues. SNS will deliver the message to all the SQS queues that are subscribed to the topic and once the message is put on each queue it will trigger any polling application as needed.
The process in diagram form would look as shown. This is your basic fan-out implementation using SNS and SQS combined.
In this basic fragment all three queues are subscribed to the topic for notifications and the same message will be published to all three in order to dispatch the image processing required.
We can expand on this scenario and include the fan out portion into an existing application for image processing. The user uploads the image and then manually triggers a function that sends the necessary message to a designated SNS topic. This will start the rest of the workflow.
We can improve on the prior diagram because Amazon S3 buckets are event aware and they can trigger a notification to SNS when an object is uploaded. In this alternative scenario, uploading the image to Amazon S3, triggers an event notification, which sends the message to the SNS topic automatically.
This section of the AWS Certified Solutions Architect - Professional learning path introduces common AWS solution architectures relevant to the AWS Certified Solutions Architect - Professional exam and the services that support them. These services form a core component of running resilient and performant architectures.
- Learn how to utilize managed services and serverless architectures to minimize cost
- Understand how to use AWS services to process streaming data
- Discover AWS services that support mobile app development
- Understand when to utilize serverless services within your AWS solutions
- Learn which AWS services to use when building a decoupled architecture
Danny has over 20 years of IT experience as a software developer, cloud engineer, and technical trainer. After attending a conference on cloud computing in 2009, he knew he wanted to build his career around what was still a very new, emerging technology at the time — and share this transformational knowledge with others. He has spoken to IT professional audiences at local, regional, and national user groups and conferences. He has delivered in-person classroom and virtual training, interactive webinars, and authored video training courses covering many different technologies, including Amazon Web Services. He currently has six active AWS certifications, including certifications at the Professional and Specialty level.