Options for Operating Programmatically with AWS
Using the AWS Command Line Interface
AWS Systems Manager
AWS Secrets Manager
AWS Cloud Development Kit (CDK)
The course is part of this learning path
This course provides detail on the AWS Management & Governance services relevant to the AWS Certified Developer - Associate exam.
Want more? Try a lab playground or do a Lab Challenge!
- Learn how AWS AppConfig can reduce errors in configuration changes and prevent application downtime
- Understand how the AWS Cloud Development Kit (CDK) can be used to model and provision application resources using common programming languages
- Get a high-level understanding of Amazon CloudWatch
- Learn about the features and use cases of the service
- Create your own CloudWatch dashboard to monitor the items that are important to you
- Understand how CloudWatch dashboards can be shared across accounts
- Understand the cost structure of CloudWatch dashboards and the limitations of the service
- Review how monitored metrics go into an ALARM state
- Learn about the challenges of creating CloudWatch Alarms and the benefits of using machine learning in alarm management
- Know how to create a CloudWatch Alarm using Anomaly Detection
- Learn what types of metrics are suitable for use with Anomaly Detection
- Create your own CloudWatch log subscription
- Learn how AWS CloudTrail enables auditing and governance of your AWS account
- Understand how Amazon CloudWatch Logs enables you to monitor and store your system, application, and custom log files
- Explain what AWS CloudFormation is and what it’s used for
- Determine the benefits of AWS CloudFormation
- Understand what each of the core components are and what they are used for
- Create a CloudFormation Stack using an existing AWS template
- Learn what VPC flow logs are and what they are used for
- Determine options for operating programmatically with AWS, including the AWS CLI, APIs, and SDKs
- Learn about the capabilities of AWS Systems Manager for managing applications and infrastructure
- Understand how AWS Secrets Manager can be used to securely encrypt application secrets
Firstly, why do we have services that are dedicated to centrally storing parameters and secrets for us? They allow us to implement security best practices preventing us from having to hardcode any credentials within custom applications. This minimises administrative burden when it comes to rotation, in addition to enhancing the security posture of our AWS accounts. Having a central store of all of our secrets makes it easy for applications to retrieve the value of a secret.
Ok, so let’s start by establishing what both the Parameter Store and Secrets Manager are used for before we compare them against each other.
So the Parameter store is a feature that is a part of the AWS Systems Manager service, also known as SSM, and this service gives you centralized visibility and control allowing for operational, application, change and node management across your AWS infrastructure.
AWS Systems Manager Parameter Store feature sits under the Application management element of SSM. The Parameter Stores main function is to provide you with a means of centrally storing parameters to be used within your environment allowing your applications to retrieve the parameter value via simple API calls as and when required. Additional security features can be added to these parameters, for example using encryption to hide sensitive data, such as passwords and other secrets. Other examples of parameters may include environment variables, database strings, plain text data strings, and AMI-IDs, basically anything that could be a parameter.
When setting up your parameter, you can configure it as as a single string of any value, a StringList, which would contain multiple string separated by commas, or a SecureString which is backed by the AWS Key Management Service to provide a means of encrypting your parameter value.
So let’s now take a quick look at AWS Secrets Manager. AWS Secrets Manager holds, protects and contains sensitive secret information for you, allowing other services and applications to call for the secret via a simple API call. This negates the need for your application developers to hard-code any secret or credentials within your applications, instead, when a secret value is required, an API call to AWS Secrets Manager is triggered which will then return the value.
Within Secrets Manager, a secret is something that you want to remain hidden and protected instead of being available as open access to anyone who can read it. This can include database credentials across Amazon Redshift clusters and Amazon RDS, instance passwords, plaintext, or API keys, all of which are encrypted by default.
So that’s a very high level as to what the Parameter Store of SSM and AWS Secrets Manager is used for, and you’re probably wondering what the difference is, I mean they both store secrets and parameters that can be retrieved by your applications via an API call as its needed, so you might be a little confused, so let me break the differences down.
Firstly, let me talk about encryption, with the Parameter store, encryption is available, however it is not enabled by default. Not all of the parameters that are stored need to be, some do not contain any sensitive information and so they do not need to be encrypted. However, some should have that added security and additional protection, such as passwords. In this instance you need to use the SecureString option allowing you to select a KMS CMK to encrypt the parameter with.
Now, if we compare this to Secrets Manager, which by definition in its name, is used to store sensitive data, ‘secrets’ and so encryption is enabled by default. In fact you can’t disable encryption. So for every secret stored in Secrets Manager you must select one of the following KMS CMK keys to perform the encryption:
- The default AWS owned KMS key of DefaultEncryptionKey
- The AWS managed key of aws/secretsmanager
- Or a customer-managed KMS key of your own
So to clarify, the Parameter store does offer encryption, but not be default, whereas with Secrets Manager it is enabled by default and can’t be disabled.
The next point I want to raise is the issue of rotation, and this point is specific to your secrets, such as your passwords. When using the Parameter store to save credentials, then, as we already know we can have them encrypted, however, security best practices dictate that you should rotate your credentials frequently to prevent them from being compromised over time. Unfortunately, the Parameter Store doesn’t offer the ability to implement any automated password rotation lifecycles for your parameter values.
For example, if you have a custom application that connects to an RDS database then you could use the Parameter Store to securely store the credentials in a SecureString parameter. This would allow your application to retrieve the credentials to connect to the DB when required. However, let’s now consider you rotated and changed your RDS password to meet security requirements and best practices, the credentials within the Parameter store would now contain the out of date credentials and the application would fail to connect to the DB. To rectify this you would need to manually update the parameter value to match the new credentials set in RDS.
If we look at rotation from the perspective of AWS Secrets Manager, then it’s a slightly different story. Secrets Manager was designed with automated rotation in mind, to a degree at least. By default, and at the time of writing this course, AWS Secrets Manager is able to natively update and rotate the credential held on the following DB automatically:
- RDS, including the supported DB engines
- Amazon Aurora on Amazon RDS
- MySQL on Amazon RDS
- PostgreSQL on Amazon RDS
- Oracle on Amazon RDS
- MariaDB on Amazon RDS
- Microsoft SQL Server on Amazon RDS
- Amazon DocumentDB
- Amazon Redshift
This is managed and implemented with the assistance of inbuilt Lambda Functions to perform the rotation of the credentials, and these values are then reflected in the secret value stored in AWS Secrets Manager.
This means AWS Secrets Manager can automatically rotate your passwords for you to align with best practices and you never have to manually update your secret value or your application.
If you wanted to perform automatic rotation for other services, including other databases then you would have to write your own Lambda function to perform the operation. If you would like to investigate this further, please see the AWS Documentation here: https://docs.aws.amazon.com/secretsmanager/latest/userguide/rotating-secrets-lambda-function-overview.html
Let’s now take a look at the main use case of the Parameter Store and AWS Secrets Manager. The Parameter store as I briefly highlighted earlier can be used for a variety of use cases, including:
- Database strings
- Product codes
- Plain text data strings
- License keys
- And basically, any environment variable that you need
As you can see, the parameter store offers a wide use case range which can include both secret and non-secret values. Non-secret values can be stored in plain-text without any encryption required, and secret values can be encrypted using KMS as we know.
Secrets Manager is only really designed for Secrets which are always encrypted by default. So it’s designed for a more refined use case, focusing on storing those more sensitive values.
Interestingly and an important point to mention is that whichever service you are using SSM Parameter Store, or Secrets Manager, the values are referenceable using AWS CloudFormation. So from a deployment use case, both can be used as external values enabling CloudFormation to reference.
When working in larger environments you might want to have a centralized store of your secret values instead of having multiple stores between different accounts. With AWS Secrets Manager you have the capability to share your secrets securely between AWS accounts as and when required. More information on this process including a demonstration can be found in our existing course here, and it will explain how to securely allow multiple identities in multiple AWS accounts to access secrets from within another AWS account using AWS secrets manager and resource-based policies: https://cloudacademy.com/course/aws-secrets-manager/sharing-secrets-with-aws-secrets-manager/
Unfortunately with the Parameter Store, it’s not possible to share parameters between different AWS accounts, so you can’t have a centralised location of all of your parameter values between your accounts, you need to create a different separate store in each AWS account.
Between both the Parameter Store and Secrets Manager you can store different sized parameters and secrets.
The Parameter Store provides 2 different tiers when storing your parameter values, these being standard and advanced. With Standard you are able to store values up to 4 KB in size, with Advanced this increases to an 8 KB parameter size, however the advanced incurs a cost where as Standard is free. You can change your parameter tier from Standard to Advanced, however you can’t then revert it back.
AWS Secrets Manager doesn’t offer different tiering levels for your secrets, instead it offers a flat maximum secret size of 10 KB, which is useful to help your store certificates which might have a long chain of trust.
Finally, let me provide a quick overview of how these services are priced, starting with the Parameter Store.
As I just mentioned, the Parameter Store uses 2 different tiers, Standard and Advanced. Any of the parameters that you configure within the Standard Tier are free, however any parameters stored in the Advanced tier then you will pay for both the number of parameters you have in addition to the number of API interaction with that parameter per month.
For accurate pricing information on storage and API interaction for the Advanced tier please see the pricing page for SSM Parameter Store here: https://aws.amazon.com/systems-manager/pricing/
AWS Secrets Manager does not offer a free tier like the Parameter Store does, however you are charged for the storage of each secret in addition to the number of API calls made upon your secrets that you have stored each month. For accurate pricing for Secrets Manager, please refer to the pricing documentation here: https://aws.amazon.com/secrets-manager/pricing/
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 150+ courses relating to Cloud reaching over 180,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.