The AWS cloud has thousands of services and features.  It’s easy to be amazed and overwhelmed by the number of choices available. 

However, inside AWS, the three primary types of services used by customers are compute, storage, and databases.

My focus, in this course, is on the managed database offerings available from AWS and the types of workloads they support.

Databases are the foundation of modern application development.  A database’s implementation and how data is structured will determine how well an application will perform as it scales.

There are two primary types of databases, relational and non-relational.  

Relational databases, sometimes called SQL databases because they use the Structured Query Language to manage information storage and retrieval, have been commercially available since the 1970s and are optimized around data storage.

Sometimes the Structured Query Language is called Ess-Queue-Ell and other times it’s pronounced Sequel.  Both are correct.

Non-relational databases, also called NoSQL databases because data is stored and retrieved primarily using methods other than SQL, became popular in the 21st Century.  Over the past 20+ years, developers have created web-based applications that needed to process large amounts of unstructured and semi-structured data quickly and reliably.  

NoSQL databases are often distributed databases where data is stored on multiple computers or nodes.

A database is any type of mechanism used for storing, managing, and retrieving information.  It is a repository--or collection--of data.  

Though, an application developer probably thinks of a database as a place to put the stuff their application needs to run.

There are nine primary categories of databases available on AWS.

1. Relational Databases
2. Key-Value Databases
3. Document Databases
4. In-Memory Stores
5. Graph Databases
6. Columnar Databases
7. Time Series Databases
8. Quantum Ledger Databases
9. Search

Each one of these database types is optimized to support a specific type of workload.  

Matching an application with the appropriate database type is essential for highly performant and cost-efficient operation.

I can remember a time when choosing a database was a platform choice.  The underlying technology was almost secondary.  Three or four vendors would be considered, one would be chosen as a primary platform, and every application would be built using it.  It was less of a database management system and more of a vendor relationship.

Many people had a favorite database and felt that it could be used to manage any type of workload.  On a small scale, this might be true.  

If I have a small car, I can take 2-3 kids to school every day without issue.

However, what happens when I need to get 20 kids to school?  The small car would still work but would be terribly inefficient.  It would take several trips, take a considerable amount of time, be very hard on the car, and waste resources.  A better option is the school bus.

Thankfully, the days of choosing a single database solution for all applications are long past.  

While it is possible to make a general-purpose database manage just about any type of workload, it will not scale as demand increases. 

One of the reasons this is important is because the cloud has the promise of agility.  That is, being able to respond to changes in a business environment as they happen.  

This includes the ability to grow as needed and it’s called scalability.

Another promise of the cloud, one that is talked about but not always understood, is elasticity.  That is, it’s great to be able to scale up to meet demand but, elasticity allows for the opposite to happen.  When demand decreases, so does the scale and its related expenses.

Who wants to pay for idle resources?

Today, it’s important for application developers to examine the data and consider its size, shape, and computational requirements for processing and analysis.  These three things determine what type of database is needed for a particular application to allow for scalability and elasticity.  

This also means there could be multiple databases used by an application; each one serving a unique purpose. 

The two primary workload types are operational and analytical.

Operational applications are the ones often referred to as Online Transactional Processing applications, OLTP.  

OLTP applications are among the most common built today.  A transaction is a record of an exchange.  OLTP is centered around a set of common business processes that is regular, repeatable, and durable.  

It could be something like e-commerce, a content management system, or information management.  

Data goes in and reports come out.  This is regular, expected work.  Often, the database on the back end is relational. The data is very structured, and the results are predictable.

The other type of workload is the analytical application.  Online Analytics Processing applications, OLAP, are run--as needed--for things like business intelligence workloads and data analysis. 

The goal is to gain insight.  Workloads are often Retrospective, Streaming, and Predictive.

Retrospective analytics examine an organization’s history.  What happened last quarter, last year, or--maybe--the last five years.  

Streaming workloads are gathering data, in real-time, to discover trends or raise an alarm.

Predictive analytics uses data to try to look into the future.  There are applications that are beginning to use Machine Learning and Artificial Intelligence to improve.  

Applications that process analytics workloads are run on-demand.  It is unknown what questions they’re going to answer.  The data used has little structure and the workloads, themselves, are unpredictable.

Databases power 21st Century applications.

The efficient use of resources in the cloud requires organizations to be agile.  Agility implies being responsive to change.  However, because costs in the cloud are primarily based on consumption, it is important to consider the requirements of scalability and elasticity.

Resources should grow as demand increases but they also need to shrink as that demand subsides.

Online Transactional  Processing Applications, OLTP, are usually powered by relational databases.  The data is highly structured, controlled, and predictable.

Online Analytics Processing Applications, OLAP, are often powered on the backend using non-relational databases.  The data for these applications is either semi-structured or unstructured, the workloads are unpredictable, and the output is used to answer questions about the unknown. 

That’s a little about what databases do, but why is structured data important?  The answer to that question involves something called a schema, and you’ll learn about it in the next lecture on relational databases.

Lectures

Course Introduction - Relational Databases - NoSQL Databases - Types of Managed NoSQL on AWS - Part 1 - Types of Managed NoSQL on AWS - Part 2 - Summary and Conclusion