I want to talk in detail about what RDS Performance Insights is compared to Amazon CloudWatch and RDS Enhanced Monitoring and discuss database load.

CPU utilization and database load can be related, but they are independent of each other. A database can be under high load, but have low CPU utilization. Likewise, a CPU could be experiencing high utilization levels, but be running only a few queries.

Performance Insights is a feature of Amazon RDS that provides visibility into the performance of SQL queries inside RDS databases.

More than that, it was designed to be used by people with a wide range of database management experience. Wide range includes people that are new to database administration, as well as experienced professionals.

AWS recognizes that there are IT departments that either don't have a DBA or if they do, it is somebody working part-time.

In my career, I've been an accidental DBA more than once. My primary job roles have been around systems administration, network management, and systems engineering. I understand databases and know how they work. I can create and run queries. However, I have not been very comfortable with database internals.

The last time I had the role of DBA given to me, my management did not recognize the value of having someone act as a steward or caretaker of the database. Because of this, the DBA quit and it was up to me to discover, mostly through trial and error, how to address issues.

It was a sub-optimal experience. Learning on the job using production systems is a dangerous gamble. Nobody wins.

When Amazon RDS was first introduced, Amazon Cloudwatch was used exclusively for metrics collection and CloudWatch Logs for log management. With CloudWatch, metrics are captured once every minute from the hypervisor layer. If you're familiar with monitoring EC2 instances, RDS monitoring using CloudWatch is identical to EC2's detailed monitoring. This hypervisor-based monitoring is limited to, depending on the instance type, between 15 and 18 different metrics. 

CloudWatch monitoring uses a traditional approach to database monitoring. Systems and network administrators expect computing devices to behave in a certain way. More accurately, the expectation is that computing devices like databases will fail in specific ways and it is important to watch for it.

Predicting failure is difficult in the cloud because the number of ways something can fail has increased. CloudWatch metrics were not enough.

Enhanced Monitoring does metric data collection on the RDS instance. It uses a lightweight agent to get metric data from the operating system.

While CloudWatch has between 15 and 18 metrics available, depending on the database engine, Enhanced Monitoring provides over 50.

Enhanced Monitoring defaults to collecting metric data at 60-second intervals, but can be set to once per second.

Because data is being collected from the operating system, it can present a list of processes and threads like the Linux top command.

This provides more data points than CloudWatch, but like CloudWatch, it is still collecting data about the database instance and not from the database engine.

Performance Insights collects metric data from the database engine to monitor the actual load on a database. The metrics are visible via a dashboard inside the RDS console using CloudWatch programmatically by using an API or the command line interface. The RDS Performance Insights dashboard is an interactive graph filtered by one of four attributes: SQL, user, host, and weight.

Monitoring systems like Amazon CloudWatch and Enhanced Monitoring can show 10 or more graphs of time series metrics that require end users like me to figure out what's happening inside a database. Like an episode of a television crime show or a murder mystery, I have to hunt for clues to figure out what happened, when it happened and what caused it.

Dashboard visualization such as these present facts, but it is up to me to figure out what story is being told. Sometimes playing the role of a detective not only do I have to figure out what caused the issue, I have to determine if there really was a problem at all. For example, it is easy to blame a database for performance issues, but sometimes those issues originate in an application or within the network configuration.

Performance Insights simplifies the storytelling by condensing the performance data points into a single metric that tell a very specific story. Now, instead of a murder mystery with 10 or more possible suspects each with their own motive, weapon, and possible accomplices, there is one suspect to follow. Also, because of how Performance Insights was designed, I can determine whether or not there truly was a victim. The database itself might be fine. This condensed metric measures database load and I will cover that in the next lecture.