AWS Logging Mechanisms
The course is part of these learning pathsSee 2 more
Before diving deeper into the technical aspects of this course, this lecture provides an overview as to why as an organization would find it beneficial to perform logging and monitoring of their environment. Data collated through different services is so valuable to operational teams it's key that this data is retained, understood, and viewed to enable you to maintain a high level of service, both operationally and from a security standpoint.
The lecture begins by highlighting why we should perform logging before providing a number of different benefits to your organization. These benefits include: 'Audit Control' allowing you to retain data for both internal and external auditors to comply with controls. 'Incident Detection and Resolution' allowing you to pinpoint root cause. 'Monitoring and alerting' which helps to proactively respond to potential threats. 'Trend Analysis' which can be used to provide a baseline of normal performance helping you to detect anomalies easier. 'Understanding your infrastructure' having more information about the performance and state of your infrastructure is fundamental when it comes to maintaining a stable environment. By the end of this lecture, you will understand the importance of logging and how this data can help your organization in a number of different scenarios.
Hello, and welcome to this short lecture, where I want to discuss a few of the different benefits that logging can bring you and your infrastructure. For some, many people consider logging an afterthought, something that is implemented after it's too late. This is often the case where an incident or breach of security has occurred that resulted in a delay of resolution and safeguarding of your environment. In hindsight of these situations, logging would have been a great idea to have had running and implemented in the first place to rectify the event quickly and efficiently, or even prevent it from happening in the first place.
So how can logging help?
Generally, logs are created by services and applications which contain a huge amount of information, which is recorded and retained on persistent storage, to be reviewed and analyzed at any time that it might be needed. Some logs can be monitored in real time, allowing automatic responses to be carried out, depending on the data contents of the log. From an auditing perspective, these logs are invaluable. They often contain vast amounts of metadata, including date stamps, source information such as IP address or usernames, and this is especially true when you're looking at CloudTail logs. These logs can be used to help you achieve specific compliance certifications that require evidence of traceable and auditable actions that have been carried out.
Being able to resolve an incident as quickly as possible is paramount within your organization. Whether it's a priority one, two, or three, being able to gain as much insight into what happened just before and just after the incident can significantly reduce your time to resolution. Using logs to ascertain the state of your environment before and after and even during the incident provides clarity and enables you to detect where the incident occurred, allowing you to pinpoint your efforts in a specific area. Quicker resolution results in a better customer experience for your organization.
By monitoring the data within your logs, you're able to quickly identify potential issues that you want to be made aware of as soon as they occur. By combining this monitoring of logs with thresholds and alerts, you are able to receive automatic notifications of potential issues, threats, and incidents, prior to them becoming a production issue. By logging what's happening within your applications, network, and other cloud infrastructure, you are able to build a baseline of performance and establish what's routine and what isn't. By having this baseline, you are able to identify threats and anomalies easier through the use of third party tools and management services.
To have a thorough understanding of what's happening within your infrastructure provides a huge benefit to your operational teams. Having an inside look of how your infrastructure is performing and communicating helps achieve the previous benefits that I've already discussed, and having more data about how your environment is running far outweighs the disadvantage of not having enough information, especially when it really matters to your business in the case of incidents and security breaches.
That now brings me to the end of this lecture. There are many more reasons as to why you should be capturing data that can be logged. But I just wanted to provide a few key points to you.
Coming up next, I shall be looking at how you can implement CloudWatch logs to collect different metrics.
Stuart has been working within the IT industry for two decades covering a huge range of topic areas and technologies, from data center and network infrastructure design, to cloud architecture and implementation.
To date, Stuart has created 90+ courses relating to Cloud reaching over 100,000 students, mostly within the AWS category and with a heavy focus on security and compliance.
Stuart is a member of the AWS Community Builders Program for his contributions towards AWS.
He is AWS certified and accredited in addition to being a published author covering topics across the AWS landscape.
In January 2016 Stuart was awarded ‘Expert of the Year Award 2015’ from Experts Exchange for his knowledge share within cloud services to the community.
Stuart enjoys writing about cloud technologies and you will find many of his articles within our blog pages.