Creating and Running a Scheduled Event with AWS Lambda

Start course

In this course, we explore some of the common use cases for serverless functions and get started with implementing serverless functions in a simple application.



Hello, and welcome to this lecture where we continue to explore use cases for serverless functions. In this lecture, we are going to create and run a scheduled event using AWS Lambda. First, we'll learn to recognize and explain how scheduled events can be handled by an AWS Lambda function. We will then create a scheduled event using Lambda, which will monitor a webpage every minute or so for us. Once we have our scheduled monitoring tasks set up and running, I'll then show you how you can monitor your Lambda function using AWS CloudWatch and how we can create simple alarms to notify us if an error or change occurs. Now AWS Lambda allows you to implement and deploy small computing units by delivering simple functions. Now scheduled tasks, similar to Linux Cron jobs, are a perfect fit for this computing model. You can configure AWS Lambda to be invoked periodically via the Amazon CloudWatch event. Now this approach is particularly useful when your code has very few dependencies and you simply need to execute it on a periodic basis. For example, removing temporary files every minute or archiving logs every hour. A common use case is a periodic task in order to compute statistics, monitor the status of a system, automates simple daily tasks, or any other computing job that is required to be done on a regular basis. And eventually, these tasks may involve heavy computation that you cannot execute synchronously, i.e. on demand. And as an example, you may want to reindex a search engine, compute weekly aggregation statistics or perhaps perform third-party checks or integrations before renewing a monthly membership for your customers. Most of these scenarios would normally require you to launch, configure and maintain one or more service. With AWS Lambda scheduled invocation, you won't need to worry about managing those servers or provisioning them. You may want to consider migrating your current period up jobs to AWS Lambda, as Lambda can run Node.js, Python, Java, Ruby or Go scripts. So as a software engineer or web developer, this enables you to easily set up and manage your scheduled tasks without having to provision and deal with too much infrastructure, as well as making things easier. Using Lambda functions in this way can simplify logging and monitoring management. And without the machine required to run it, can provide better cost efficiencies. Now you still need to implement your basic logic. And of course you need to make sure your Lambda function can access all of the resources that it needs. So to help you get started with using Lambda to automate tasks, let's create a function to monitor a website homepage every minute and to generate some custom logs based on its content. First of all, let's create a new Lambda function with the correct trigger configuration. Once we're logged into the console, we can select AWS Lambda under Compute and click on the Get Started link. Now we can select a blank function. We don't need to use a blueprint for what we're going to do. We do have to select a trigger that will invoke our function. Now remember the difference between triggers that can invoke Lambda and then services that Lambda can call or be called by. So these triggers are a simple way of setting up invocation events. So here we'll choose the CloudWatch Events Schedule. Now the events schedule trigger is a relatively new addition to Lambda triggers, and it allows you to specify a rule name and description and to choose a timing expression. Now this takes a lot of the heavy lifting out of creating Cron jobs, so it immediately saves you time and effort with scheduled tasks. Let's call this rule monitor website task. You can see here we have a range of timer options which covers most use cases. We want to execute our Lambda function every minute, which is also the smallest interval currently available. Therefore, we can select that rate of one minute. Now you can enter a Cron expression, if you're familiar with the Cron syntax. And we can create our own specific tasks if we want. One thing to keep in mind is that the times are in UTC. So for our example, we use one minute intervals. Make sure you check the enable trigger box so the CloudWatch integration will be activated as soon as we create our new Lambda function. So at this point the configuration required by CloudWatch to invoke your scheduled Lambda function is pretty much ready to go. Now we just need to implement the monitoring logic for our scheduled task. Now I'm going to code this out in Python, but of course you can use any language that's available or supported by Lambda, such as JavaScript, Java, Ruby, Go, et cetera. If it's not a natively supported language, you can import your runtime files. Of course they will have a little bit of an overhead, but basically it's possible. So the logic for our job will be pretty simple. We want to monitor the existence of a given word or sentence on our webpage. So specifically, we'll look for the AWS string on the CloudAcademy homepage each minute using CloudWatch. If the string is there, no action is taken. If the string is missing however, an alert will be triggered and an email notification sent. So we'll define our Lambda handler and return OK. Otherwise, throw something else. And let's record what we have not achieved. Okay, we need to set the role, our execution role. Can create a new one or we can use an existing role, which we'll do here. All right, as far as performance is concerned, allocating more RAM will also ensure better CPU and networking performance. Now since our task needs to load a remote webpage, let's allocate more than the default value. So we'll go for 256 megabytes of RAM to give yourselves a bit more processing speed. Now by doubling the RAM handler, we're going to cut the execution time roughly in half. So therefore, there shouldn't be any cost difference. But of course, that's gonna vary depending on each use case. Now important to note is that we don't need access to our VPC resources, so we don't need to access a custom service or RDS databases or any other resources located in our VPC. We're gonna run this just in our isolated Lambda container. Now if we did want to access other resources, we need to enable the VPC access and also to configure our VPC for public Internet access. CloudWatch will invoke this Lambda function with our special event structure. So in the meantime, we can set our test event, and this is always a really important part of creating a Lambda function. The first time we click Save, we're asked to insert a test event, and that given event will look similar to what we're gonna put here as our test event. I've just mocked this up, give it a time value. Now we should be a little more familiar with the test event by now. And if you recall, we can use the default test events provided by AWS or create our own when we first save our function. And we can always click the Actions, configure test event to revisit this test set up. So after each test, we can inspect the function's output and the corresponding logs in the bottom right of the panel here. Everything looks to be in order, so we'll click create function to finalize the steps. So let's test our event out. Uh-huh, all right, looks like we got a syntax error in here. So we'll go back to our Lambda function code. First off, we've got a line break out of place here. Let's just fixed that up. Silly me, I typed that wrong. And let's test it again. I love fixing errors. Let's just check our log output if there's anything valuable in there. Yep, silly coding error. Better, okay, cool. All right, so that's working as it should be. Right, onwards and upwards. Now that we've got our Lambda function created and connected to our CloudWatch event, let's monitor its execution. So now let's see how we monitor our function statistics and how we can be notified whenever something goes wrong. Now in our simple scenario, we'd like to receive an alarm if the Lambda logic fails, meaning whenever AWS is not found on the CloudAcademy page. So first of all, let's select the Monitoring tab on our Lambda function. And you can see here we've got four key metrics: the invocation count, our invocation duration, our invocation errors, and we've got one recorded there already from our bug; and we got out throttled indications. So the last two graphs should be straight lines if you haven't had the same debugging issue that I had. So ideally, those two metrics should remain zeroes. In order to understand what's going on, we can always inspect our CloudWatch logs. So each Lambda function is associated with a log group, which is gonna contain lots of log streams. And Lambda is natively integrated with CloudWatch. So every indication will be automatically logged, including all your function prints, which is great, especially when you're debugging. Every invocation will correspond to a series of logs like the following. We'll have a start, a request ID, X, version Y, we'll have our N request ID, a report request ID, the duration, the build duration, memory size, maximum memory used and the actual memory built. So between the start and end, we're gonna find all the print statements logged by our function. In our case, we're printing the received event as found or not found. Or found was it now? I think we put it as an error. Also remember that the output of your functions are not going to be logged, right? So therefore, we need to make sure that the log contains all the information needed for debugging purposes. Just to clarify that the output of our function won't be logged, just the error code that we've, so how do we monitor Lambda errors? Now each time our Lambda function logic fails, we'll raise an exception. And that's going to generate a new invocation error. And in this case, we'd like to receive a warning. We'd like to receive, so let's see how we can configure CloudWatch to receive an email every time our Lambda function fails. By clicking on the invocation errors chart, we'll land on the corresponding CloudWatch metric page. Now here we can explore the metrics history and create new custom alarms based on the state. In the bottom right hand corner here, if we click on this, we can create an alarm. Now we can see here that the... So we're shown the alarm configuration pop-up, and this is where we can name our new alarm. We can tune its threshold and decide who will be receiving the warning. So let's enter our name. We'll give it MonitorWebsiteAlarms as a name. We wanna receive it whenever the error metric is bigger or equal to one and for consecutive periods of one minute. And a course we can play with these rules depending on what sort of tolerance we wanna have for timeouts or errors. In the Action section near the bottom, we can create a new notification list. So let's name this one our monitor website email, and we'll insert an email address. We could put a comma-separated list of emails in here. We certainly need to make sure that we're gonna verify at least one of these for the trigger to actually work. So we'll click create alarm to confirm it. Now we have to confirm the email address. So I've gone back to my email client and I've just clicked the confirmation message that I've received. The confirmation emails will subscribe each address to this corresponding topic. No warning emails will be sent until the email addresses is successfully confirmed. So you do have to confirm that subscription. So now we can test our Lambda event. Now let's mess with our Lambda function to see if a fail condition triggers an email notification. So what we're gonna do I think is we'll change the text we're looking for in our Python code to search for the word zombie. And the website check for zombie should fail on the CloudAcademy home page, and it should generate a notification email for us. So when we do get an error, let's just check this. Remember, the CloudWatch event trigger is going to occur every minute. So I'll just segue through time a little here and bingo, here we go. We've had an alarm event, and the email I've just received as a notification outlines the detail you'll see here. So test invocations are also failed, and they generate an invocation, sorry, they generate an email, and the log output will change to not found as it should do. Now in our case, we're assuming that the only possible source of errors is the exception. So in the case of missing text on the CloudAcademy homepage, we'll throw an exception. And if we had more complex scenarios, understanding the cause of a failure can be a little bit harder, and we may want to create custom CloudWatch metrics and/or use logs to record as much information as possible in order to help us with the debugging effort. So on our monitoring tab now, we can see there's several logged errors, which is showing it that the function is working as expected. Okay, so that concludes our scheduled event monitoring lecture. Let's summarize what we have been over. First we created a Lambda function using the CloudWatch trigger. Next we added a schedule routing with a simple Python script. Then we created an alarm to notify us if the event we defined occurred on the page we were monitoring. So now we have an understanding of how to create and run scheduled events with Lambda functions. Thank you for your attention, and any questions or feedback, please reach out to us

About the Author
Andrew Larkin
Head of Content
Learning Paths

Andrew is fanatical about helping business teams gain the maximum ROI possible from adopting, using, and optimizing Public Cloud Services. Having built  70+ Cloud Academy courses, Andrew has helped over 50,000 students master cloud computing by sharing the skills and experiences he gained during 20+  years leading digital teams in code and consulting. Before joining Cloud Academy, Andrew worked for AWS and for AWS technology partners Ooyala and Adobe.