Q&A on Cloud Migration, Security in the Cloud, and more with Andrew Larkin, AWS Expert
In this Q&A, I had the chance to sit down with Cloud Academy’s Content Lead for AWS, Andrew Larkin. Andrew has created many of our AWS learning...Learn More
Today, we’ll be talking about serverless technology with the CEO and founder of Serverless, Austen Collins. We will be talking about why serverless is hot right now, how it works with AWS Lambda and the Serverless Framework.
‘There is a lot of confusion around this term, so I will try to provide some clarity here.
“Serverless” means that, while the server does actually exist, the developer does not have to think about it. A serverless architecture does contain servers, but again, the developer doesn’t have to worry about them.
The name “serverless” is about as technically accurate as the Cloud, is technically accurate. If you’re a developer, this means that you don’t have to worry about operations as much. It frees up a lot of your time so that you can focus on delivering logic, delivering solutions, making things, building things, not maintaining things.
We decided to name our company Serverless before the concept was really a big deal—we didn’t really know how it was going to turn out. Now, we’re Serverless, Inc. We make the Serverless Framework and there is a concept out there also called the Serverless Architecture.’
In a previous webinar, Alex Casalboni – Cloud Academy Cloud Evangelist and Austen Collins explore an exciting new trend that’s redefining the cloud computing industry in this in-depth webinar designed to teach you the basics of serverless computing and design.
‘Serverless abstracts cloud computing at a higher level than the previous generations of cloud computing products. Serverless is kind of the gold standard for a cloud computing abstraction. It is basically just code. All you have to do is upload your code and the provider will run it for you. You don’t have to worry about maintaining high availability, you don’t have to worry about scaling the provider. Your cloud computing provider handles all that for you.
Another way to think about it is that serverless is just the newer, better platform as a service (PaaS). The tricks of serverless, the qualities of what we know as serverless today, all came from this new compute service called AWS Lambda.’
‘AWS Lambda was created and released in 2014 and it’s basically a combination of fantastic ideas. As a developer and engineer, I saw Lambda as the compute service I’d been waiting for because it’s a combination of these great ideas.
Microservice: AWS Lamba is micro-service based. When you’re uploading an AWS Lambda function, you write just enough code to perform one job and then you zip that up and you upload it as an AWS Lambda function. It is an independent unit of deployment, which is inherently a microservice. When you’re building entire applications on AWS Lambda you are just zipping up your separate jobs and uploading them as separate Lambda functions, separate units of deployments. This is inherently a micro-service architecture.
When you’re running a big production system, all of the components in your application are basically separate and isolated. So, when something goes wrong, the blast radius is limited to that micro-service and you can go in at any time and update or modify a simple Lambda function or add new laminate functions without affecting your system as a whole.
Zero administration: Next up, the quality of serverless is most closely associated with zero administration. The zero administration nature of AWS Lambda is the big driver of this term. When you upload an AWS Lambda function, Amazon worries about scaling it out and keeping it up, and the developer doesn’t have to. The AWS Lambda functions will scale to meet any load.
All you have to do is tell Amazon to increase your concurrency limit from time to time.
Event-driven: AWS Lambda is event-driven. This is another cool quality of serverless and it is something that I think we just beginning to see. Basically, these AWS Lambda functions will respond to lower level infrastructure events on your Amazon account. For example, when something is saved to a database or uploaded to storage like S3, these Lambda functions will automatically listen for specific events on those types of infrastructure services and run in response to anything that happens on them. Any state change, basically. You don’t have to manage a separate system for handling the list or logic. AWS handles that for you, which is fantastic. There is a lot that you can do with Lambda just thanks to its event-driven nature.
Easy access to managed services: Also, another big benefit that people don’t talk about as much for AWS Lambda is easy to access to other managed services. Half the value of Lambda, I argue, is all of the great services that Amazon provides that are so easy to work with inside of Lambda. It’s not just compute that you’re getting, you’re also able to easily use other higher level services from Amazon like DynamoDB and S3. We see a lot of people offering Lambda functions with the Serverless Framework, and about 80% of them are using AWS services. The fact that you have this great ecosystem of higher level services available to you is really great.
Pay for execution: This is a very innovative, efficient pricing model from Amazon. It basically comes down to the fact that, if your Lambda function never runs, you won’t get charged. You are only charged when your Lambda function runs and for how long it runs. You can upload as many Lambda functions as you want today and as long as they never run, you’ll never receive a single charge. There is basically no such thing as under-utilization with AWS Lambda. You’re not incurring those costs.’
‘The way I describe it best, the big benefit of serverless is that it offers the lowest total cost of ownership. If you’re trying to justify embracing a serverless architecture, migrating some jobs over to AWS Lambda to your boss, to your company, to your manager, this is really what you need to tell them. You have a compute service that does not take up a lot of time to operate and it doesn’t incur any costs until it runs. It is micro service-based, which really helps you remain agile.
This is why some are embracing serverless architecture so quickly. Both start ups and large enterprise firms alike want this specific benefit. They want to be able to deliver features and software faster than ever without having to worry about maintaining it.
With AWS Lambda, the thing that excited me from day one was the idea that it would increase my productivity and potential as a developer. You don’t have to worry about paying for these functions until they run, and you don’t have to worry about operating them. You have essentially flattened the last couple of barriers between the developer and being able to provision more logic, more solutions, more results than ever before. That’s really exciting. A lot of the energy that was previously allocated to operating and maintaining systems is liberated and what you get is a huge increase in productivity and potential. This is what excites me as a developer because I like building things.’
In this interview, Alex Casalboni and Austen Collins discuss the long-term vision of Serverless and the Serverless Framework.
‘The first thing you’ll notice when you actually want to build something large on AWS Lambda (and it’s probably the same for Azure functions and Google Cloud functions) is that what you’re actually making is a really large distributing system. This is hard to manage without help.
I had this realization on day one. I said I wanted to refactor a project to run exclusively on AWS Lambda. I provisioned my first Lambda function via the AWS console. It wasn’t that difficult, but then I wanted to convert an entire application to run on Lambda and I quickly realized, “Wow, this is going to take at least 100 Lambda functions.” One hundred separate, independent units of deployment and are a lot of small pieces to manage.
Furthermore, you have to manage the infrastructure dependencies across all of the AWS Lambda functions. For example, if the Lambda functions need access to a DynamoDB table or to an S3 bucket, you have to manage permissions across all of them. And, you have to replicate all of these functions across different stages or environments so that you have a dev and staging or a test stage and a production stage. Doing all of this stuff manually without a tool to help is very, very challenging. I immediately realized that this is the perfect problem for framework to solve.
Basically, the CLI is designed and managed for the lifecycle of your serverless application. That is, it’s going to create a scaffolding for you and it’s going to help you through your first deployment. It will help you through roll backs. It will help you test your code and run that code locally. Ultimately it will remove it whenever you’re done, if you’re ever done. Again, these functions don’t charge you until they run. It’s easy just to provision infinite amounts of logic and let it sit there. The Serverless Framework helps you handle this entire lifecycle of managing a lot of AWS Lambda functions and their resource dependencies.’
In a previous Cloud Academy webinar, David Wells, Senior Software Engineer at Serverless, explored some different use cases for Serverless technology with examples from companies and other open source projects.
‘The main purpose of the framework is application lifecycle management. Also, the framework manages code and infrastructure. It’s not just a tool that uploads your code to AWS Lambda, but it also provisions infrastructure for you. It’s a bit different than your average framework in that regard. Additionally, it does scaffolding. It will scaffold a bunch of different templates for you, different types of serverless architectures, and it will perform a ton of useful automations for application lifecycle management.
It supports multiple languages. Though it’s written in Node.js, we seek to support all of the languages that AWS Lambda supports: Node.js, Python, Java, and any other language that Lambda introduces to the future. You could actually deploy Lambda functions in Python, Lambda functions in Java. If you have a team with a variety of experience and languages, they could all use this and work in the language of their choice.
It’s also multi-provider, which is a big effort for us. Developers should be free to deploy functions everywhere. The idea that you could deploy functions to multiple clouds is really appealing because you’re able to take advantage of the higher value services that a lot of these clouds are providing.
For example, a lot of people want to use Google’s machine learning stuff. We’re going to help you deploy a serverless function or a Google Cloud function right on Google so that you can do that. Even if you already have a bunch of AWS Lambda functions. We basically want to provide a uniform experience for deploying functions anywhere. We have a lot of stuff coming out in this regard. It’s going to get pretty exciting to see how people rig up these serverless architectures across clouds and deploy functions everywhere.
Provision management is a huge feature of the framework. Again, a serverless architecture is a massively distributed architecture and it’s both wonderful and a challenge at times, especially from the perspective of permissions. Every single piece needs to have the permissions managed and kept secure. The Serverless Framework practices the principle of least privilege when it comes to permissions. It will automatically set up permissions for your Lambda functions and the infrastructure that they’re interacting with and lock it down so that there aren’t excessive permissions between.
The Serverless Framework offers a great local development story. The great challenge of Lambda and these new serverless services is that they’re cloud-based and hard to emulate locally. This is where the Serverless Framework steps in. It does a lot of local emulation. Some of it is built into the framework, some of it is offered through plugins. It has a grey serverless offline plugin that will create a local server on your machine and emulate all of the API gateways so that you can test out your back end locally. It’s a pretty powerful run by a few fantastic community members. If you really want to build large serverless architectures or just a few Lambda functions, having that local development story is one of the biggest benefits.
Additionally, the Serverless Framework is extensible. I believe that any developer tool should be extensible. As a developer I like to hack, I like to change things. We made a big effort to make the Serverless Framework extensible from day one. Also, the day this went viral and I saw the requests and how people wanted to manage their workloads, I quickly realized how subjective workflows are. We just needed to make the framework customizable. It’s all basically one big plugin architecture. You can write a plugin to replace the functionality of the framework or you can add new functionality.
Finally, we have a fantastic community behind the Serverless Framework. Everyone is super enthusiastic and helpful. No matter what level of experience you have, you’re going to find large enterprise firms in our chat room alongside fine startup people and hobbyists. I’d say about half of our users have at least two years of Amazon experience.’
For more information about the Serverless Framework, including use cases and tips for going serverless, check out the full interview with Austen in our webinar, Serverless Framework and the new AWS Lambda features.
At present, most backends/APIs are dominated by a REST architecture, which allows us to create and access data in a structured manner. This approach has solved a lot of the issues that we previously faced regarding routes, resources, naming conventions, accessibility, etc. But this is 2...
One of the most applauded announcements at re:Invent 2016 was AWS Step Functions. Step Functions is basically an orchestration service for AWS Lambda and activity-based tasks. Thanks to SFN, you can control multiple executions of your processes using Lambda Functions and activity worker...
AWS Lambda is one of the best solutions for managing a data collection pipeline and for implementing a serverless architecture. In this post, we'll discover how to build a serverless data pipeline in three simple steps using AWS Lambda Functions, Kinesis Streams, Amazon Simple Queue Ser...
Fight for Serverless Cloud Domination ContinuesIn this post, we’ll take a close look at Azure Functions, and we’ll extend it to the analysis that my colleague Alex started earlier in a previous post on Google Cloud Functions vs AWS Lambda. Rather than declaring a "winner," we will hig...