
AWS Lambda and AWS API Gateway have made creating serverless APIs extremely easy. Developers can simply create Lambda functions, configure an API Gateway, and start responding to RESTful endpoint calls. While this all seems pretty straightforward on the surface, there are plenty of pitfalls that can make working with these services frustrating.
There are, for example, lots of confusing and conflicting configurations in API Gateway. Managing deployments and resources can be tricky, especially when publishing to multiple stages (e.g. dev, staging, prod, etc.). Even structuring your application code and dependencies can be difficult to wrap your head around when working with multiple functions.
In this post I’m going to show you how to setup and deploy a serverless API using the Serverless framework and Lambda API, a lightweight web framework for your serverless applications using AWS Lambda and API Gateway. We’ll create some sample routes, handle CORS, and discuss managing authentication. Let’s get started.
Requirements
First we need to install the Serverless framework, using our Terminal:
1 |
$ npm install -g serverless |
Next we need to clone the Serverless API Sample project from Github. Navigate to the folder you wish to create the project in and then:
1 |
$ git clone https://github.com/jeremydaly/serverless-api-sample.git |
Navigate to the serverless-api-sample
folder and install our Node dependencies:
1 |
$ npm install |
And that’s it. Now we’re ready to start working on our API.
The serverless.yml file
Let’s open the serverless.yml file. I’ve set this up to be very basic. You can learn more about this file and its options here. I suggest you do that when you get a chance as this is a very powerful tool.
For now, we just need to configure one thing to get started. On line 8 there is a property named profile
. This refers to the name of your local AWS profile. If you only have one account, then it is probably named default
. If you don’t have a local AWS profile set up, you can configure one using this: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
Once your profile is configured, update the <YOUR AWS PROFILE>
with your profile name. Save that file.
A few other things of note…
There are a few more parts of the serverless.yml file that you should know about. The iamRoleStatements
section creates a new role for your Lambda function. Again, I’ve set this up to be basic. All it does is allow logs to be created and for an S3 bucket to be created to store your deployments.
The functions
section is another important piece. As shown below, this creates a function called serverless-api
and names it using the service name and then deployment stage (more on that later). It has a handler
, which specifies which module and which function to route requests to. Finally it attaches an http
event that responds to any
HTTP method that matches a path that starts with v1/
. The {proxy+}
part of the path is API Gateway’s all-encompassing proxy resource that routes any path, no matter how deep, to the specified resource.
1 2 3 4 5 6 7 8 9 10 |
# Functions functions: serverless-api-sample: name: ${self:service}-${self:provider.stage}-serverless-api-sample handler: handler.router timeout: 30 events: - http: path: 'v1/{proxy+}' method: any |
Understanding the handler function
Open the file named handler.js
. This is our handler module that we specified in the serverless.yml function. Let’s skip to line 113 first and look at the following:
1 2 3 |
module.exports.router = (event, context, callback) => { ... } |
Here we are exporting a function call router
(as in handler: handler.router
from serverless.yml). This is the function that will be called when we route API requests to this Lambda function.
Let’s jump back to the top of the script and look at line 12:
1 2 |
// Require and init API router module const app = require('lambda-api')({ version: 'v1.0', base: 'v1' }) |
Here we require lambda-api
and instantiate it. This module allows for a version number to be set and a base. The base
(in this case v1
) is used to preface routes, meaning you don’t need to specify the version in every route you create, just the path. We now have an instance of lambda-api
in our app
variable. Let’s create some routes.
Creating Routes
Lambda API is similar to other Node.js web frameworks like Fastify and Express, so if you’ve used any of those, then this should seem very familiar. Full documentation can be found at https://github.com/jeremydaly/lambda-api.
Creating routes is easy. Call the convenience method for the HTTP method you wish to create the route for and specify the route and a function that receives two arguments. For example, a GET
method for /posts
would look like this:
1 |
app.get('/posts', (req,res) => { }) |
Once we’ve created our route, we can now write our code to respond to the request. This is a normal Javascript function, so you can put your code directly in the function block or pass it off to another module. Just make sure you pass the res
variable. This contains the RESPONSE
object that is needed to return the request to the user.
The RESPONSE
object (https://github.com/jeremydaly/lambda-api#response) allows you to manipulate and send the response. You can set the status with the .status()
method, add headers with the .header()
method, and return the contents of the response with the .send()
method. There are also convenience methods like .json()
and .html()
that will add the correct Content-Type
headers for you.
The methods are all chainable, so you can call:
1 |
res.header('Content-Type','application/json').status(200).send({ status: 'ok' }) |
Or with a convenience method:
1 |
res.status(200).json({ status: 'ok' }) |
You can even respond with an error by calling the .error()
method:
1 |
res.error('This is an error') |
And if you want to set the error code:
1 |
res.status(404).error('This is a 404 error') |
Path parameters and query strings
The REQUEST
object automatically parses path parameters and query strings for you. For example:
1 2 3 4 5 |
app.put('/posts/:post_id', (req,res) => { res.status(200).json({ params: req.params }) }) |
This creates a PUT
route with a post_id
path parameter. If you PUT something to https://<your-api-endpoint>/v1/posts/1234, then the req.params
will contain a javascript object like this:
1 2 3 |
{ "post_id": "1234" } |
You can create as many params as you’d like:
1 2 3 4 5 |
app.put('/posts/:post_id/:foo/:bar', (req,res) => { res.status(200).json({ params: req.params }) }) |
Query strings are automatically parsed into an object and are available using req.query
. Our previous route to https://<your-api-endpoint>/v1/posts/1234/?test=true&foo=bar would return:
1 2 3 4 5 6 7 8 9 |
{ "query": { "test": "true", "foo": "bar" }, "params": { "post_id": "123" } } |
Processing the BODY
Most POST and PUT routes will expect some kind of BODY input. Lambda API handles this for you automatically regardless of what you send in the body. If you post JSON, the module will attempt to parse it into a Javascript object. If it can’t be parsed, it will just include the raw string. If you post FORM variables, Lambda API will parse and decode them as long as you send in a Content-Type: "application/x-www-form-urlencoded"
header. The parsed object is then available via req.body
.
Middleware
Middleware allows you to process the request BEFORE it goes to a specific route. This is useful for things like authentication or CORS. Middleware is defined using the .use()
method and takes a function as its single argument. This function takes three arguments, the REQUEST
and RESPONSE
objects and a next
function. When called, the next
function tells the middleware to move on to the next middleware or to the route if there are no more defined.
If we wanted to perform authorization before every request, we could use:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
// Add Authorization Middleware app.use((req,res,next) => { // Check for Authorization Bearer token if (req.auth.type === 'Bearer') { // Get the Bearer token value let token = req.auth.value // Set the token in the request scope req.token = token // Do some checking here to make sure it is valid (set an auth flag) req.auth = true } // Call next to continue processing next() }) |
Notice that we check the req.auth
parameter. Lambda API will automatically parse several types of authorization schemas and normalize them for you. If this is a “Bearer” token authorization, we can grab the token value using req.auth.value
and then do something with that to confirm that the request is authorized. We may look this up in a database or cache and then flag the request as authorized, or throw an error. The REQUEST
object is writable, so setting req.auth=true
will allow other middleware and routes to access that value. When we are done processing, we call the next()
function to move on.
CORS
If you are writing an API that will be accessed directly from a web browser, then you’ll need to implement Cross-Origin Resource Sharing (CORS). API Gateway has a built-in CORS implementation, but it is a static implementation and requires extra configuration. CORS is nothing more than setting the correct headers when responding to a request from a web browser. Middleware allows you to return the correct CORS headers by simply setting the headers directly or using the res.cors()
convenience method:
1 2 3 4 5 6 7 8 9 |
// Add CORS Middleware app.use((req,res,next) => { // Add default CORS headers for every request res.cors() // Call next to continue processing next() }) |
If you’d like to get even fancier, you can use the referrer information and crosscheck that against a list of approved URLs. Then you could manipulate your Access-Control-Allow-Origin
header to only allow certain domains. You can customize the CORS headers by passing in an options object.
Browsers also require a preflight call to an OPTIONS
method. This checks to see if the route has the proper CORS headers set. The easiest way to do this is to just set an OPTIONS
route with a wildcard. This will create an OPTIONS
method for every route you have defined.
1 2 3 4 |
// Default Options for CORS preflight app.options('/*', (req,res) => { res.status(200).json({}) }) |
Error Handling
Error handling is automatic by default, so calling res.error('Some error occured)
will return a formatted error response. It will also log the error using console.log
so it will be accessible in your Cloudwatch logs. If you’d like to override errors, you can use Lambda API’s Error Handling (https://github.com/jeremydaly/lambda-api#error-handling) feature.
Running our routes
Unlike Fastify or Express.js, Lambda API doesn’t respond directly to HTTP requests on a specific port. Instead, it just accepts the event
passed through the handler function and processes that. Within our router
function we call app.run()
with the event, context and callback passed from the handler:
1 2 |
// Run the request app.run(event,context,callback) |
This will process the event, route it correctly, and return the response.
Testing our API locally
Before we deploy our API, we’re going to want to test the routes locally. In the sample project I’ve included five events that replicate what API Gateway could send to your Lambda function. You can test your functions using these with Serverless’ invoke local
command:
1 2 3 4 5 |
$ sls invoke local -f serverless-api-sample -p test/get_sample.json $ sls invoke local -f serverless-api-sample -p test/post_sample.json $ sls invoke local -f serverless-api-sample -p test/put_sample.json $ sls invoke local -f serverless-api-sample -p test/delete_sample.json $ sls invoke local -f serverless-api-sample -p test/form_sample.json |
The GET
sample will return the following from our sample project:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
{ "headers": { "Content-Type": "application/json", "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Methods": "GET, PUT, POST, DELETE, OPTIONS", "Access-Control-Allow-Headers": "Content-Type, Authorization, Content-Length, X-Requested-With" }, "statusCode": 200, "body": { "status": "ok", "version": "v1.0", "auth": true, "body": null, "query": { "qs1": "q1" } } } |
Deploying our API
Now we actually want people to be able to call our API. We can deploy to Amazon Web Services with a single Serverless command:
1 |
$ sls deploy |
This will deploy your service to the default dev
stage and return something like this:
1 2 3 4 5 6 7 8 9 10 11 12 |
Serverless: Stack update finished... Service Information service: serverless-api stage: dev region: us-east-1 stack: serverless-api-dev api keys: None endpoints: ANY - https://<YOUR-API-ID>.execute-api.us-east-1.amazonaws.com/dev/v1/{proxy+} functions: serverless-api: serverless-api-dev-serverless-api |
And that’s it! Now you can GET
, POST
, PUT
, and DELETE
to https://<YOUR-API-ID>.execute-api.us-east-1.amazonaws.com/dev/v1/posts
Deployment Stages
The Serverless framework is very good at handling deployment stages. Having multiple stages lets you create different versions of your API for testing or other purposes. I’ve configured the sample project to use dev
as the default stage, but you can specify other stages using the -s
option:
1 |
$ sls deploy -s staging |
I’ve also configured the sample project to use the serverless-stage-manager which allows you to configure a list of allowable stages. This is helpful so that you don’t accidentally deploy to the “pord” instead of “prod” stage.
Where do we go from here?
I hope this post gave you the basics needed to create your first (or perhaps a better) serverless API using Lambda and API Gateway. The capabilities are near endless with these powerful services from Amazon Web Services. Combine those with the ease of use of the Serverless framework and Lambda API and you should be able to create some pretty amazing serverless applications.
I would suggest reading up on the Serverless framework at serverless.com.
Also, the documentation for Lambda API can be found on Github: https://github.com/jeremydaly/lambda-api. v0.5 is loaded with features (like binary support, route prefixing, etc.) that cover lots of use cases for your serverless applications.
If you plan on integrating database connections into your Lambda functions, which you will probably need to do at some point, check out How To: Reuse Database Connections in AWS Lambda and How To: Manage RDS Connections from AWS Lambda Serverless Functions.
Another great resource is the Serverless Optimize Plugin. With a few configurations you can optimize your functions so that they only use the necessary dependencies. Check out How To: Optimize the Serverless Optimizer Plugin for more tips. By default, your entire Serverless project is contained in each Lambda function, which doesn’t make a ton of sense. This plugin will fix that and transpile your code if necessary.
Finally, I would suggest reading up on configuring AWS resources via Cloudformation. You can do some pretty amazing things using the resources
section of the serverless.yml file. AWS resources can be deployed per stage which lets you do some very useful things. You most likely don’t want to spin up database instances with it, but it’s great for creating SQS queues, SNS topics, DynamoDB tables and more.
Tags: amazon web services, api, api gateway, aws lambda, lambda-api, nodejs, serverless
Did you like this post? 👍 Do you want more? 🙌 Follow me on Twitter or check out some of the projects I’m working on.
Jeremy, You have created a great framework in Lambda API. The write-up is very useful. May I suggest adding a sentence or two in the beginning of this tutorial stating what the use case is about? ie, what problem are you looking to solve? This will help readers to get an understanding early on what the article is about and help in understanding the rest of the document. Just my two cents.
Thanks for the suggestion, Srini. I will definitely add some more context.
Hi Jeremy,
Thanks for the extremely helpful and educational article! Everything has been smooth up until I try to restrict requests for a specific hostname. I’ve tried adding the origin to res.cors() and app.options() but can still access it from Postman. Postman shows the correct header settings but they’re not being enforced. I’ve also tried adding them directly in the API Gateway settings. Am I missing something?
Thanks!
Hi John,
CORS is a browser-based security construct that prevents cross-origin requests (https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS). Direct API calls (using something like Postman or cURL) ignore these headers.
Hope that helps,
Jeremy
Loving the simplicity and familiarity of your framework Jeremy, great work. thank you.
Please forgive my ignorance I have been messing with serverless only off and on in the evenings for a few months
I guess I don’t understand why you would go this route of calling a framework that routes the urls to different functions instead of having separate functions based on api gateway urls. That seems like it breaks the concept of not having a monolith that needs to be deployed all the time. As well as keeping functional components in their own serverless services.
I could very well be misunderstanding using lambda for web and other such tasks.
Hi Buddy,
Single-purpose functions with dedicated API Gateway routes are typically the preferred way to create serverless APIs. The “Lambda API” framework can be used to process single routes as well. It does all the parsing and response formatting for you, as well as gives you a number of convenience functions to make typical workflows easier. However, there are several circumstances where you might want to have several routes point to the same function. This could be because of CloudFormation resource limits, the need to combine infrequently accessed routes to minimize cold start times, etc.
Hope that makes sense,
Jeremy
Hi friend.
Greate framework. I am start not with Lambda, Lambda-API and TypeScript.
Can you show this example in TypeScript?
Hi Jean,
I don’t have any examples in Typescript, but I will work on putting some together.
Thanks,
Jeremy
Does Lambda-api support Nested Routers?
We wanna separate the routes into nested route js file.
Thanks
Yes, you can use the route prefixing system to created nested routers: https://github.com/jeremydaly/lambda-api#route-prefixing
Hi Jeremy,
Thanks for the very well written tutorial based on your amazing lambda-api framework.
The idea to not have any huge dependencies like express makes my lambda turns from 5 Mb to 70 Kb, pretty amazing.
Thanks a lot !
Keep the good work 🙂
Thanks, Luiz! I’m glad it is helpful.
Hi Jeremy,
Thank you & great framework. Could you please describe how file uploads can be handled in this?
For example, if I need to upload an image to an S3 bucket, how can it be done?
Thanks,
Hi, I encountered your question left unanswered, so although you asked Jeremy I hope he won’t be angry if I put my 2 cents. Usually you don’t upload images to s3 bucket through Lambda function but directly to S3. This is because S3 outperforms Lambda on this field (I/O operations).
What you would do is you would call Lambda to generate a pre-signed URL to upload to S3.
This URL has expiry time which you can manage yourself.
Having this URL you do upload directly to S3.
Here you can have a look at my example of AWS Lambda generating such URL:
https://github.com/piczmar/sls-rest-upload-s3/blob/master/handler.js#L10
and here are some AWS docs: https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObject.html
Hope you find it helpful.
Hello Jeremy, I was wondering here, do you have any examples you can point to for passing the ‘res’ object off to another module? I don’t want to fill up my handler with so many calls and I’ve been scouring the internet for an example but they all seem to put all the calls in the serverless.yml file and create todos for the handler.