Off-by-none: Issue #28

Live from ServerlessDays Boston…

Welcome to Issue #28 of Off-by-none. Thank you so much for being a part of the community! 🙌

Last week we looked at Lyft’s AWS bill and what it means for multi-cloud. This week we’re live at ServerlessDays Boston, plus we’ve got plenty of great stories and content from the community.

Lots of serverless stuff to get to, so here we go! 🚀

Charity Majors giving the opening keynote at ServerlessDays Boston

Serverless Product Announcements 📢

Meet the Adobe I/O Team: Mihai Corlan on Building Adobe’s Serverless Platform
Adobe’s developer tools is introducing Adobe I/O Runtime, a serverless platform built on top of OpenWhisk. You likely won’t build all your apps there, but this is an interesting way to interact with your Adobe data from collocated execution environments. Props for not naming them “functions.” 👍

Easy Observability and Monitoring with Lambda Layers
The team over at IOpipe put together a handy guide that shows you three ways to add their service using Lambda Layers. While the guide is about their service, you can apply these to any Layer.

Manage table resources in serverless
Chris Feist created a new plugin for the Serverless Framework that reduces the amount of boilerplate needed to create DynamoDB tables.

SQL order from API chaos
A new “API composition platform” called Transposit just recently appeared. “Transposit is a zero-ops platform that brings the power of a relational database to the API ecosystem. Our relational engine provides the ability to write SQL and JavaScript to query and transform your data as though each data connection were a virtual table in a single relational database.” Hmm. 🤔

Serverless Use Cases 🗺

Dyson Fan Control over MQTT via Serverless
Here’s an interesting use case. Nathan Glover set up a couple of Lambda functions to control a Dyson Fan by using the public MQTT endpoint.

Build a serverless data pipeline with AWS S3 Lamba and DynamoDB
More of a how-to, but goes to show an interesting use case for putting dependencies into a separate layer.

Async APIs
Richard Boyd’s post addresses a common use case for long-running or throttled asynchronous events.

If you’re new to Serverless… 🐣

What can serverless do for Node.js developers?
If you’re a Node.js developer and want to know how you can up your game with serverless, this short article will give you a good primer.

Best Practices for Serverless Development
Here’s another best practices post that will give you some good ideas whether you’re new to serverless or a hardened veteran.

How to Get Started With Serverless, Express and AWS Lambda
If you are thinking about migrating your Express app to a monolithic Lambda function, this post’s for you. While not a best practice, this is a common use case for those just getting started. I’ll allow it (for now). 😉

Serverless FAQ
Have questions about serverless? Know someone that does? This short post by the Serverless Gurus answers some of the common questions that our newbie friends might have.

Deploy your existing Nodejs APIs in serverless 
Another post about using the Serverless Framework to migrate your Node.js workload to Lambda. Some helpful setup tips in here as well.

Create a Highly Scalable Image Processing Service on AWS Lambda and API Gateway in 10 Minutes
This is a detailed walkthrough of how to use the AWS console to setup an image processing service with Python. This is a good post for the beginner because it walks you through all the configurations by hand. But please, learn how to write this as IaC before you go into production.

Serverless Tutorials 👷‍♂️

Everything you ever wanted to know about the Amazon DynamoDB console but were afraid to ask: A detailed walkthrough
If you’ve been using DynamoDB for awhile, you’ve likely been using CloudFormation to build your tables. This is a good post to show us how powerful the console is when you want to explore your data and configurations.

How to Configure and Connect to Serverless MySQL Database
For many a developer, you will need to pry RDBMS from their cold dead hands. That’s okay. I still use MySQL with some of my serverless applications (yes, it is possible). This is a good tutorial to show you how to get a cluster configured and connect from an execution environment.

Dynamize your resources’ parameters with the Serverless Framework!
This is a helpful post that shows you how to inject parameters into your serverless application configurations. This is extremely useful for managing variables between stages.

How to set up AWS accounts for multiple product environments
This is an extremely common use case, and if you’re not in the business of setting up nested AWS accounts all the time, it’s easy to get confused. Paul Swail has this nice write up that should save you some time.

Creating an AWS DocumentDB Cluster
The Serverless Gurus have been busy this past week. Here is another great post that will walk you through setting up a new AWS DocumentDB Cluster. Whether you should or not is another question.

Sharing Lambda Layers and Restricting Your Own Usage
Zac Charles has a great post that gives you some of the ins-and-outs of sharing Lambda Layers. If you’re planning on publishing your own, this is a useful guide to make sure you get the permissions right.

Securing APIs in Serverless (AWS Lambda)
A detailed post that shows you how to secure your API Gateways using Amazon Cognito. Not always as straightforward as you might think.

Tutorial: Setting up a private subnet on AWS
If you need to use VPCs with your serverless components, use this guide to configure your environments correctly.

Serverless Stories 📖

How Shamrock transacts billions of dollars with Serverless Framework Enterprise
Great story about Shamrock moving from containers to Lambda using the Serverless Framework. They are apparently using a “multi-cloud” strategy, so let’s see how that plays out.

ETL (Extract, Transform, Load) in Insights
Shiyang Fei discusses the decision making process that Compass used to choose between Apache AirFlow and AWS Step Functions. Lots of pros and cons outlined in here, and spoiler alert: they chose Step Functions.

Reflections on Serverless:From SOA to Serverless part 2
An interesting discussion from Diego Pacheco describing the architecture evolution from SOA to Serverless. Part 2 focuses on strategy evolution and architecture strategy.

Serverless Reads 👓

What AWS Lambda Users Should Know About Azure Functions, and Vice Versa
I’m a huge AWS fan, but it’s good for us serverless developers to see what other cloud providers are offering. Mahdi Azarboon gives us a breakdown of the differences between Azure Functions and Lambda.

Understanding the scaling behaviour of DynamoDB OnDemand tables
Yan Cui gives us another excellent post that takes a deep dive into DynamoDB on-demand tables. He’s got everything you need to know about costs, auto-scaling, and pre-warming to handle massive throughput.

Lambda Concurrency Limits and SQS Triggers Don’t Mix Well (Sometimes)
Another great post by Zac Charles that highlights some of the issues around SQS Triggers with Lambda functions. The ability to consume SQS with Lambda was a welcome addition, but you need to be a bit prescriptive when configuring your concurrency and redrive policy settings.

A Serverless 2.0 point of view
Alain Rouen says the evolution in the serverless movement is arriving and explains why it is important to understand it and how it will change the way we build apps.

3 Tips to Control the Cost of AWS Lambda
Emrah Samdan from Thundra gives us three excellent tips to help you reduce your Lambda costs.

Can you use database pooling with serverless?
Alastair Taft asks this question and gives us five possible solutions for using database pooling with Postgres and AWS Lambda. Luckily for us, AWS is working on better solutions for this problem.

The main stories from QCon London ’19
QCon London was last week, and Alex Wauters gives us a good roundup of the event. There is an interesting discussion about serverless in here.

Introduction to Monitoring Serverless Applications
A good read that will give you the basics.

When you’re wondering what AWS has been up to… 🛠

AWS Amplify Console supports instant cache invalidation and delta deployments on every code commit
Great new feature that will automatically invalidate CloudFront caches when you publish new code. If you’ve ever needed to do a manual cache invalidation, you’ll appreciate this.

Automate Releases to the AWS Serverless Application Repository using AWS CodePipeline
Eliminate the need to use the SDK or console to publish changes to your SAR apps. No more extra code needed. 👍

AWS Step Functions Adds Tag-Based Permissions
Control access based on tags using AWS Identity and Access Management. Very cool.

New Amazon SNS Console Now Available
Updated user interface in the new Amazon SNS Console is optimized for screens of all sizes, making configuration, management, and monitoring more accessible on a variety of devices.

Upcoming Serverless Events 🗓

Serverless Application Troubleshooting
This webinar, tomorrow (March 13th), features Erez Berkner talking about gathering the right data and improving your team’s velocity with distributed serverless visibility and monitoring.

AWS Lambda Security: Inside & Out
Mike Deck, Principal Solutions Architect at AWS, and Ory Segal, CTO & co-founder at PureSec, are running a webinar covering deep topics in serverless and AWS Lambda security. Happening on April 3, 2019.

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Zac Charles (@zaccharles). Zac is Lead Engineer at JUST EAT and has been quite prolific lately with his serverless blog posts. He shares a lot of his work with C# and .NET on AWS Lambda and has a number of open source projects including LambdaNative and LambdaRemoteDebug. He’s also an active voice on Twitter. Keep the great content coming, Zac. The community appreciates it! 🙌

Final Thoughts 🤔

ServerlessDays Boston is in full swing and there have already been several amazing talks, with many more to go. If you’re here, come say “HI!” and make sure you grab some “Off-by-none” stickers. Looking forward to writing a few follow up posts on what we learned today. Just need to decompress first.

I hope you enjoyed this issue of Off-by-none. Please feel free to send feedback and suggestions so I can keep making this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or how you’d like to contribute to Off-by-none.

And please do me the honor of sharing this newsletter with your friends and coworkers who might be interested in serverless. It would be greatly appreciated. 👍

Take care,
Jeremy

Aurora Serverless Data API: An (updated) First Look

Update June 5, 2019: The Data API team has released another update that adds improvements to the JSON serialization of the responses. Any unused type fields will be removed, which makes the response size 80+% smaller.

Update June 4, 2019: After playing around with the updated Data API, I found myself writing a few wrappers to handle parameter formation, transaction management, and response formatting. I ended up writing a full-blown client library for it. I call it the “Data API Client“, and it’s available now on GitHub and NPM.

Update May 31, 2019: AWS has released an updated version of the Data API (see here). There have been a number of improvements (especially to the speed, security, and transaction handling). I’ve updated this post to reflect the new changes/improvements.

On Tuesday, November 20, 2018, AWS announced the release of the new Aurora Serverless Data API. This has been a long awaited feature and has been at the top of many a person’s #awswishlist. As you can imagine, there was quite a bit of fanfare over this on Twitter.

Obviously, I too was excited. The prospect of not needing to use VPCs with Lambda functions to access an RDS database is pretty compelling. Think about all those cold start savings. Plus, connection management with serverless and RDBMS has been quite tricky. I even wrote an NPM package to help deal with the max_connections issue and the inevitable zombies 🧟‍♂️ roaming around your RDS cluster. So AWS’s RDS via HTTP seems like the perfect solution, right? Well, not so fast. 😞 (Update May 31, 2019: There have been a ton of improvements, so read the full post.)

Continue Reading…

Off-by-none: Issue #11

After this, there is no turning back

Welcome to Issue #11 of Off-by-none. I’m happy that you’re here! 🙌

Last week we recapped ServerlessNYC and talked quite a bit about serverless adoption. This week we’re going to point out some more resources for those getting started, as well as offer up plenty of options if you’re looking to take the red pill and go down the serverless rabbit hole. 🐇

Here we go! 🕺

What to read when you want to amp up your serverless knowledge… 🔈

Danilo Poccia has written a free ebook, Agile Development for Serverless Platforms. This book is over 100 pages and has a great section on architectural patterns. There is plenty to learn from this free resource and it is well worth a look. 📖

The team over at Financial Engines wrote a guide to help us with managing disaster recovery with DynamoDB. AWS DynamoDB: Backup and Restore Strategies looks at both Point-in-Time Recovery and On-Demand Backups. Lots of useful information here including configuration and pricing. 👨🏻‍💻

Finally, Thundra published a great piece that shows us how to Debug AWS Lambda Node.js Functions in Production Without Code Change. I really like the idea of automated instrumentation as it cuts down the burden on developers and keeps your code a bit cleaner. It can also ensure we don’t lock ourselves in to a specific software vendor. 📈

When you want to get started with serverless… 🏋️‍♂️

There have been a lot of new “Getting Started with Serverless” posts this week. I really like that more people are starting to create this type of content. The more that’s out there, the more likely someone is to come across it and get to that serverless “aha” moment. If you’re new to serverless, here are a few posts to get you started:

And don’t forget that the #NoServerNovember Challenge (hosted by Serverless, Inc.) is still going on. These challenges will give you something interesting to work on and let you go beyond the standard “Hello World” tutorial.

When you’re not ready to give up RDBS with serverless… 🤓

In our inaugural issue we introduced the serverless-mysql package with my Managing MySQL at Serverless Scale post. David Zhang (@Zigzhang) has taken this even further and created a five part series to help others get started. In his first post, Serverless & RDBS (Part 1) — Set up AWS RDS Aurora and Lambda with serverless, David lays out some background, then gives you full examples to get you up and running.

He’s also published Part 2 (Set up EC2 instance to securely connect to your Aurora DB) and Part 3 (Set up database migrations with umzug) with the final two parts (Set up continuous deployment to migrate database with CircleCI and Set up local development environment with serverless-offline and Docker) coming soon. These are sure to be helpful guides for anyone looking to build serverless apps with RDBS backends.

Of course, re:Invent is right around the corner, so let’s hope we get HTTP endpoints for RDS! 😬

When you feel like there are a lot of conferences… ✈️

Speaking of re:Invent, it is less than two weeks away! 🎉 This is the first year that I’m attending so I’ve been looking for tips like this and this. I’m excited for some of the sessions I’m attending and will be at several events as well. If we haven’t connected already, please contact me so we can meet up.

In other conference news, Serverless Computing London is happening right now and it is chockfull of great speakers. Follow their Twitter feed to see some snippets from the event. Some of the slide decks have been posted as well, so check those out. I was looking at Timirah James’ Function Composition in a Serverless World talk, good stuff. Hopefully we’ll see the videos posted soon. ⚡️

Also, ServerlessDays BOSTON finally has a date! The event is scheduled for March 12, 2019 at the Microsoft New England Research & Development Center. More information about our call for papers and sponsorship opportunities is coming soon. 🎊

When you realize that AWS has no plans to slow down their serverless innovations… 🚀

AWS has released several new features recently that could have a profound impact on our serverless applications. Some of these are pretty exciting. Now just imagine what they are going to announce at re:Invent! Here are just a few of the recent updates:

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Alex Casalboni (@alex_casalboni). Alex is an AWS Technical Evangelist, Serverless champion, co-organizer of ServerlessDays Milan and the serverless meetup there, contributor to serverless open source projects, and a regular conference speaker spreading the serverless gospel. He also helps coordinate ServerlessDays conferences around the word, including helping me and the Boston team. Thanks for all you do, Alex!

Final Thoughts 🤔

As much as I still worry that serverless adoption will be slower than I had hoped, the amount of innovation and new faces in the community is really encouraging. I’m already aware of a few announcements planned for re:Invent, but I also know that there will be a ton more. Other cloud providers are also pushing serverless innovations, and I expect Google and Azure to be announcing new things soon as well.

Serverless still has a long way to go, but all of these new tools, platforms, cloud provider features, conferences, and enthusiasm from the community, is helping to expose this paradigm to a much larger audience. I’m going to continue to write and promote it as much as I can, because there is little doubt in my mind that this is the future of application development.

I hope you’ve enjoyed this issue of Off-by-none. Feedback and suggestions are always welcome and appreciated. It helps me make this newsletter better each week. Please feel free to contact me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, and if you’d like to contribute to Off-by-none.

Go build some great serverless apps and spread the word. 📣

See you next week,
Jeremy

P.S. If you liked this newsletter, please share with your friends and coworkers. I’d really appreciate it! 😉

🚀 Project Update:

Serverless MySQL: v1.1.0 Released

Serverless MySQL v1.1.0 adds additional transaction support capabilities to allow users to retrieve interim query results for use with future queries. This is useful for getting the insertId from previous queries when performing transactions. Read More...

Off-by-none: Issue #1

The awaiting is the hardest part

Welcome to the inaugural issue of Off-by-none! 🎉 I’m so happy that you’re here. 🙌 For those of you that have been email subscribers for a while, this will look a little different. I hope you’ll stick with me; I think this will be interesting. For those of you that are new, welcome, and thank you for joining!

I’m a constant learner, and I’ve found one of the best ways to learn is to take great notes, and then share what you’ve learned with others. This is the goal of Off-by-none. Not to be perfect, not get on a high horse 🐴 and tell you what you should and should not do, but to encourage collaboration and experimentation. Let’s push the limits of serverless. Then let’s take what we’ve discovered, and disseminate it to the masses, so we can learn from each other’s mistakes AND successes.

This newsletter has been awhile in the making. I’ve got all sorts of ideas, but this too will be a constant experiment. Your feedback, ideas, contributions, and encouragement will go a long way to making this what I truly hope it will be. Please reach out to me and share your thoughts and suggestions. They will be greatly appreciated. 🙇‍♂️

Let’s get started!

When you have thousands of concurrents user and limited MySQL connections… 🚀

Lots of people think RDBMS and Serverless don’t mix, and unless you use a series of hacks and tricks, then you’re likely to have scaling problems given the way serverless containers handle concurrent connections. In the past, I’ve often mitigated this issue by increasing the max_connections setting and gracefully handling errors. But now with Aurora Serverless, you can’t change that setting, which means LOTs of Too many connections errors.

I’ll use DynamoDB for the majority of my serverless datastore needs, but it can’t match MySQL’s ability to create normalized data structures, enforce declarative constants, provide referential integrity, and enable ACID-compliant transactions. Sometimes, you just need a relational database. And sometimes, serverless forces us to think about things differently. 🤔

Even though serverless functions are stateless, we can still reuse connections. Not only that, but status commands, like retrieving the number of max_connections and counting threads from the PROCESSLIST, are fairly inexpensive queries. I decided to run some tests to see how efficiently (and effectively) serverless functions could manage their own distributed database connections. Turns out, pretty darn well! 😜

I spent some time and turned all my learnings from the past several years (as well as my new experiments) into a new NPM module called Serverless MySQL. It manages connections for you, cleans up zombies to maintain total connection use thresholds, and even makes working with MySQL connections in Node.js super easy by adding async/await support and simple transaction workflows. I wrote a post about it called Managing MySQL at Serverless Scale that goes into a lot more detail.

You should still size your MySQL instance/cluster appropriately, but for one of my tests, I simulated 500 concurrent users per second with only 90 connections available. Not only did Lambda and MySQL not throw any errors, but the average response time was only 41 ms with the max being just over 3 seconds. Not too shabby. Bump your max_connections up to something more reasonable for that kind of load, and you should never have to gracefully handle connection limit errors again. 🤘🏻

When you’re looking for some light serverless reading… 📚

I’ve had plenty to say about cold starts in the past, but how much do they really matter, and how do they compare across the Big-3 cloud providers (AWS, Azure, and Google Cloud)? Mikhail Shilkov does a pretty darn good job answering those questions in his post Serverless: Cold Start War. Definitely worth the read. ❄️

Erica Windisch start writing again too, and you won’t be disappointed. Lessons from building a Serverless Data Pipeline with AWS Kinesis and Lambda is a relatively short read, but it is loaded with invaluable insights. If you’re thinking about using Kinesis in production, you should probably read this. 🤓

Looking for some serverless best practices? Paul Johnston (yes, the Paul Johnston, legendary cofounder of ServerlessDays), published a post with some “relatively accepted” best practices. There is a lot of debate over these, and Michael Hart and Tyler Love from Bustle jumped in with quite a difference of opinion. Both the article and the Twitter thread are good reads.  I have a feeling that this isn’t the last of this debate. 🍿

Finally, sometimes things are better late than never. Chris Munns, AWS Senior Serverless Developer Advocate, put out a post that contains a quarterly recap of all things AWS Serverless. AWS moves so fast that it’s often hard to keep up with all their innovations. This post outlines all the cool stuff AWS did with Serverless in Q2 2018. ⚡️

When getting through airport security is harder than implementing serverless security… ✈️

I’ve got some travel plans coming up! I’ll be heading to ServerlessDays NYC on October 30th and I will be going to AWS re:Invent this year! I’m hoping to meet a lot of you in person, but I’ll also be taking copious notes and will be sure to share what I learn with all of you. If you’ll be at either event, connect with me via Twitter, LinkedIn, Facebook, or email so we can meet up (and maybe grab a few 🍻).

When you join the “My post made it to the front page of Hacker News and my site crashed” club… 🏅

I’ve been very grateful for all the positive responses to my Serverless Microservice Patterns for AWS post. Lots of people found it very useful and have been using it as a reference, which is great. Someone did me the honor of posting the link to Hacker News, and on Sunday it made its way to #7! Well, my self-hosted WordPress blog running on a single EC2 instance couldn’t handle the traffic, and my site crashed. 💥🤦🏻‍♂️

But it turns out that I’m in good company. Yan Cui had the same thing happen to him earlier this year. He decided to make his blog serverless by using Shifter, a service that converts your WordPress blog into a static, serverless site. I really like this idea, but at the same time, I’m also really sick of WordPress in general. I’ve actually been thinking about this for quite some time and should have more thoughts to share soon.

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Slobodan Stojanović (@slobodan_). Not only did he write the book on Serverless Applications with Node.js, but he also has a number of great serverless posts (like here and here). But the reason I chose Slobodan this week is because of a recent talk he gave at Frontend Conference in Zurich. I wasn’t able to be there in person, but when I saw his slides, I was inspired. Convincing people to adopt serverless is a bit of a passion of mine, and the fact that Slobodan was introducing serverless to an entirely new audience (frontend developers) is a positive step towards more adoption. And, I’m also super jealous by how good his artwork is!

Final Thoughts

I hope you enjoyed this first issue of Off-by-none. It wouldn’t be possible without you and the contributions of a vibrant Serverless community. Please send me your feedback so I can continue to make this newsletter better each week. Reach out via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or even how you’d like to contribute to Off-by-none.

Thanks again! And I hope to see you again next week! 🤞

Until then,
Jeremy

Aurora Serverless: The Good, the Bad and the Scalable

Amazon announced the General Availability of Aurora Serverless on August 9, 2018. I have been playing around with the preview of Aurora Serverless for a few months, and I must say that overall, I’m very impressed. There are A LOT of limitations with this first release, but I believe that Amazon will do what Amazon does best, and keep iterating until this thing is rock solid.

The announcement gives a great overview and the official User Guide is chock full of interesting and useful information, so I definitely suggest giving those a read. In this post, I want to dive a little bit deeper and discuss the pros and cons of Aurora Serverless. I also want to dig into some of the technical details, pricing comparisons, and look more closely at the limitations.

Audio Version

Continue Reading…

How To: Manage RDS Connections from AWS Lambda Serverless Functions

Someone asked a great question on my How To: Reuse Database Connections in AWS Lambda post about how to end the unused connections left over by expired Lambda functions:

I’m playing around with AWS lambda and connections to an RDS database and am finding that for the containers that are not reused the connection remains. I found before that sometimes the connections would just die eventually. I was wondering, is there some way to manage and/or end the connections without needing to wait for them to end on their own? The main issue I’m worried about is that these unused connections would remain for an excessive amount of time and prevent new connections that will actually be used from being made due to the limit on the number of connections.

🧟‍♂️ Zombie RDS connections leftover on container expiration can become a problem when you start to reach a high number of concurrent Lambda executions. My guess is that this is why AWS is launching Aurora Serverless, to deal with relational databases at scale. At the time of this writing it is still in preview mode.

Update September 2, 2018: I wrote an NPM module that manages MySQL connections for you in serverless environments. Check it out here.

Update August 9, 2018: Aurora Serverless is now Generally Available!

Overall, I’ve found that Lambda is pretty good about closing database connections when the container expires, but even if it does it reliably, it still doesn’t solve the MAX CONNECTIONS problem. Here are several strategies that I’ve used to deal with this issue.

Continue Reading…

How To: Normalize URLs Stored in MySQL

I came across an interesting problem the other day. As part of our URL normalization strategy at AlertMe, we have been adding a trailing slash to URLs without file extensions. We did a lot of research when deciding on this tactic and the general consensus around the web was to use trailing slashes for directories and (obviously) no slashes on filenames. See this article from the official Google Webmasters blog: https://webmasters.googleblog.com/2010/04/to-slash-or-not-to-slash.html (I know it’s old, but the concept is still relevant).

We even tested a number of publisher URLs to see what their redirection strategies were. Every one we tested responded correctly to both the slash and no-slash versions of the URL. Some redirected to a trailing slash, some redirected to no trailing slash, but they all returned (or redirected to) the intended page.

Continue Reading…

How To: Reuse Database Connections in AWS Lambda

Update 9/2/2018: I wrote an NPM module that manages MySQL connections for you in serverless environments. Check it out here.

I work with AWS Lambda quite a bit. The ability to use this Functions-as-a-Service (FaaS) has dramatically reduced the complexity and hardware needs of the apps I work on. This is what’s known as a “Serverless” architecture since we do not need to provision any servers in order to run these functions. FaaS is great for a number of use cases (like processing images) because it will scale immediately and near infinitely when there are spikes in traffic. There’s no longer the need to run several underutilized processing servers just waiting for someone to request a large job.

AWS Lambda is event-driven, so it’s also possible to have it respond to API requests through AWS’s API Gateway. However, since Lambda is stateless, you’ll most likely need to query a persistent datastore in order for it to do anything exciting. Setting up a new database connection is relatively expensive. In my experience it typically takes more than 200ms. If we have to reconnect to the database every time we run our Lambda functions (especially if we’re responding to an API request) then we are already adding over 200ms to the total response time. Add that to your queries and whatever additional processing you need to perform and it becomes unusable under normal circumstance. Luckily, Lambda lets us “freeze” and then “thaw” these types of connections.

Update 4/5/2018: After running some new tests, it appears that “warm” functions now average anywhere between 4 and 20ms to connect to RDS instances in the same VPC. Cold starts still average greater than 100ms. Lambda does handle setting up DB connections really well under heavy load, but I still favor connection reuse as it cuts several milliseconds off your execution time.

Continue Reading…