Product Guy, Serverless Advocate & Startup Veteran

My name is Jeremy Daly. I appreciate the visit. 👍 I’ve been managing the development of complex web and mobile applications for businesses across the globe for over 20 years. I’m currently the Chief Technology Officer at AlertMe, but I always keep myself busy with several side projects and consulting clients.

I write a lot about serverless and I like to share thoughts and stories about programmingproduct managemententrepreneurship and productivity. Sometimes I’ll write reviews or have random thoughts that I need to get out of my head. I also like to post a how-to article every once in a while. Feel free to jump in to one of the categories above, view all my posts, or view my latest posts below.

If you’d like to get updates, please subscribe to my weekly newsletter, or follow me on TwitterGithub and Facebook.


My Latest Posts:

Serverless Tip: Don’t overpay when waiting on remote API calls

Our serverless applications become a lot more interesting when they interact with third-party APIs like Twilio, SendGrid, Twitter, MailChimp, Stripe, IBM Watson and others. Most of these APIs respond relatively quickly (within a few hundred milliseconds or so), allowing us to include them in the execution of synchronous workflows (like our own API calls).  Sometimes we run these calls asynchronously as background tasks completely disconnected from any type of front end user experience.

Regardless how they’re executed, the Lambda functions calling them need to stay running while they wait for a response. Unfortunately, Step Functions don’t have a way to create HTTP requests and wait for a response. And even if they did, you’d at least have to pay for the cost of the transition, which can get a bit expensive at scale. This may not seem like a big deal on the surface, but depending on your memory configuration, the cost can really start to add up.

In this post we’ll look at the impact of memory configuration on the performance of remote API calls, run a cost analysis, and explore ways to optimize our Lambda functions to minimize cost and execution time when dealing with third-party APIs.

Continue Reading…

re:Capping re:Invent: AWS goes all-in on Serverless

Last week I spent six incredibly exhausting days in Las Vegas at the AWS re:Invent conference. More than 50,000 developers, partners, customers, and cloud enthusiasts came together to experience this annual event that continues to grow year after year. This was my first time attending, and while I wasn’t quite sure what to expect, I left with not just the feeling that I got my money’s worth, but that AWS is doing everything in their power to help customers like me succeed.

There have already been some really good wrap-up posts about the event. Take a look at James Beswick’s What I learned from AWS re:Invent 2018, Paul Swail’s What new use cases do the re:Invent 2018 serverless announcements open up?, and All the Serverless announcements at re:Invent 2018 from the Serverless, Inc. blog. There’s a lot of good analysis in these posts, so rather than simply rehash everything, I figured I touch on a few of the announcements that I think really matter. We’ll get to that in a minute, but first I want to point out a few things about Amazon Web Services that I learned this past week.

Continue Reading…

Aurora Serverless Data API: A First Look

On Tuesday, November 20, 2018, AWS announced the release of the new Aurora Serverless Data API. This has been a long awaited feature and has been at the top of many a person’s #awswishlist. As you can imagine, there was quite a bit of fanfare over this on Twitter.

Obviously, I too was excited. The prospect of not needing to use VPCs with Lambda functions to access an RDS database is pretty compelling. Think about all those cold start savings. Plus, connection management with serverless and RDBMS has been quite tricky. I even wrote an NPM package to help deal with the max_connections issue and the inevitable zombies 🧟‍♂️ roaming around your RDS cluster. So AWS’s RDS via HTTP seems like the perfect solution, right? Well, not so fast. 😞

Continue Reading…

🚀 Project Update:

Lambda API: v0.9.2 Released

Lambda API v0.9.2 has been released and contains additional updates and fixes for the index.d.ts TypeScript declarations file. Thanks again to @hassankhan and @Wintereise for submitting the changes. The release is immediately available via NPM. Read More...
🚀 Project Update:

Lambda API: v0.9.1 Released

Lambda API v0.9.1 has been released to include the index.d.ts TypeScript declarations file in the NPM package (thanks again, @hassankhan). The release is immediately available via NPM. Read More...
🚀 Project Update:

Lambda API: v0.9 Released

v0.9 adds new features to give developers better control over error handling and serialization. A TypeScript declaration file has also been added along with some additional API Gateway inputs that are now available in the REQUEST object. Read More...

Takeaways from ServerlessNYC 2018

I had the opportunity to attend ServerlessNYC this week (a ServerlessDays community conference) and had an absolutely amazing time. The conference was really well-organized (thanks Iguazio), the speakers were great, and I was able to have some very interesting (and enlightening) conversations with many attendees and presenters. In this post I’ve summarized some of the key takeaways from the event as well as provided some of my own thoughts.

Note: There were several talks that were focused on a specific product or service. While I found these talks to be very interesting, I didn’t include them in this post. I tried to cover the topics and lessons that can be applied to serverless in general.

Update November 16, 2018: Some videos have been posted, so I’ve provided the links to them.

Audio Version:

Continue Reading…

What 15 Minute Lambda Functions Tells Us About the Future of Serverless

Amazon Web Services recently announced that they increased the maximum execution time of Lambda functions from 5 to 15 minutes. In addition to this, they also introduced the new “Applications” menu in the Lambda Console, a tool that aggregates functions, resources, event sources and metrics based on services defined by SAM or CloudFormation templates. With AWS re:Invent just around the corner, I’m sure these announcements are just the tip of the iceberg with regards to AWS’s plans for Lambda and its suite of complementary managed services.

While these may seem like incremental improvements to the casual observer, they actually give us an interesting glimpse into the future of serverless computing. Cloud providers, especially AWS, continue to push the limits of what serverless can and should be. In this post, we’ll discuss why these two announcements represent significant progress into serverless becoming the dominant force in cloud computing.

Continue Reading…

🚀 Project Update:

Lambda API: v0.8.1 Released

Lambda API v0.8.1 has been released to patch an issue with middleware responses and a path prefixing options bug. The release is immediately available via NPM. Read More...

An Introduction to Serverless Microservices

Thinking about microservices, especially their communication patterns, can be a bit of a mind-bending experience for developers. The idea of splitting an application into several (if not hundreds of) independent services, can leave even the most experienced developer scratching their head and questioning their choices. Add serverless event-driven architecture into the mix, eliminating the idea of state between invocations, and introducing a new per function concurrency model that supports near limitless scaling, it’s not surprising that many developers find this confusing. 😕 But it doesn’t have to be. 😀

In this post, we’ll outline a few principles of microservices and then discuss how we might implement them using serverless. If you are familiar with microservices and how they communicate, this post should highlight how these patterns are adapted to fit a serverless model. If you’re new to microservices, hopefully you’ll get enough of the basics to start you on your serverless microservices journey. We’ll also touch on the idea of orchestration versus choreography and when one might be a better choice than the other with serverless architectures. I hope you’ll walk away from this realizing both the power of the serverless microservices approach and that the basic fundamentals are actually quite simple.  👊

Audio Version:

Continue Reading…

🚀 Project Update:

Serverless MySQL: v1.1.0 Released

Serverless MySQL v1.1.0 adds additional transaction support capabilities to allow users to retrieve interim query results for use with future queries. This is useful for getting the insertId from previous queries when performing transactions. Read More...

Serverless Security: Locking Down Your Apps with FunctionShield

I’ve written quite extensively about serverless security, and while you don’t need to be an expert on the matter, there are a number of common sense principles that every developer should know. Serverless infrastructures (specifically FaaS and managed services) certainly benefit from an increased security posture given that the cloud provider is handling things like software patching, network security, and to some extent, even DDoS mitigation. But at the end of the day, your application is only as secure as its weakest link, and with serverless, that pretty much always comes down to application layer security.

In this post we’re going to look at ways to mitigate some of these application layer security issues by using some simple strategies as well as a free tool called FunctionShield.

Audio Version:

Continue Reading…

Managing MySQL at Serverless Scale

“What? You can’t use MySQL with serverless functions, you’ll just exhaust all the connections as soon as it starts to scale! And what about zombie connections? Lambda doesn’t clean those up for you, meaning you’ll potentially have hundreds of sleeping threads blocking new connections and throwing errors. It can’t be done!”  ~ Naysayer

I really like DynamoDB and BigTable (even Cosmos DB is pretty cool), and for most of my serverless applications, they would be my first choice as a datastore. But I still have a love for relational databases, especially MySQL. It had always been my goto choice, perfect for building normalized data structures, enforcing declarative constants, providing referential integrity, and enabling ACID-compliant transactions. Plus the elegance of SQL (structured query language) makes organizing, retrieving and updating your data drop dead simple.

But now we have SERVERLESS. And Serverless functions (like AWS Lambda, Google Cloud Functions, and Azure Functions) scale almost infinitely by creating separate instances for each concurrent user. This is a MAJOR PROBLEM for RDBS solutions like MySQL, because available connections can be quickly maxed out by concurrent functions competing for access. Reusing database connections doesn’t help, and even the release of Aurora Serverless doesn’t solve the max_connections problem. Sure there are some tricks we can use to mitigate the problem, but ultimately, using MySQL with serverless is a massive headache.

Well, maybe not anymore. 😀 I’ve been dealing with MySQL scaling issues and serverless functions for years now, and I’ve finally incorporated all of my learning into a simple, easy to use NPM module that (I hope) will solve your Serverless MySQL problems.

Continue Reading…

Jeremy goes to AWS re:Invent 2018

It’s official! I’m going to AWS re:Invent 2018. 🙌

My goal from this trip is to learn, learn, learn… and then share, share, share.   There are over 30 sessions that talk about serverless, plus 40,000 other people there to meet and learn from! I’m so excited. 🙃

I know that many of you will be there, but for those of you who can’t be, I’ll do my best to share insights, tips, how-tos, best practices and more. I’ll even have a drink for you if you’d like 🍺 (no arm twisting necessary)!

Continue Reading…

Serverless Microservice Patterns for AWS

I’m a huge fan of building microservices with serverless systems. Serverless gives us the power to focus on just the code and our data without worrying about the maintenance and configuration of the underlying compute resources. Cloud providers (like AWS), also give us a huge number of managed services that we can stitch together to create incredibly powerful, and massively scalable serverless microservices.

I’ve read a lot of posts that mention serverless microservices, but they often don’t go into much detail. I feel like that can leave people confused and make it harder for them to implement their own solutions. Since I work with serverless microservices all the time, I figured I’d compile a list of design patterns and how to implement them in AWS. I came up with 19 of them, though I’m sure there are plenty more.

In this post we’ll look at all 19 in detail so that you can use them as templates to start designing your own serverless microservices.

Audio Version:

Continue Reading…

🚀 Project Update:

Lambda API: v0.8 Released

Lambda v0.8 is finally here and was well worth the wait! New features include allowing middleware to accept multiple handlers, new convenience methods for cache control and signing S3 URLs, and async/await support for the main function handler. And best of all, new LOGGING and SAMPLING support for you to add more observability into your APIs and web applications. Read More...

Aurora Serverless: The Good, the Bad and the Scalable

Amazon announced the General Availability of Aurora Serverless on August 9, 2018. I have been playing around with the preview of Aurora Serverless for a few months, and I must say that overall, I’m very impressed. There are A LOT of limitations with this first release, but I believe that Amazon will do what Amazon does best, and keep iterating until this thing is rock solid.

The announcement gives a great overview and the official User Guide is chock full of interesting and useful information, so I definitely suggest giving those a read. In this post, I want to dive a little bit deeper and discuss the pros and cons of Aurora Serverless. I also want to dig into some of the technical details, pricing comparisons, and look more closely at the limitations.

Audio Version

Continue Reading…

A Tale of Two Teams

Audio Version:

It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness… ~ A Tale of Two Cities by Charles Dickens

There is a revolution happening in the tech world. An emerging paradigm that’s letting development teams focus on business value instead of technical orchestration. It is helping teams create and iterate faster, without worrying about the limits or configurations of an underlying infrastructure. It is enabling the emergence of new tools and services that foster greater developer freedom. Freedom to experiment. Freedom to do more with less. Freedom to immediately create value by publishing their work without the traditional barriers created by operational limits.

Continue Reading…

How To: Add Test Coverage to your Serverless Applications

Writing serverless functions brings developers closer and closer to the stack that runs their code. While this gives them a tremendous amount of freedom, it also adds additional responsibility. Serverless applications require developers to think more about security and optimizations, as well as perform other tasks that were traditionally assigned to operations teams. And of course, code quality and proper testing continue to be at the top of the list for production-level applications. In this post, we’ll look at how to add test coverage to our Node.js applications and how we can apply it to our Serverless framework projects. ⚡️

Continue Reading…

25 Serverless Peeps You Need To Follow

In my never ending quest to consume all things serverless, I often find myself scouring the Interwebs for new and interesting serverless articles, blog posts, videos, and podcasts. There are more and more people doing fascinating things with serverless every day, so finding content is becoming easier and easier. However, this increase in content comes with an increase in noise as well. Cutting through that noise isn’t always easy. 🙉

Great content with valuable insights

I personally love reading articles that introduce new use cases or optimizations for serverless. Stories about companies using serverless in production and how their architectures are set up are also extremely interesting.. I’ve been working in the serverless space for several years now, and have come across a number of people who produce and/or share really great content. I’ve put together a list of 25 people that I follow and enjoy their content regularly. Hopefully these people will help you learn to love serverless as much as I do. ❤️⚡️

Continue Reading…

How To: Tag Your Lambda Functions for Smarter Serverless Applications

As our serverless applications start to grow in complexity and scope, we often find ourselves publishing dozens if not hundreds of functions to handle our expanding workloads. It’s no secret that serverless development workflows have been a challenge for a lot of organizations. Some best practices are starting to emerge, but many development teams are simply mixing their existing workflows with frameworks like Serverless and AWS SAM to build, test and deploy their serverless applications.

Beyond workflows, another challenge serverless developers encounter as their applications expand, is simply trying to keep all of their functions organized. You may have several functions and resources as part of a microservice contained in their own git repo. Or you might simply put all your functions in a single repository for better common library sharing. Regardless of how code is organized locally, much of that is lost when all your functions end up in a big long list in the AWS Lambda console. In this post we’ll look at how we can use AWS’s resource tagging as a way to apply structure to our deployed functions. This not only give us more insight into our applications, but can be used to apply Cost-Allocation Tags to our billing reports as well. 👍

Continue Reading…

Thinking Serverless (Big and Small)

I’ve been reading and writing a lot of about serverless lately, and one of the things I realized, is that most articles talk about how SCALABLE serverless architectures are. This, of course, is one of the major benefits of using serverless to build your applications. The ability to scale to thousands of concurrent requests per second without needing to manage your own servers, is simply amazing. 🙌

However, not needing to manage any servers has other benefits beyond the capabilities to achieve web scale. Having on-demand compute space also make serverless the perfect candidate for smaller workloads. In this post, let’s discuss how we can utilize serverless to handle our “less than unicorn 🦄” services and the benefits this can bring.

Continue Reading…

Lambda Warmer: Optimize AWS Lambda Function Cold Starts

At a recent AWS Startup Day event in Boston, MA, Chris Munns, the Senior Developer Advocate for Serverless at AWS, discussed Lambda cold starts and how to mitigate them. According to Chris (although he acknowledge that it is a “hack”) using the CloudWatch Events “ping” method is really the only way to do it right now. He gave a number of really good tips to pre-warm your functions “correctly”:

  • Don’t ping more often than every 5 minutes
  • Invoke the function directly (i.e. don’t use API Gateway to invoke it)
  • Pass in a test payload that can be identified as such
  • Create handler logic that replies accordingly without running the whole function

Continue Reading…

15 Key Takeaways from the Serverless Talk at AWS Startup Day

I love learning about the capabilities of AWS Lambda functions, and typically consume any article or piece of documentation I come across on the subject. When I heard that Chris Munns, Senior Developer Advocate for Serverless at AWS, was going to be speaking at AWS Startup Day in Boston, I was excited. I was able to attend his talk, The Best Practices and Hard Lessons Learned of Serverless Applications, and it was well worth it.

Chris said during his talk that all of the information he presented is on the AWS Serverless site. However, there is A LOT of information out there, so it was nice to have him consolidate it down for us into a 45 minute talk. There was some really insightful information shared and lots of great questions. I was aware of many of the topics discussed, but there were several clarifications and explanations (especially around the inner workings of Lambda) that were really helpful. 👍

Continue Reading…

Mixing VPC and Non-VPC Lambda Functions for Higher Performing Microservices

I came across a post the in the Serverless forums that asked how to disable the VPC for a single function within a Serverless project. This got me thinking about how other people structure their serverless microservices, so I wanted to throw out some ideas. I often mix my Lambda functions between VPC and non-VPC depending on their use and data requirements. In this post, I’ll outline some ways you can structure your Lambda microservices to isolate services, make execution faster, and maybe even save you some money. ⚡️💰

Continue Reading…

5 Reasons Why Your Serverless Application Might Be A Security Risk

There has been a lot of buzz lately about serverless security. People are certainly talking about it more and sharing great articles on the topic, but many serverless developers (especially new ones) are still making the same critical mistakes. Every time a serverless function is deployed, its unique security challenges need to be addressed. Every time. I’ve researched and written extensively about serverless security (see Securing Serverless: A Newbie’s Guide). I’ve read countless articles on the subject. And while there is no shortage of information available, let’s be honest: developers are busy building applications, not pouring through hundreds of articles.

I know, it sounds boring, but I would encourage you to do your research on serverless security. Serverless applications are different than traditional, server-hosted applications. Much of the security responsibility falls on the developer, and not following best practices opens you (or your company) up to an attack. But I know you’re busy. I totally get it. So rather than forcing you to read a bunch of long articles 😴 or watch a plethora of videos 🙈, I’ve whittled it all down to the five biggest serverless security risks for you. Sure, there are a lot of other things to consider, but IMO, these are the most important ones. Nothing here hasn’t been said before. But If you do nothing more than follow these principles, your serverless applications will be much more secure. 🔒

Continue Reading…

Serverless Consumers with Lambda and SQS Triggers

On Wednesday, June 27, 2018, Amazon Web Services released SQS triggers for Lambda functions. Those of you who have been building serverless applications with AWS Lambda probably know how big of a deal this is. Until now, the AWS Simple Queue Service (SQS) was generally a pain to deal with for serverless applications. Communicating with SQS is simple and straightforward, but there was no way to automatically consume messages without implementing a series of hacks. In general, these hacks “worked” and were fairly manageable. However, as your services became more complex, dealing with concurrency and managing fan out made your applications brittle and error prone. SQS triggers solve all of these problems. 👊

Update December 6, 2018: At some point over the last few months AWS fixed the issue with the concurrency limits and the redrive policy. See Additional experiments with concurrency and redrive polices below.

Audio Version (please note that this audio version is out of date given the new updates)

Continue Reading…

Event Injection: A New Serverless Attack Vector

As more and more developers and companies adopt serverless architecture, the likelihood of hackers exploiting these applications increases dramatically. The shared security model of cloud providers extends much further with serverless offerings, but application security is still the developer’s responsibility. There has been a lot of hype about #NoOPS with serverless environments 🤥, which is simply not true 😡. Many traditional applications are frontended with WAFs (web application firewalls), RASPs (runtime application self-protection), EPPs (endpoint protection platforms) and WSGs (web security gateways) that inspect incoming and outgoing traffic. These extra layers of protection can save developers from themselves when making common programming mistakes that would otherwise leave their applications vulnerable. With serverless, these all go away. 😳

Continue Reading…

10 Things You Need To Know When Building Serverless Applications

I am a HUGE fan of serverless architectures. This new type of compute not only opens up more possibilities for developers, but can support highly-scalable, complex applications for a fraction of the cost compared to provisioning virtual servers. My first question when planning a new application is always, “Can I build this with serverless?” Spoiler alert, the answer is almost always YES!

I’ve been building serverless applications since the release of AWS Lambda in 2015, so answering the question above is pretty easy for me. However, a lot of people I talk to who are new to serverless often have many questions (and misconceptions). I want you to be successful, so below I’ve create a list of 10 things you need to know when building a serverless application. These are things I wish I knew when I started, so hopefully they’ll help you get up to speed a faster and start building some amazing applications.

Continue Reading…

🚀 Project Update:

Lambda API: v0.7 Released

v0.7 adds new features to control middleware execution based on path, plus additional parsing of the AWS Lambda context object. ESLint and coverage reports using Istanbul and Coveralls were also added to ensure code quality and adequate test coverage. Read More...

How To: Optimize the Serverless Optimizer Plugin

I’m sure you’re already well aware of how awesome the ⚡ Serverless Framework is for managing and deploying your serverless applications. And you’re probably aware that there are several great plugins available that make Serverless even better. But did you know that there was a plugin to optimize your functions and reduce the size of your deployment packages? Or are you already using this plugin to optimize your functions, but hate how it takes too long to optimize locally run functions? In this post I’ll share some quick tips to help you optimize your Serverless Optimizer experience.

Continue Reading…

Transducers: Supercharge your functional JavaScript

This is the first in a series of posts on functional programming in JavaScript. My goal is to make these ideas more accessible to all levels of programmers. Feedback about style, content, etc., would all be greatly appreciated.

One thing that perplexed me early on in my functional programming days was the concept of transducers. I spent a lot of time Googling and found some great articles that went deep into the theory and the underlying mechanics. However, the practical use of them still seemed a bit out of reach. In this post I’ll attempt to explain transducers in a more understandable way and hopefully give you the confidence to use them in your functional JavaScript. While this article attempts to make transducers more accessible, you will need to have some basic knowledge of functional programming in JavaScript. Specifically, you should know about function composition and iterator functions like .map(), .filter(), and most importantly, .reduce(). If you are unfamiliar with these concepts, go get a grasp on them first.

Continue Reading…

🚀 Project Update:

Lambda API: v0.6 Released

v0.6 is all about making the serverless developer's life easier! New support for both callback-style and async-await in route functions and middleware, new HTTP method routing features, and route debugging tools. Plus Etag support and automatic authorization parsing. Read More...

Solving the Cold Start Problem

Dear AWS Lambda Team,

I have a serious problem: I love AWS Lambda! In fact, I love it so much that I’ve pretty much gone all in on this whole #serverless thing. I use Lambda for almost everything now. I use it to build backend data processing pipelines, distribute long running tasks, and respond to API requests. Heck, I even built an Alexa app just for fun. I found myself building so many RESTful APIs using Lambda and API Gateway that I went ahead and created the open source Lambda API web framework to allow users to more efficiently route and respond to API Gateway requests.

Serverless technologies, like Lambda, have revolutionized how developers think about building applications. Abstracting away the underlying compute layer and replacing it with on-demand, near-infinitely scalable function containers is brilliant. As we would say out here in Boston, “you guys are wicked smaht.” But I think you missed something very important. In your efforts to conform to the “pay only for the compute time you consume” promise of serverless, you inadvertently handicapped the service. My biggest complaint, and the number one objection that I hear from most of the “serverless-is-not-ready-for-primetime” naysayers, are Cold Starts.

Continue Reading…

How To: Manage Serverless Environment Variables Per Stage

I often find myself creating four separate stages for each ⚡ Serverless Framework project I work on: dev, staging, prod, and local. Obviously the first three are meant to be deployed to the cloud, but the last one, local, is meant to run and test interactions with local resources. It’s also great to have an offline version (like when you’re on a plane ✈ or have terrible wifi somewhere). Plus, development is much faster because you’re not waiting for round trips to the server. 😉

A really great feature of Serverless is the ability to configure ENVIRONMENT variables in the serverless.yml file. This lets us store important global information like database names, service endpoints and more. We can even reference passwords securely using AWS’s Service Manager Parameter Store and decode encrypted secrets on deployment, keeping them safe from developers and source repositories alike. 😬 Just reference the variable with ${ssm:/myapp/my-secure-value~true} in your configuration file.

Continue Reading…

🚀 Project Update:

Lambda API: v0.5 Released

v0.5 takes advantage of AWS Lambda's recently released support for Node v8.10 and has removed its Bluebird promise dependency in favor of async/await. Lambda API is now faster and adds built-in CORS support, additional wildcard features, new HTTP header management methods and more. Read More...

How To: Stub “.promise()” in AWS-SDK Node.js

Since AWS released support for Node v8.10 in Lambda, I was able to refactor Lambda API to use async/await instead of Bluebird promises. The code is not only much cleaner now, but I was able to remove a lot of unnecessary overhead as well. As part of the refactoring, I decided to use AWS-SDK’s native promise implementation by appending .promise() to the end of an S3 getObject call. This works perfectly in production and the code is super compact and simple:

The issue came with stubbing the call using Sinon.js. With the old promise method, I was using promisifyAll() to wrap new AWS.S3() and then stubbing the getObjectAsync method. If you’re not familiar with stubbing AWS services, read my post: How To: Stub AWS Services in Lambda Functions using Serverless, Sinon.JS and Promises.

Continue Reading…

How To: Manage RDS Connections from AWS Lambda Serverless Functions

Someone asked a great question on my How To: Reuse Database Connections in AWS Lambda post about how to end the unused connections left over by expired Lambda functions:

I’m playing around with AWS lambda and connections to an RDS database and am finding that for the containers that are not reused the connection remains. I found before that sometimes the connections would just die eventually. I was wondering, is there some way to manage and/or end the connections without needing to wait for them to end on their own? The main issue I’m worried about is that these unused connections would remain for an excessive amount of time and prevent new connections that will actually be used from being made due to the limit on the number of connections.

🧟‍♂️ Zombie RDS connections leftover on container expiration can become a problem when you start to reach a high number of concurrent Lambda executions. My guess is that this is why AWS is launching Aurora Serverless, to deal with relational databases at scale. At the time of this writing it is still in preview mode.

Update September 2, 2018: I wrote an NPM module that manages MySQL connections for you in serverless environments. Check it out here.

Update August 9, 2018: Aurora Serverless is now Generally Available!

Overall, I’ve found that Lambda is pretty good about closing database connections when the container expires, but even if it does it reliably, it still doesn’t solve the MAX CONNECTIONS problem. Here are several strategies that I’ve used to deal with this issue.

Continue Reading…

Is Code Really Self-Documenting?

In my 20+ years of programming, I’ve encountered a near endless amount of opinions on everything from coding styles to programming paradigms to the great whitespace debate. Obviously, I have strong opinions on a number of these. But for me, the one that bothers me the most is this notion that “code is self-documenting.” 😾

I know what you’re probably thinking: “of course not all code is self-documenting, only well-written code is.” I don’t entirely disagree. I can generally look at someone else’s code and understand exactly WHAT it is doing. However, often it’s not obvious WHY they did it that way, or even why they did it in the first place. In my opinion, the programmer’s intent (the WHY) is just as important as the HOW when it comes to properly documenting software.

So whether you agree with me or not, let’s explore how to better document our software by writing cleaner code, following some general commenting etiquette, and commenting more effectively to make you and your team more productive. 👍

Continue Reading…

Securing Serverless: A Newbie’s Guide

So you’ve decided to build a serverless application. That’s awesome! May I be the first to welcome you to the future. 🤖 I bet you’ve done a lot of research. You’ve probably even deployed a few test functions to AWS Lambda or Google Cloud Functions and you’re ready to actually build something useful. You probably still have a bunch of unanswered questions, and that’s cool. We can still build some really great applications even if we only know the basics. However, when we start working with new things we typically make a bunch of dumb mistakes. While some are relatively innocuous, security mistakes can cause some serious damage.

I’ve been working with serverless applications since AWS launched Lambda in early 2015. Over the last few years I’ve developed many serverless applications covering a wide range of use cases. The most important thing I’ve learned: SECURE YOUR FUNCTIONS! I can tell you from personal experience, getting burned by an attack is no bueno. I’d hate to see it happen to you. 😢

To make sure it doesn’t happen to you, I’ve put together a list of 🔒Serverless Security Best Practices. This is not a comprehensive list, but it covers the things you ABSOLUTELY must do. I also give you some more things to think about as you continue on your serverless journey. 🚀

Continue Reading…

How To: Build a Serverless API with Serverless, AWS Lambda and Lambda API

AWS Lambda and AWS API Gateway have made creating serverless APIs extremely easy. Developers can simply create Lambda functions, configure an API Gateway, and start responding to RESTful endpoint calls. While this all seems pretty straightforward on the surface, there are plenty of pitfalls that can make working with these services frustrating.

There are, for example, lots of confusing and conflicting configurations in API Gateway.  Managing deployments and resources can be tricky, especially when publishing to multiple stages (e.g. dev, staging, prod, etc.). Even structuring your application code and dependencies can be difficult to wrap your head around when working with multiple functions.

In this post I’m going to show you how to setup and deploy a serverless API using the Serverless framework and Lambda API, a lightweight web framework for your serverless applications using AWS Lambda and API Gateway. We’ll create some sample routes, handle CORS, and discuss managing authentication. Let’s get started.

Continue Reading…

Off-by-none: Issue #15

It’s all about Layers…

Welcome to Issue #15 of Off-by-none. I’m glad that you could join us. 😀

Last week we recapped re:Invent and took a look at some of the excellent talks and AWS product releases. This week we’ll dig deeper into Lambda Layers and see how people are having a bit of fun with custom runtimes. Plus we’ve got more talks from re:Invent and plenty of other serverless tidbits for your mental ingestion.

Lots to get to, so let’s get started! 🚄

What to do when AWS gives people access to Custom Lambda Runtimes… ⚙️

AWS already took care of C++ and Rust for us, plus some launch partners have already added PHP and Cobol support as well. But it seems that the community is taking advantage of this new feature in a big way.

The team over at The Agile Monkeys added a Haskell runtime. Think about it, a purely functional programming language running pure functions on stateless serverless functions! Okay, maybe that’s a bit much, but if you’re a hardcore functional programmer, you may want to give this a look. 😎

Graham Krizek added Bash support, which is pretty darn cool. He even included executables like aws, scp, git, wget and a whole lot more. Think about all the interesting and powerful use cases this opens up. Just this git support alone adds a number of possibilities. 🤓

Data scientists rejoice! You can now run R on Lambda thanks to this tutorial by Philipp Schirmer. There might be some memory limitations, but overall this looks like a workable solution for all you number crunchers. 📊

There’s also this proof of concept for a Serverless Open Runtime for AWS Lambda. Definitely an interesting concept, especially the language agnostic middlewares piece. Could turn out to be a terrible idea, but definitely something to keep your eye on. 🤷‍♂️

When you want to know how to use AWS Lambda Layers… 🥞

You can certainly build Lambda Layers on your own, but several companies are now providing them as a way for you to easily instrument your code. Epsagon, PureSec, Thundra, DataDog, IOpipe, and more, have all built Layers that you can simply plug in to your existing Lambda functions without modifying your code. That’s pretty easy.

Of course, our friend Paul Johnston has some thoughts on Lambda Layers and Custom Runtimes, including initial thoughts on best practices.

If you’re looking to help influence the future of Lambda Layers, take a look at this RFC on how to handle permissions with LayerVersions in SAM. AWS always appreciates feedback from the community, so feel free to throw your hat in the ring and add your comments. 🎩

When you refuse to believe you’ve watched all the good re:Invent talks… 📺

Not all of these are available to watch, but there is still a ton of amazing re:Invent content out there that you probably missed, even if you were at re:Invent! Here are three more talks that I found to be super interesting.

Inside AWS: Technology Choices for Modern Applications (SRV305)
Tim Bray, a Senior Principal Engineer from AWS, talks us through how AWS dogfoods serverless to power many of their own services. Even API Gateway runs on Lambda. He notes that “capacity planning sucks” and that you should “use serverless whenever possible.” This talk is full of great advice, including ways to “minimize state hydration”, plus some helpful notes on the three integration patterns. Watch the talk

Reddit’s Serverless & Compute Infrastructure at Scale (STP18)
Anand Mariappan & Jesjit Birak from Reddit take us through their latest redesign process and the steps they took to avoid another incident like “the Digg Mass Exodus of 2010.” The overall process was helpful to understand, but their method for scaling their video ingestion system using serverless tech is really interesting. A great lesson for enterprises here, as they built this to run along side their existing monolith. Watch the talk

Close Loops & Opening Minds: How to Take Control of Systems, Big & Small (ARC337)
Colm MacCárthaigh, another Senior Principal Engineer from AWS, lays out ten patterns to use while building control planes for distributed systems. Since all of our serverless applications are distributed, this makes for a really useful guide when building our own applications. Colm dives a bit into control theory, but keeps the advice practical so that you can apply these techniques immediately. Watch the talk

When you’re still debating what database to use with your serverless app… ⚖️

If you plan on using DynamoDB, you may want to look at Alex DeBrie’s DynamoDB On-Demand: When, why and how to use it in your serverless applications. Plus, lots of your burning DynamoDB questions are answered in here.

If you still want to go the relational database route, check out A crash course on Serverless with AWS — Building APIs with Lambda and Aurora Serverless by Adnan Rahić. This is a great post to get you started, I just wish he didn’t use an MySQL ORM. 🤦🏻‍♂️

And speaking of MySQL, I released a new version of serverless-mysql that fixes an ENQUEUE issue. If you’re not familiar with it, this module helps you with Managing MySQL at Serverless Scale.

What to do when you need more serverless content… 🙏

Jon Vines gives us some ideas about Breaking Down the Serverless Monolith. It’s tempting to load up functions with a lot of capabilities as it keeps things “simple” and is familiar to most developers. Some good lessons learned are outlined in this post.

If you’re interested in learning some more best practices, take a look at Five Essential Principles for Developing Lambdas. I think most of these are pretty solid (especially single-purpose lambdas), plus there are some examples, which is quite helpful.

Another great thing about single-purpose functions is that they can be optimized for their specific job. Case in point, don’t overpay when waiting on remote API calls by using the appropriate memory configurations.

For you serverless security buffs, take a look at Ory Segal’s 6 Cloud Security Predictions for 2019. And if you want some hands-on experience, try going through this Serverless Security Workshop. 🔒

When you remember that Microsoft Azure has serverless functions too… ⚡️

Mikhail Shilkov is Making Sense of Azure Durable Functions for you with his new (very detailed) post. Though the title suggests this is all about Microsoft’s solution, there is quite a bit of background on microservices, event-driven applications, serverless function composition and more. Definitely worth the read if you’ve got 20 minutes or so to spare. 📖

Kate Baroni, a Software Architect at Microsoft Azure, shows us how an Azure Function can orchestrate a real-time, serverless, big data pipeline.  Plus, if you’re interested, there are some links to related posts that go into more detail. I love finding interesting use cases like this, but it’s curious to me that Azure is doing complex orchestrations within a single function (with no mention of Durable Functions). This has always been a big anti-pattern with AWS Lambda, but maybe not with Microsoft? 😕

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Simon Wardley (@swardley). Simon invented Wardley Maps, which changes the way we look at strategic planning. You can read all about how it works here (and I suggest you do). Beyond that, Simon is a huge proponent of serverless and has been predicting for quite some time that it is the future of computing. He has a number of brilliant talks about serverless (including ServerlessDays Hamburg and Serverlessconf San Francisco 2018), plus his Twitter feed often contains entertaining back-and-forth arguments as to why serverless adoption is inevitable (see this recent Twitter thread). I’m a big fan of Simon and appreciate the work he is doing to make the case for serverless.

Final Thoughts 🤔

Lambda Layers is exposing serverless computing to a number of new communities, and people have been rushing to add support for all kinds of runtimes and service integrations. A recent report by Gartner identified “serverless computing” as the number one key trend for 2019 and noted that “more than 20 percent of global enterprises will have deployed serverless computing technologies by 2020.”

We are still early in this journey, but as Simon Wardley says, “No more questions on serverless. It’s not an ‘if’ but ‘when’. Get on with learning.” This is sage advice, and what we’re encouraging with this newsletter.

I hope you’ve enjoyed this issue of Off-by-none. I love getting your feedback. It is always most welcome and much appreciated. Your suggestions help me make this newsletter better each week. Please feel free to contact me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, and if you’d like to contribute to Off-by-none.

Go build some amazing serverless apps!

Take care,
Jeremy

Off-by-none: Issue #14

re:Capping re:Invent…

Welcome to Issue #14 of Off-by-none. I just spent a week in Vegas at AWS re:Invent and have I got a lot to share with you!

Last week we pondered if RDBMS were a good fit for serverless, overcame some common serverless objections and geeked out over serverless security. This week we’ll recap re:Invent, take a look at some of the amazing sessions and speakers, and review another 7,000 AWS product announcements (or something like that).

Buckle your seatbelt and let’s get started! 🏎

What to do if you’re suffering from re:FOMO… 😿

Unless you’ve been asleep for the last several months, you’re probably aware that AWS threw quite the shindig last week in Las Vegas. If you weren’t able to attend, don’t worry about it, we’ve got you covered. Because remember, what happens in Vegas, ends up on the Internet.

re:Capping re:Invent: AWS goes all-in on Serverless is my post that outlines some of the key announcements and what they all mean. I think I learned enough to write several books, so expect more posts to be coming.

What I learned from AWS re:Invent 2018 by James Beswick is also another great recap with a warning for cloud consultants and an important message about TCO. Paul Swail asks the question, What new use cases do the re:Invent 2018 serverless announcements open up? (answers included, of course). And if you want to read about all the Serverless announcements at re:Invent 2018, Alex DeBrie and Jared Short from Serverless, Inc. give you the full rundown.

When you’re looking for some really good conference talks… 👨‍🏫👩‍🏫

Advanced Design Patterns for DynamoDB (DAT401) 🤯
Rick Houlihan gave one of the most impressive talks of the entire conference. There were so many insights in this session that it was hard to keep track. He said, “We invented relational databases because storage was expensive” and “When people say NoSQL is missing JOINs, you say you’re missing the point.” He stressed that modeling NoSQL is difficult because you need to know and understand your access patterns upfront. But once you do, you can create a single table that can support 20 or more access patterns with just two or three Global Secondary Indexes (GSIs). Seriously mind-blowing stuff. Plus he stressed using serverless to validate your products. One of his best quotes was, “Don’t fail fast, fail cheap.” This could be the best 60 minutes you ever spend.

Watch the talk and checkout Best Practices for DynamoDB

From Monolith to Modern Apps: Best Practices (SRV322) 🎸
Paras Bhuva and Tom Laszewski (with a little help from Fender’s VP of IT, Chris Ingraham) gave an excellent talk that outlined how enterprises are adopting serverless for a variety of use cases. Companies like Reuters and Hearst are using it for analytics, Finra is using it for fraud detection, and Expedia is using it for operations. Paras walks us through the design of a modern application and stresses that teams want/need to “reduce their undifferentiated heavy lifting.” It is a very interesting session that really highlights the power, speed, and diversity of serverless applications.

Watch the talk

Serverless Architectural Patterns and Best Practices (ARC305) 🗺
Drew Dennis and Maitreya Raganath gave another really interesting talk that explored some architectural patterns and best practices. I see many people struggle with their serverless application designs because they aren’t quite sure how to stitch together all the managed services to create efficient pipelines. This talk looks at several common patterns including those for web applications, stream data processing, and data lakes.

Watch the talk

Applying Principles of Chaos Engineering to Serverless (DVC305)
Yan Cui (aka @theburningmonk), gave an awesome talk on Chaos Engineering and how we can apply those principles to serverless. The topics in here are so good that I don’t think I can do it justice by trying to sum this session up. Just do yourself a favor and watch it.

Watch the talk

A Serverless Journey: AWS Lambda Under the Hood (SRV409) 👩‍🔧
Holly Mesrobian and Marc Booker took us on a deep dive into how Lambda actually works. Lots of really interesting information, but perhaps the best part of the talk was this…

Reducing the cold start VPC issue by using a secure tunnel with a remote NAT and no longer stealing hundreds of IPs from CIDR blocks in your VPC subnets? Yeah, that’s a pretty big deal.

Watch the talk

There were so many amazing talks that I can’t possibly list them all. Be sure to check out AWS’s playlist on YouTube for an extensive list of recorded sessions. You can also check out this post by Jennine Townsend that lists some of the more notable sessions.

Just when you think that AWS might be running out of ideas… 🚀

Nope. Not only does AWS continue to make massive investments in its global infrastructure, hardware components and product offerings, but it also continues to break through the limits of serverless computing. Here are some of the important serverless announcements from last week.

When you’re still looking for some more serverless content… 👍

Joe Emison wrote a really great article that discusses The Serverless Sea Change. The post goes deep into the impact that serverless can have on companies and outlines an example of the dramatic cost savings that can be achieved. He makes an astute point that “ten times more lines of code, is ten times more technical debt.” Spending more time researching and less time coding will make maintaining your serverless applications much easier and a heckuva lot cheaper. 💰

Marcin Zasepa pointed out that Version 3 of the AWS SDK for JavaScript is written in TypeScript, so that’s pretty cool. 🤓

If you’re looking to jump in and start using some new AWS features, you can learn How to publish and use AWS Lambda Layers with the Serverless Framework.

You can also learn What’s New with Serverless at AWS during a webinar on December 11, 2018 @ 2pm ET. There are a lot of new things happening, so this might be a good opportunity to get a crash course.

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Corey Quinn (@QuinnyPig). Corey is a cloud economist that helps companies save money on their AWS bills. But he’s also the brains behind the Last Week in AWS newsletter, host of Screaming in the Cloud, occasional blogger, regular conference speaker, and all around nice guy. Don’t let the snark fool you, he is a huge proponent (and user) of serverless technologies, but also a vocal critic of AWS when necessary (which keeps them honest and on their toes). Corey shared this newsletter last week with his audience and many of you are reading this because of him. So here’s a huge thank you to Corey for helping me spread the serverless word. 🙌

Final Thoughts 🤔

This was another long one, but last week was a whirlwind of information and announcements that have cemented serverless as the future of cloud computing. I want to thank AWS and all the support staff that helped put together and run this amazing conference. And I also want to thank AWS for continuing to support their customers and pushing serverless innovation. The next few years are going to be really exciting.

I hope you’ve enjoyed this issue of Off-by-none. Your feedback is always most welcome and much appreciated. Your suggestions help me make this newsletter better each week. Please feel free to contact me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, and if you’d like to contribute to Off-by-none.

AWS just released enough products and features to keep us busy until next year’s re:Invent. So let’s go build some amazing serverless apps!

See you next week,
Jeremy