Product Guy, Serverless Advocate & Startup Veteran

My name is Jeremy Daly. I appreciate the visit. 👍 I’ve been managing the development of complex web and mobile applications for businesses across the globe for over 20 years. I’m currently the Chief Technology Officer at AlertMe, but I always keep myself busy with several side projects and consulting clients. I’m also an AWS Serverless Hero.

I write a lot about serverless and I like to share thoughts and stories about programmingproduct managemententrepreneurship and productivity. Sometimes I’ll write reviews or have random thoughts that I need to get out of my head. I also like to post a how-to article every once in a while. Feel free to jump in to one of the categories above, view all my posts, or view my latest posts below.

If you want to keep up with what I’m working on, you can follow me on TwitterGithub and Facebook.

Also, if you’re interested in serverless, please subscribe to Off-by-none, a weekly newsletter that focuses on all things serverless, and be sure to listen to Serverless Chats, a weekly podcast that discusses all things serverless.

My Latest Posts:

🚀 Project Update:

Lambda API: v0.10.3 Released

Lambda API v0.10.3 has been released to fix a few minor bugs around route prefixing and base64Encoding (thanks @Sleavely and @btakita), and to add a whitelist for headers on error cases (@hussfelt). Read More...

The Dynamic Composer (an AWS serverless pattern)

I’m a big fan of following the Single Responsibility Principle when creating Lambda functions in my serverless applications. The idea of each function doing “one thing well” allows you to easily separate discrete pieces of business logic into reusable components. In addition, the Lambda concurrency model, along with the ability to add fine-grained IAM permissions per function, gives you a tremendous amount of control over the security, scalability, and cost of each part of your application.

However, there are several drawbacks with this approach that often attract criticism. These include things like increased complexity, higher likelihood of cold starts, separation of log files, and the inability to easily compose functions. I think there is merit to these criticisms, but I have personally found the benefits to far outweigh any of the negatives. A little bit of googling should help you find ways to mitigate many of these concerns, but I want to focus on the one that seems to trip most people up: function composition.

Continue Reading…

🚀 Project Update:

Lambda API: v0.10.2 Released

Lambda API v0.10.2 has been released to add additional TypeScript support (@maxmellen), fix an issue with multiValueHeaders containing a null value (@stawecki), and support for overriding console.log for logging output (@Sleavely). Read More...
🚀 Project Update:

Serverless MySQL: v1.5.0 released

Serverless MySQL v1.5.0 is a maintenance/security update that also adds a new feature to transaction handling allowing you to return null queries to simplify conditional transaction workflows. Read More...

How to switch from RDBMS to DynamoDB in 20 easy steps…

I posted a thread on Twitter with some thoughts on how to how to switch from RDBMS to DynamoDB. Some people have asked me to turn it into a blog post to make it easier to follow. So here it is… with some bonus steps at the end. Enjoy! 😁

Continue Reading…

🚀 Project Update:

Serverless MySQL: v1.4.0 Released

Thanks to contributions from the community, Serverless MySQL v1.4.0 adds the ability to bring your own MySQL and Promise libraries, an enhancement that lets you use multiple instances, and some more TypeScript updates. Read More...

Developing Serverless Applications Locally with the “serverless-cloudside-plugin”

Developing and testing serverless applications locally can be a challenge. Even with tools like SAM and the Serverless Framework, you often end up mocking your cloud resources, or resorting to tricks (like using pseudo-variables) to build ARNs and service endpoint URLs manually. While these workarounds may have the desired result, they also complicate our configuration files with (potentially brittle) user-constructed strings, which duplicates information already available to CloudFormation.

This is a common problem for me and other serverless developers I know. So I decided to come up with a solution.

Continue Reading…

🚀 Project Update:

Lambda API: v0.10.1 Released

Lambda API v0.10.1 has been released to fix an issue with the "statusCode" reporting incorrectly in error logs. The arity requirement for handler functions has also been relaxed since it's possible that they may not be needed within a route definition. Read More...

Throttling Third-Party API calls with AWS Lambda

In the serverless world, we often get the impression that our applications can scale without limits. With the right design (and enough money), this is theoretically possible. But in reality, many components of our serverless applications DO have limits. Whether these are physical limits, like network throughput or CPU capacity, or soft limits, like AWS Account Limits or third-party API quotas, our serverless applications still need to be able to handle periods of high load. And more importantly, our end users should experience minimal, if any, negative effects when we reach these thresholds.

There are many ways to add resiliency to our serverless applications, but this post is going to focus on dealing specifically with quotas in third-party APIs. We’ll look at how we can use a combination of SQS, CloudWatch Events, and Lambda functions to implement a precisely controlled throttling system. We’ll also discuss how you can implement (almost) guaranteed ordering, state management (for multi-tiered quotas), and how to plan for failure. Let’s get started!

Continue Reading…

How To: Use SNS and SQS to Distribute and Throttle Events

An extremely useful AWS serverless microservice pattern is to distribute an event to one or more SQS queues using SNS. This gives us the ability to use multiple SQS queues to “buffer” events so that we can throttle queue processing to alleviate pressure on downstream resources. For example, if we have an event that needs to write information to a relational database AND trigger another process that needs to call a third-party API, this pattern would be a great fit.

This is a variation of the Distributed Trigger Pattern, but in this example, the SNS topic AND the SQS queues are contained within a single microservice. It is certainly possible to subscribe other microservices to this SNS topic as well, but we’ll stick with intra-service subscriptions for now. The diagram below represents a high-level view of how we might trigger an SNS topic (API Gateway → Lambda → SNS), with SNS then distributing the message to the SQS queues. Let’s call it the Distributed Queue Pattern.

Distributed Queue Pattern

This post assumes you know the basics of setting up a serverless application, and will focus on just the SNS topic subscriptions, permissions, and implementation best practices. Let’s get started!

Continue Reading…

🚀 Project Update:

Lambda API: v0.10 Released

Lambda API v0.10 adds the ability for you to seamlessly switch your Lambdas between API Gateway and Application Load Balancers. New execution stacks enables method-based middleware and more wildcard functionality. Plus full support for multi-value headers and multi-value query string parameters. Read More...

Stop Calling Everything Serverless!

I’ve been building serverless applications since AWS Lambda went GA in early 2015. I’m not saying that makes me an expert on the subject, but as I’ve watched the ecosystem mature and the community expand, I have formed some opinions around what it means exactly to be “serverless.” I often see tweets or articles that talk about serverless in a way that’s, let’s say, incompatible with my interpretation. This sometimes makes my blood boil, because I believe that “serverless” isn’t a buzzword, and that it actually stands for something important.

I’m sure that many people believe that this is just a semantic argument, but I disagree. When we refer to something as being “serverless”, there should be an agreed upon understanding of not only what that means, but also what it empowers you to do. If we continue to let marketers hijack the term, then it will become a buzzword with absolutely no discernible meaning whatsoever. In this post, we’ll look at how some leaders in the serverless space have defined it, I’ll add some of my thoughts, and then offer my own definition at the end.

Continue Reading…

Serverless Tip: Don’t overpay when waiting on remote API calls

Our serverless applications become a lot more interesting when they interact with third-party APIs like Twilio, SendGrid, Twitter, MailChimp, Stripe, IBM Watson and others. Most of these APIs respond relatively quickly (within a few hundred milliseconds or so), allowing us to include them in the execution of synchronous workflows (like our own API calls).  Sometimes we run these calls asynchronously as background tasks completely disconnected from any type of front end user experience.

Regardless how they’re executed, the Lambda functions calling them need to stay running while they wait for a response. Unfortunately, Step Functions don’t have a way to create HTTP requests and wait for a response. And even if they did, you’d at least have to pay for the cost of the transition, which can get a bit expensive at scale. This may not seem like a big deal on the surface, but depending on your memory configuration, the cost can really start to add up.

In this post we’ll look at the impact of memory configuration on the performance of remote API calls, run a cost analysis, and explore ways to optimize our Lambda functions to minimize cost and execution time when dealing with third-party APIs.

Continue Reading…

re:Capping re:Invent: AWS goes all-in on Serverless

Last week I spent six incredibly exhausting days in Las Vegas at the AWS re:Invent conference. More than 50,000 developers, partners, customers, and cloud enthusiasts came together to experience this annual event that continues to grow year after year. This was my first time attending, and while I wasn’t quite sure what to expect, I left with not just the feeling that I got my money’s worth, but that AWS is doing everything in their power to help customers like me succeed.

There have already been some really good wrap-up posts about the event. Take a look at James Beswick’s What I learned from AWS re:Invent 2018, Paul Swail’s What new use cases do the re:Invent 2018 serverless announcements open up?, and All the Serverless announcements at re:Invent 2018 from the Serverless, Inc. blog. There’s a lot of good analysis in these posts, so rather than simply rehash everything, I figured I touch on a few of the announcements that I think really matter. We’ll get to that in a minute, but first I want to point out a few things about Amazon Web Services that I learned this past week.

Continue Reading…

Aurora Serverless Data API: An (updated) First Look

Update June 5, 2019: The Data API team has released another update that adds improvements to the JSON serialization of the responses. Any unused type fields will be removed, which makes the response size 80+% smaller.

Update June 4, 2019: After playing around with the updated Data API, I found myself writing a few wrappers to handle parameter formation, transaction management, and response formatting. I ended up writing a full-blown client library for it. I call it the “Data API Client“, and it’s available now on GitHub and NPM.

Update May 31, 2019: AWS has released an updated version of the Data API (see here). There have been a number of improvements (especially to the speed, security, and transaction handling). I’ve updated this post to reflect the new changes/improvements.

On Tuesday, November 20, 2018, AWS announced the release of the new Aurora Serverless Data API. This has been a long awaited feature and has been at the top of many a person’s #awswishlist. As you can imagine, there was quite a bit of fanfare over this on Twitter.

Obviously, I too was excited. The prospect of not needing to use VPCs with Lambda functions to access an RDS database is pretty compelling. Think about all those cold start savings. Plus, connection management with serverless and RDBMS has been quite tricky. I even wrote an NPM package to help deal with the max_connections issue and the inevitable zombies 🧟‍♂️ roaming around your RDS cluster. So AWS’s RDS via HTTP seems like the perfect solution, right? Well, not so fast. 😞 (Update May 31, 2019: There have been a ton of improvements, so read the full post.)

Continue Reading…

🚀 Project Update:

Lambda API: v0.9.2 Released

Lambda API v0.9.2 has been released and contains additional updates and fixes for the index.d.ts TypeScript declarations file. Thanks again to @hassankhan and @Wintereise for submitting the changes. The release is immediately available via NPM. Read More...
🚀 Project Update:

Lambda API: v0.9.1 Released

Lambda API v0.9.1 has been released to include the index.d.ts TypeScript declarations file in the NPM package (thanks again, @hassankhan). The release is immediately available via NPM. Read More...
🚀 Project Update:

Lambda API: v0.9 Released

v0.9 adds new features to give developers better control over error handling and serialization. A TypeScript declaration file has also been added along with some additional API Gateway inputs that are now available in the REQUEST object. Read More...

Takeaways from ServerlessNYC 2018

I had the opportunity to attend ServerlessNYC this week (a ServerlessDays community conference) and had an absolutely amazing time. The conference was really well-organized (thanks Iguazio), the speakers were great, and I was able to have some very interesting (and enlightening) conversations with many attendees and presenters. In this post I’ve summarized some of the key takeaways from the event as well as provided some of my own thoughts.

Note: There were several talks that were focused on a specific product or service. While I found these talks to be very interesting, I didn’t include them in this post. I tried to cover the topics and lessons that can be applied to serverless in general.

Update November 16, 2018: Some videos have been posted, so I’ve provided the links to them.

Audio Version:

Continue Reading…

What 15 Minute Lambda Functions Tells Us About the Future of Serverless

Amazon Web Services recently announced that they increased the maximum execution time of Lambda functions from 5 to 15 minutes. In addition to this, they also introduced the new “Applications” menu in the Lambda Console, a tool that aggregates functions, resources, event sources and metrics based on services defined by SAM or CloudFormation templates. With AWS re:Invent just around the corner, I’m sure these announcements are just the tip of the iceberg with regards to AWS’s plans for Lambda and its suite of complementary managed services.

While these may seem like incremental improvements to the casual observer, they actually give us an interesting glimpse into the future of serverless computing. Cloud providers, especially AWS, continue to push the limits of what serverless can and should be. In this post, we’ll discuss why these two announcements represent significant progress into serverless becoming the dominant force in cloud computing.

Continue Reading…

🚀 Project Update:

Lambda API: v0.8.1 Released

Lambda API v0.8.1 has been released to patch an issue with middleware responses and a path prefixing options bug. The release is immediately available via NPM. Read More...

An Introduction to Serverless Microservices

Thinking about microservices, especially their communication patterns, can be a bit of a mind-bending experience for developers. The idea of splitting an application into several (if not hundreds of) independent services, can leave even the most experienced developer scratching their head and questioning their choices. Add serverless event-driven architecture into the mix, eliminating the idea of state between invocations, and introducing a new per function concurrency model that supports near limitless scaling, it’s not surprising that many developers find this confusing. 😕 But it doesn’t have to be. 😀

In this post, we’ll outline a few principles of microservices and then discuss how we might implement them using serverless. If you are familiar with microservices and how they communicate, this post should highlight how these patterns are adapted to fit a serverless model. If you’re new to microservices, hopefully you’ll get enough of the basics to start you on your serverless microservices journey. We’ll also touch on the idea of orchestration versus choreography and when one might be a better choice than the other with serverless architectures. I hope you’ll walk away from this realizing both the power of the serverless microservices approach and that the basic fundamentals are actually quite simple.  👊

Audio Version:

Continue Reading…

🚀 Project Update:

Serverless MySQL: v1.1.0 Released

Serverless MySQL v1.1.0 adds additional transaction support capabilities to allow users to retrieve interim query results for use with future queries. This is useful for getting the insertId from previous queries when performing transactions. Read More...

Serverless Security: Locking Down Your Apps with FunctionShield

I’ve written quite extensively about serverless security, and while you don’t need to be an expert on the matter, there are a number of common sense principles that every developer should know. Serverless infrastructures (specifically FaaS and managed services) certainly benefit from an increased security posture given that the cloud provider is handling things like software patching, network security, and to some extent, even DDoS mitigation. But at the end of the day, your application is only as secure as its weakest link, and with serverless, that pretty much always comes down to application layer security.

In this post we’re going to look at ways to mitigate some of these application layer security issues by using some simple strategies as well as a free tool called FunctionShield.

Audio Version:

Continue Reading…

Managing MySQL at Serverless Scale

“What? You can’t use MySQL with serverless functions, you’ll just exhaust all the connections as soon as it starts to scale! And what about zombie connections? Lambda doesn’t clean those up for you, meaning you’ll potentially have hundreds of sleeping threads blocking new connections and throwing errors. It can’t be done!”  ~ Naysayer

I really like DynamoDB and BigTable (even Cosmos DB is pretty cool), and for most of my serverless applications, they would be my first choice as a datastore. But I still have a love for relational databases, especially MySQL. It had always been my goto choice, perfect for building normalized data structures, enforcing declarative constants, providing referential integrity, and enabling ACID-compliant transactions. Plus the elegance of SQL (structured query language) makes organizing, retrieving and updating your data drop dead simple.

But now we have SERVERLESS. And Serverless functions (like AWS Lambda, Google Cloud Functions, and Azure Functions) scale almost infinitely by creating separate instances for each concurrent user. This is a MAJOR PROBLEM for RDBS solutions like MySQL, because available connections can be quickly maxed out by concurrent functions competing for access. Reusing database connections doesn’t help, and even the release of Aurora Serverless doesn’t solve the max_connections problem. Sure there are some tricks we can use to mitigate the problem, but ultimately, using MySQL with serverless is a massive headache.

Well, maybe not anymore. 😀 I’ve been dealing with MySQL scaling issues and serverless functions for years now, and I’ve finally incorporated all of my learning into a simple, easy to use NPM module that (I hope) will solve your Serverless MySQL problems.

Continue Reading…

Jeremy goes to AWS re:Invent 2018

It’s official! I’m going to AWS re:Invent 2018. 🙌

My goal from this trip is to learn, learn, learn… and then share, share, share.   There are over 30 sessions that talk about serverless, plus 40,000 other people there to meet and learn from! I’m so excited. 🙃

I know that many of you will be there, but for those of you who can’t be, I’ll do my best to share insights, tips, how-tos, best practices and more. I’ll even have a drink for you if you’d like 🍺 (no arm twisting necessary)!

Continue Reading…

Serverless Microservice Patterns for AWS

I’m a huge fan of building microservices with serverless systems. Serverless gives us the power to focus on just the code and our data without worrying about the maintenance and configuration of the underlying compute resources. Cloud providers (like AWS), also give us a huge number of managed services that we can stitch together to create incredibly powerful, and massively scalable serverless microservices.

I’ve read a lot of posts that mention serverless microservices, but they often don’t go into much detail. I feel like that can leave people confused and make it harder for them to implement their own solutions. Since I work with serverless microservices all the time, I figured I’d compile a list of design patterns and how to implement them in AWS. I came up with 19 of them, though I’m sure there are plenty more.

In this post we’ll look at all 19 in detail so that you can use them as templates to start designing your own serverless microservices.

Audio Version:

Continue Reading…

🚀 Project Update:

Lambda API: v0.8 Released

Lambda v0.8 is finally here and was well worth the wait! New features include allowing middleware to accept multiple handlers, new convenience methods for cache control and signing S3 URLs, and async/await support for the main function handler. And best of all, new LOGGING and SAMPLING support for you to add more observability into your APIs and web applications. Read More...

Aurora Serverless: The Good, the Bad and the Scalable

Amazon announced the General Availability of Aurora Serverless on August 9, 2018. I have been playing around with the preview of Aurora Serverless for a few months, and I must say that overall, I’m very impressed. There are A LOT of limitations with this first release, but I believe that Amazon will do what Amazon does best, and keep iterating until this thing is rock solid.

The announcement gives a great overview and the official User Guide is chock full of interesting and useful information, so I definitely suggest giving those a read. In this post, I want to dive a little bit deeper and discuss the pros and cons of Aurora Serverless. I also want to dig into some of the technical details, pricing comparisons, and look more closely at the limitations.

Audio Version

Continue Reading…

A Tale of Two Teams

Audio Version:

It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness… ~ A Tale of Two Cities by Charles Dickens

There is a revolution happening in the tech world. An emerging paradigm that’s letting development teams focus on business value instead of technical orchestration. It is helping teams create and iterate faster, without worrying about the limits or configurations of an underlying infrastructure. It is enabling the emergence of new tools and services that foster greater developer freedom. Freedom to experiment. Freedom to do more with less. Freedom to immediately create value by publishing their work without the traditional barriers created by operational limits.

Continue Reading…

How To: Add Test Coverage to your Serverless Applications

Writing serverless functions brings developers closer and closer to the stack that runs their code. While this gives them a tremendous amount of freedom, it also adds additional responsibility. Serverless applications require developers to think more about security and optimizations, as well as perform other tasks that were traditionally assigned to operations teams. And of course, code quality and proper testing continue to be at the top of the list for production-level applications. In this post, we’ll look at how to add test coverage to our Node.js applications and how we can apply it to our Serverless framework projects. ⚡️

Continue Reading…

Serverless Peeps You Need To Follow

In my never ending quest to consume all things serverless, I often find myself scouring the Interwebs for new and interesting serverless articles, blog posts, videos, and podcasts. There are more and more people doing fascinating things with serverless every day, so finding content is becoming easier and easier. However, this increase in content comes with an increase in noise as well. Cutting through that noise isn’t always easy. 🙉

Great content with valuable insights

I personally love reading articles that introduce new use cases or optimizations for serverless. Stories about companies using serverless in production and how their architectures are set up are also extremely interesting. I’ve been working in the serverless space for several years now, and have come across a number of people who produce and/or share really great content. I’ve put together a list of people that I follow and enjoy their content regularly. Hopefully these people will help you learn to love serverless as much as I do. ❤️⚡️

Continue Reading…

How To: Tag Your Lambda Functions for Smarter Serverless Applications

As our serverless applications start to grow in complexity and scope, we often find ourselves publishing dozens if not hundreds of functions to handle our expanding workloads. It’s no secret that serverless development workflows have been a challenge for a lot of organizations. Some best practices are starting to emerge, but many development teams are simply mixing their existing workflows with frameworks like Serverless and AWS SAM to build, test and deploy their serverless applications.

Beyond workflows, another challenge serverless developers encounter as their applications expand, is simply trying to keep all of their functions organized. You may have several functions and resources as part of a microservice contained in their own git repo. Or you might simply put all your functions in a single repository for better common library sharing. Regardless of how code is organized locally, much of that is lost when all your functions end up in a big long list in the AWS Lambda console. In this post we’ll look at how we can use AWS’s resource tagging as a way to apply structure to our deployed functions. This not only give us more insight into our applications, but can be used to apply Cost-Allocation Tags to our billing reports as well. 👍

Continue Reading…

Thinking Serverless (Big and Small)

I’ve been reading and writing a lot of about serverless lately, and one of the things I realized, is that most articles talk about how SCALABLE serverless architectures are. This, of course, is one of the major benefits of using serverless to build your applications. The ability to scale to thousands of concurrent requests per second without needing to manage your own servers, is simply amazing. 🙌

However, not needing to manage any servers has other benefits beyond the capabilities to achieve web scale. Having on-demand compute space also make serverless the perfect candidate for smaller workloads. In this post, let’s discuss how we can utilize serverless to handle our “less than unicorn 🦄” services and the benefits this can bring.

Continue Reading…

Lambda Warmer: Optimize AWS Lambda Function Cold Starts

At a recent AWS Startup Day event in Boston, MA, Chris Munns, the Senior Developer Advocate for Serverless at AWS, discussed Lambda cold starts and how to mitigate them. According to Chris (although he acknowledge that it is a “hack”) using the CloudWatch Events “ping” method is really the only way to do it right now. He gave a number of really good tips to pre-warm your functions “correctly”:

  • Don’t ping more often than every 5 minutes
  • Invoke the function directly (i.e. don’t use API Gateway to invoke it)
  • Pass in a test payload that can be identified as such
  • Create handler logic that replies accordingly without running the whole function

Continue Reading…

15 Key Takeaways from the Serverless Talk at AWS Startup Day

I love learning about the capabilities of AWS Lambda functions, and typically consume any article or piece of documentation I come across on the subject. When I heard that Chris Munns, Senior Developer Advocate for Serverless at AWS, was going to be speaking at AWS Startup Day in Boston, I was excited. I was able to attend his talk, The Best Practices and Hard Lessons Learned of Serverless Applications, and it was well worth it.

Chris said during his talk that all of the information he presented is on the AWS Serverless site. However, there is A LOT of information out there, so it was nice to have him consolidate it down for us into a 45 minute talk. There was some really insightful information shared and lots of great questions. I was aware of many of the topics discussed, but there were several clarifications and explanations (especially around the inner workings of Lambda) that were really helpful. 👍

Continue Reading…

Mixing VPC and Non-VPC Lambda Functions for Higher Performing Microservices

I came across a post the in the Serverless forums that asked how to disable the VPC for a single function within a Serverless project. This got me thinking about how other people structure their serverless microservices, so I wanted to throw out some ideas. I often mix my Lambda functions between VPC and non-VPC depending on their use and data requirements. In this post, I’ll outline some ways you can structure your Lambda microservices to isolate services, make execution faster, and maybe even save you some money. ⚡️💰

Continue Reading…

5 Reasons Why Your Serverless Application Might Be A Security Risk

There has been a lot of buzz lately about serverless security. People are certainly talking about it more and sharing great articles on the topic, but many serverless developers (especially new ones) are still making the same critical mistakes. Every time a serverless function is deployed, its unique security challenges need to be addressed. Every time. I’ve researched and written extensively about serverless security (see Securing Serverless: A Newbie’s Guide). I’ve read countless articles on the subject. And while there is no shortage of information available, let’s be honest: developers are busy building applications, not pouring through hundreds of articles.

I know, it sounds boring, but I would encourage you to do your research on serverless security. Serverless applications are different than traditional, server-hosted applications. Much of the security responsibility falls on the developer, and not following best practices opens you (or your company) up to an attack. But I know you’re busy. I totally get it. So rather than forcing you to read a bunch of long articles 😴 or watch a plethora of videos 🙈, I’ve whittled it all down to the five biggest serverless security risks for you. Sure, there are a lot of other things to consider, but IMO, these are the most important ones. Nothing here hasn’t been said before. But If you do nothing more than follow these principles, your serverless applications will be much more secure. 🔒

Continue Reading…

Serverless Consumers with Lambda and SQS Triggers

Update November 19, 2019: AWS announced support for SQS FIFO queues as a Lambda event source (announcement here). FIFO queues guarantee message order, which means only one Lambda function is invoked per MessageGroupId.

On Wednesday, June 27, 2018, Amazon Web Services released SQS triggers for Lambda functions. Those of you who have been building serverless applications with AWS Lambda probably know how big of a deal this is. Until now, the AWS Simple Queue Service (SQS) was generally a pain to deal with for serverless applications. Communicating with SQS is simple and straightforward, but there was no way to automatically consume messages without implementing a series of hacks. In general, these hacks “worked” and were fairly manageable. However, as your services became more complex, dealing with concurrency and managing fan out made your applications brittle and error prone. SQS triggers solve all of these problems. 👊

Update December 6, 2018: At some point over the last few months AWS fixed the issue with the concurrency limits and the redrive policy. See Additional experiments with concurrency and redrive polices below.

Audio Version (please note that this audio version is out of date given the new updates)

Continue Reading…

Event Injection: Protecting your Serverless Applications

Updated January 25, 2019: This post was updated based on feedback from the community.

The shared security model of cloud providers extends much further with serverless offerings, but application security is still the developer’s responsibility. Many traditional web applications are front-ended with WAFs (web application firewalls), RASPs (runtime application self-protection), EPPs (endpoint protection platforms) and WSGs (web security gateways) that inspect incoming and outgoing traffic. These extra layers of protection can save developers from themselves when making common programming mistakes that would otherwise leave their applications vulnerable. If you’re invoking serverless functions from sources other than API Gateway, you no longer have the ability to use the protection of a WAF. 

Continue Reading…

10 Things You Need To Know When Building Serverless Applications

I am a HUGE fan of serverless architectures. This new type of compute not only opens up more possibilities for developers, but can support highly-scalable, complex applications for a fraction of the cost compared to provisioning virtual servers. My first question when planning a new application is always, “Can I build this with serverless?” Spoiler alert, the answer is almost always YES!

I’ve been building serverless applications since the release of AWS Lambda in 2015, so answering the question above is pretty easy for me. However, a lot of people I talk to who are new to serverless often have many questions (and misconceptions). I want you to be successful, so below I’ve create a list of 10 things you need to know when building a serverless application. These are things I wish I knew when I started, so hopefully they’ll help you get up to speed a faster and start building some amazing applications.

Continue Reading…

🚀 Project Update:

Lambda API: v0.7 Released

v0.7 adds new features to control middleware execution based on path, plus additional parsing of the AWS Lambda context object. ESLint and coverage reports using Istanbul and Coveralls were also added to ensure code quality and adequate test coverage. Read More...

How To: Optimize the Serverless Optimizer Plugin

I’m sure you’re already well aware of how awesome the ⚡ Serverless Framework is for managing and deploying your serverless applications. And you’re probably aware that there are several great plugins available that make Serverless even better. But did you know that there was a plugin to optimize your functions and reduce the size of your deployment packages? Or are you already using this plugin to optimize your functions, but hate how it takes too long to optimize locally run functions? In this post I’ll share some quick tips to help you optimize your Serverless Optimizer experience.

Continue Reading…

Transducers: Supercharge your functional JavaScript

This is the first in a series of posts on functional programming in JavaScript. My goal is to make these ideas more accessible to all levels of programmers. Feedback about style, content, etc., would all be greatly appreciated.

One thing that perplexed me early on in my functional programming days was the concept of transducers. I spent a lot of time Googling and found some great articles that went deep into the theory and the underlying mechanics. However, the practical use of them still seemed a bit out of reach. In this post I’ll attempt to explain transducers in a more understandable way and hopefully give you the confidence to use them in your functional JavaScript. While this article attempts to make transducers more accessible, you will need to have some basic knowledge of functional programming in JavaScript. Specifically, you should know about function composition and iterator functions like .map(), .filter(), and most importantly, .reduce(). If you are unfamiliar with these concepts, go get a grasp on them first.

Continue Reading…

🚀 Project Update:

Lambda API: v0.6 Released

v0.6 is all about making the serverless developer's life easier! New support for both callback-style and async-await in route functions and middleware, new HTTP method routing features, and route debugging tools. Plus Etag support and automatic authorization parsing. Read More...

Solving the Cold Start Problem

Dear AWS Lambda Team,

I have a serious problem: I love AWS Lambda! In fact, I love it so much that I’ve pretty much gone all in on this whole #serverless thing. I use Lambda for almost everything now. I use it to build backend data processing pipelines, distribute long running tasks, and respond to API requests. Heck, I even built an Alexa app just for fun. I found myself building so many RESTful APIs using Lambda and API Gateway that I went ahead and created the open source Lambda API web framework to allow users to more efficiently route and respond to API Gateway requests.

Serverless technologies, like Lambda, have revolutionized how developers think about building applications. Abstracting away the underlying compute layer and replacing it with on-demand, near-infinitely scalable function containers is brilliant. As we would say out here in Boston, “you guys are wicked smaht.” But I think you missed something very important. In your efforts to conform to the “pay only for the compute time you consume” promise of serverless, you inadvertently handicapped the service. My biggest complaint, and the number one objection that I hear from most of the “serverless-is-not-ready-for-primetime” naysayers, are Cold Starts.

Continue Reading…

How To: Manage Serverless Environment Variables Per Stage

I often find myself creating four separate stages for each ⚡ Serverless Framework project I work on: dev, staging, prod, and local. Obviously the first three are meant to be deployed to the cloud, but the last one, local, is meant to run and test interactions with local resources. It’s also great to have an offline version (like when you’re on a plane ✈ or have terrible wifi somewhere). Plus, development is much faster because you’re not waiting for round trips to the server. 😉

A really great feature of Serverless is the ability to configure ENVIRONMENT variables in the serverless.yml file. This lets us store important global information like database names, service endpoints and more. We can even reference passwords securely using AWS’s Service Manager Parameter Store and decode encrypted secrets on deployment, keeping them safe from developers and source repositories alike. 😬 Just reference the variable with ${ssm:/myapp/my-secure-value~true} in your configuration file.

Continue Reading…

🚀 Project Update:

Lambda API: v0.5 Released

v0.5 takes advantage of AWS Lambda's recently released support for Node v8.10 and has removed its Bluebird promise dependency in favor of async/await. Lambda API is now faster and adds built-in CORS support, additional wildcard features, new HTTP header management methods and more. Read More...

How To: Stub “.promise()” in AWS-SDK Node.js

Since AWS released support for Node v8.10 in Lambda, I was able to refactor Lambda API to use async/await instead of Bluebird promises. The code is not only much cleaner now, but I was able to remove a lot of unnecessary overhead as well. As part of the refactoring, I decided to use AWS-SDK’s native promise implementation by appending .promise() to the end of an S3 getObject call. This works perfectly in production and the code is super compact and simple:

The issue came with stubbing the call using Sinon.js. With the old promise method, I was using promisifyAll() to wrap new AWS.S3() and then stubbing the getObjectAsync method. If you’re not familiar with stubbing AWS services, read my post: How To: Stub AWS Services in Lambda Functions using Serverless, Sinon.JS and Promises.

Continue Reading…

How To: Manage RDS Connections from AWS Lambda Serverless Functions

Someone asked a great question on my How To: Reuse Database Connections in AWS Lambda post about how to end the unused connections left over by expired Lambda functions:

I’m playing around with AWS lambda and connections to an RDS database and am finding that for the containers that are not reused the connection remains. I found before that sometimes the connections would just die eventually. I was wondering, is there some way to manage and/or end the connections without needing to wait for them to end on their own? The main issue I’m worried about is that these unused connections would remain for an excessive amount of time and prevent new connections that will actually be used from being made due to the limit on the number of connections.

🧟‍♂️ Zombie RDS connections leftover on container expiration can become a problem when you start to reach a high number of concurrent Lambda executions. My guess is that this is why AWS is launching Aurora Serverless, to deal with relational databases at scale. At the time of this writing it is still in preview mode.

Update September 2, 2018: I wrote an NPM module that manages MySQL connections for you in serverless environments. Check it out here.

Update August 9, 2018: Aurora Serverless is now Generally Available!

Overall, I’ve found that Lambda is pretty good about closing database connections when the container expires, but even if it does it reliably, it still doesn’t solve the MAX CONNECTIONS problem. Here are several strategies that I’ve used to deal with this issue.

Continue Reading…

Is Code Really Self-Documenting?

In my 20+ years of programming, I’ve encountered a near endless amount of opinions on everything from coding styles to programming paradigms to the great whitespace debate. Obviously, I have strong opinions on a number of these. But for me, the one that bothers me the most is this notion that “code is self-documenting.” 😾

I know what you’re probably thinking: “of course not all code is self-documenting, only well-written code is.” I don’t entirely disagree. I can generally look at someone else’s code and understand exactly WHAT it is doing. However, often it’s not obvious WHY they did it that way, or even why they did it in the first place. In my opinion, the programmer’s intent (the WHY) is just as important as the HOW when it comes to properly documenting software.

So whether you agree with me or not, let’s explore how to better document our software by writing cleaner code, following some general commenting etiquette, and commenting more effectively to make you and your team more productive. 👍

Continue Reading…

Securing Serverless: A Newbie’s Guide

So you’ve decided to build a serverless application. That’s awesome! May I be the first to welcome you to the future. 🤖 I bet you’ve done a lot of research. You’ve probably even deployed a few test functions to AWS Lambda or Google Cloud Functions and you’re ready to actually build something useful. You probably still have a bunch of unanswered questions, and that’s cool. We can still build some really great applications even if we only know the basics. However, when we start working with new things we typically make a bunch of dumb mistakes. While some are relatively innocuous, security mistakes can cause some serious damage.

I’ve been working with serverless applications since AWS launched Lambda in early 2015. Over the last few years I’ve developed many serverless applications covering a wide range of use cases. The most important thing I’ve learned: SECURE YOUR FUNCTIONS! I can tell you from personal experience, getting burned by an attack is no bueno. I’d hate to see it happen to you. 😢

To make sure it doesn’t happen to you, I’ve put together a list of 🔒Serverless Security Best Practices. This is not a comprehensive list, but it covers the things you ABSOLUTELY must do. I also give you some more things to think about as you continue on your serverless journey. 🚀

Continue Reading…

How To: Build a Serverless API with Serverless, AWS Lambda and Lambda API

AWS Lambda and AWS API Gateway have made creating serverless APIs extremely easy. Developers can simply create Lambda functions, configure an API Gateway, and start responding to RESTful endpoint calls. While this all seems pretty straightforward on the surface, there are plenty of pitfalls that can make working with these services frustrating.

There are, for example, lots of confusing and conflicting configurations in API Gateway.  Managing deployments and resources can be tricky, especially when publishing to multiple stages (e.g. dev, staging, prod, etc.). Even structuring your application code and dependencies can be difficult to wrap your head around when working with multiple functions.

In this post I’m going to show you how to setup and deploy a serverless API using the Serverless framework and Lambda API, a lightweight web framework for your serverless applications using AWS Lambda and API Gateway. We’ll create some sample routes, handle CORS, and discuss managing authentication. Let’s get started.

Continue Reading…

Off-by-none: Issue #64

pre:Invent is just getting started… 🤯

Welcome to Issue #64 of Off-by-none. Thanks for being here! 👋

Last week, we rehashed the Fat Lambda debate and met some new AWS heroes. This week, we look at a ton of announcements and feature launches leading up to re:Invent, plus we share lots of awesome content from the serverless community.

🥳 Don’t forget! If you’re planning on going to AWS re:Invent this year, be sure to register for the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. We’re adding names to the waiting list, but we’d still love for you to join us and our awesome guests! We have some amazing sponsors (StackeryThundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless) that are making this event possible. I hope you’ll be able to join us!

Lots to get to today, so let’s get to it! 🚀

Serverless News & Product Announcements 📣

Cloud Run, a managed Knative service, is GA
This really interesting service has been in beta for awhile, and is finally available in both a fully-managed version, and one for Anthos.

Launching our knowledge base
The team over at the Dashbird has put together a robust knowledge base that is loaded with all kinds of serverless information including DynamoDB modeling, API Gateway configurations, and much more. Releases Production-Ready Gloo Enterprise 1.0 for Modernizing to Cloud Native Applications
Gloo is an interesting solution for implementing the strangler pattern for aggregating and modernize distributed APIs. I don’t know much about the new Enterprise version, but it could be a useful bridge for companies that are straddling on-prem and cloud.

Aqua Security Acquires CloudSploit to Monitor Configs
Misconfiguring your cloud resources seems to be all the rage nowadays. This pickup by Aqua Security should help add some piece of mind, especially for those staring at huge cloud configuration files.

Pulumi infrastructure-as-code tool adds .NET language support
Pulumi continues on its quest to compete with AWS CDK, CloudFormation, and Terraform by adding additional language support for C#, F#, and VB.NET.

Serverless Use Cases 🗺

AWS Lambda: Real-World Use Cases for the DevOps Engineer
If you’re in the DevOps space and are curious how serverless can make your life easier, check out this excellent post from Emrah Samdan.

Stateful Programming Models in Serverless Functions
This is a good summary by Steef-Jan Wiggers that recounts Chris Gillum’s presentation at QCon. He talks about two ways in which you can add state to your Azure Functions to accommodate specific use cases.

Serverless Concepts 🏗

Error Handling in AWS Lambda and API Gateway
Ben Arena outlines four different approaches to handling errors with API Gateway and AWS Lambda. He didn’t mention Lambda API as a possible solution, but we’ll let it slide. 😉

Comparing the multi-table and single-table approaches to designing a DynamoDB data model
Paul Swail has an excellent article that discusses the multi-table versus single-table approach to designing data models in DynamoDB. I’m a big fan of the single table approach, but your mileage may vary, as Paul explains.

Why use Google Cloud Run? Here are 5 Compelling Reasons to Opt for Serverless Containers
Geshan Manandhar makes some good points, but as much as I love the idea of Google Cloud Run, there is still a fair amount of heavy lifting involved to get up and running. Plus, not having to learn a new paradigm might seems tempting, but ultimately it will be required in order to embrace a serverless mindset.

Serverless Patterns
I saw this pop up the other day. Toby Hede has put together some serverless patterns for you to learn from and use.

Serverless Patterns implemented, part 1
And speaking of serverless patterns, Vicenç García has taken some of the patterns from my post and created code samples for implementing them. Very cool.

Designing durable serverless apps with DLQs for Amazon SNS, Amazon SQS, AWS Lambda
AWS released DLQs for SNS topics this week, and with that, a bunch of new design patterns open up for building durable applications. Otavio Ferreira has an awesome post on how to use this new feature (along with existing fallback mechanisms) to build a highly durable serverless application.

EventBridge vs CloudWatch Events, Kinesis and SNS
Very informative post by Aviad Mor about EventBridge and how it compares to Kinesis and SNS. He mentions that EventBridge requires SQS to add resiliency, but that really depends on your downstream target. Unless you expect your target to be down for more that 24 hours, there is a fair amount of durability and resiliency built in. TLDR; don’t over engineer if you don’t need to.

AWS Serverless WebSockets – Introduction Around the Pitfalls
This is a great post by Joonas Laitio that goes through the details of how API Gateway WebSockets work, and what are some of the pitfalls that you need to be wary about.

AWS Lambda and Node.js 12: Support and Benchmark
AWS also released a bunch of new runtimes for Lambda, and Ran Ribenzaft benchmarked Node.js 12. According to this post, it’s a bit slower (which seems strange), but it’s packed with new features, so the tradeoff might be worth it.

Kinesis vs SNS/SQS
John Gilbert shares why he uses Kinesis for first-class events rather than SNS to SQS. While I agree that Kinesis is a workhorse that has lots of great use cases, I think you’ll see more of these event-driven workloads shift to EventBridge as a more flexible alternative.

Serverless Tutorials 🏗

Building a serverless data pipeline using Kinesis > Lambda > S3 Glacier pt. 1
Serkan Özal outlines the first few steps to building a serverless data pipeline using Kinesis.

Using AWS Lambda and SQS to perform mass, distributed and asynchronous updates on DynamoDB Tables
Great post by Rafal Wilinski that discusses possible serverless patterns to reliably perform DynamoDB table updates across a large number of items. I’ve implemented similar patterns (without the orchestrator) and have found that creating batches of 25 records PER SQS message is incredibly efficient. That way you set your Lambda batch size to 10, and you get 250 record updates per Lambda execution for a total of 10 batch writes.

Serverless Security 🔒

7 best practices when using AWS SSM Parameter Store
This useful post by Hoang Le outlines seven common practices when using AWS Parameter Store. #5 recommends fetching your parameters at runtime, which I agree with, but I prefer not to save the data in environment variables. Maybe I’m just paranoid.

Is Cloud Computing Secure? 7 FAQs About Cloud Security Answered (2020)
Trisha Paine from Protego answers seven frequently asked questions about cloud security.

Amazon GuardDuty Supports Exporting Findings to an Amazon S3 Bucket
I’m including this in the security section because it makes more sense here. Exporting your GuardDuty findings into S3 makes aggregating data across multiple regions and accounts super simple.

Serverless Reads 🤓

O’Reilly serverless survey 2019: Concerns, what works, and what to expect
Super interesting data in these survey results. Crazy to think that only 40% of respondents have adopted serverless in their organizations. The experience versus successful outcomes data is also quite fascinating. We still have a long way to go to get people on board, but it is certainly spreading.

How does improved VPC help reducing Lambda cold start
Crespo Wang explains how the improvements to VPC networking for Lambda dramatically reduced cold starts in his organization.

Towards Serverless (FaaS) as the next step in Infrastructure-as-a-Service (IaaS) evolution
Jonathan Eisenzopf has a interesting post that provides some thoughts on the current serverless ecosystem and how computing on the edge (or even on the client using WASM) might be the next evolution of compute.

Containers Or Serverless? The Battle For Your DevOps Mindshare
I’m including this post by John Anthony Radosta because I hold a different point of view. He mentions that “the complexity of inter-calling Lambda functions made it near impossible to debug anything.” If that’s the case, it sounds to me like it may have been a poorly designed system. Granted, knowing how to design serverless systems isn’t easy, but this seems like a case of blaming the hammer instead of the carpenter.

For the audiophile… 🎧

Episode #23: Serverless Application Security with Ory Segal (Part 1)
In this episode, I chat with Ory Segal about the differences between traditional and serverless security, the importance of the CSA’s 12 Most Critical Risks for Serverless Applications, and what the future of serverless security looks like.

And the flurry of AWS announcements and launches begins… 💨

Note: There were a TON of announcements this past week. I only included a handful of them that were really interesting to me, but it’s worth checking out the What’s New blog on AWS to see them all.

Amazon SNS Adds Support for Dead-Letter Queues (DLQ)
This is an amazing update! My biggest concern with sending events to SNS with destinations other than Lambda, was not being able to capture failures and replay messages. Now you can add an SQS queue to handle any failed delivery. Game changer.

AWS Lambda Supports Amazon SQS FIFO (First-In-First-Out) as an Event Source
Here’s an amazing edition to the Lambda event source catalog. SQS support has been available for quite some time, but SQS FIFO queues required a CloudWatch Events Rule hack to trigger your Lambda function every minute. Now you can simply subscribe a Lambda, and it will process your ordered messages serially as soon as they enter the queue.

You now can configure table settings when you restore a table from your Amazon DynamoDB backups
Love this feature! If you wanted to restore data to a new table, you had to restore all of the indexes and capacity settings. Now you can restore data without that extra overhead, which might even give you some flexibility with your data model.

Amazon DynamoDB adaptive capacity now handles imbalanced workloads better by isolating frequently accessed items automatically
This is a super cool feature. DynamoDB will now take frequently accessed items (like popular product records) and automatically distribute them to different partitions to avoid creating hot partitions and requiring more throughput.

AWS Lambda now supports Node.js 12
If you’ve been hoping for import/export statements, async stack traces, and private class variables, the wait is over.

AWS Lambda now supports Python 3.8
This is the newest major release of the Python language, and contains many new features such as assignment expressions, positional-only arguments, and typing improvements.

AWS Lambda now supports Java 11
You can use Java 11 features such as its improved HTTP Client API and new methods for reading and writing strings when authoring your functions.

AWS Step Functions adds Amazon EMR service integration
AWS Step Functions is now integrated with Amazon EMR, making it faster to build and easier to monitor EMR big data processing workflows. Read more here.

AWS AppSync adds Real-Time enhancements with Pure WebSockets support for GraphQL Subscriptions
Real-time updates to connected clients is becoming a standard practice in modern applications, and this new feature from AppSync, which basically handles all of this for you, is absolutely amazing. Read more here.

Amazon Transcribe Now Supports Speech-to-text in 8 Additional Languages
Repeating my previous comment for prosperity: “I’m looking forward to them fully supporting English.”

Support record-level insert, update, and delete on Amazon S3 with Amazon EMR
Super interesting use cases are possible with this, including a whole bunch of data privacy compliance. More about this here.

AWS CodePipeline Enables Passing Variables Between Actions At Execution Time
You can now pass variables from one action to another in your pipeline and dynamically configure your actions with variables that will be evaluated at execution time.

Amazon CloudWatch Launches Embedded Metric Format
This certainly adds a bunch of new capabilities, but it does seem to add quite a bit of heavy-lifting to your logging strategy.

AWS CloudFormation Launches Resource Import
So this is a super cool feature that allows you to import existing resources into CloudFormation stacks. Still waiting for the feature that allows you to export a resource configured in the console to a CloudFormation template. More detail here.

Introducing AWS Data Exchange
AWS Data Exchange is a new service that makes it easy for millions of AWS customers to securely find, subscribe to, and use third-party data in the cloud.

Thoughts from Twitter 🐦

I have some serious questions for you Twitter. What currently prevents the wide adoption of chaos engineering in your organization? ~ Adrian Hornsby
According to Adrian’s poll, it looks like most people feel as though there is “enough chaos in production” already, which obviously isn’t the point. It’s a fascinating discipline that most orgs would definitely benefit from.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

November 20, 2019 – Serverless for PCI DSS Blueprint (Webinar)

November 22, 2019 – Is Serverless SecureLess? (Webinar)

December 2-6, 2019 – AWS re:Invent 2019

December 14, 2019 – ServerlessDays Fukuoka Japan

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Jeff Hollan (@jeffhollan). Jeff is the Principal Program Manager for Microsoft Azure Functions. I know this issue is mostly about AWS and re:Invent, but let’s not forget that Microsoft launched a bunch of cool serverless stuff at Microsoft Ignite a couple of weeks ago. Jeff is leading the charge with Azure Functions, and when he gets free time, you will likely find him chatting up how to leverage cloud technology and serverless at various conferences and events around the world. Thank you, Jeff, for your advocacy and continued push for serverless innovation! 🙌

Final Thoughts 🤔

The number of announcements from AWS, two weeks before re:Invent, is already overwhelming. I know more are coming next week, and the ones at re:Invent are going to be mind-blowing! If you like to geek out on this stuff like I do, hold on to your hats, because the next few weeks are going to be a wild ride.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Take care,

Off-by-none: Issue #63

The Fat Lambda Debate Rages On… ⚖️

Welcome to Issue #63 of Off-by-none. Thanks for spending some time with us! 😀

Last week, we heard about some exciting serverless investment/acquisition news. This week, we dive back into the #FatLambda debate, meet some new heroes, and of course, we’ve got a ton of great content from the serverless community.

🥳 Don’t forget! If you’re planning on going to AWS re:Invent this year, be sure to register for the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. We’re adding names to the waiting list, but we’d still love for you to join us and our awesome guests! We have some amazing sponsors (StackeryThundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless) that are making this event possible. I hope you’ll be able to join us!

Okay, let’s get right to it. Lots of great stuff to get to! 🏆

When you’re weighing the benefits of a single-purpose function…

Last week, I posted a tweet that outlined a sort of conundrum I was having with EventBridge. My latest project has multiple services subscribed to several events that use EventBridge as the messaging bus. This works really well, but my quandary was more about what the event processing architecture should look like. Should each service have separate Lambda functions to process each type of incoming event? Or should each service have just ONE Lambda function that processes ALL incoming events?

I added a poll to the tweet, and as you can see, team “One Lambda PER Event” beat out team “Fat Lambda” by a 3-to-1 margin.

So clearly we have a winner, right? Well, if you read through the thread, you’ll see a number of wise serverless sages make excellent points as to why Fat Lambdas might be preferable. As I mention in my tweet, single-purpose Lambdas let you isolate business logic, provide fine-grained IAM permissions, specify concurrency per operation, and more. All of this is at the expense of creating lots of configuration and boilerplate code to handle (sometimes) fairly trivial operations.

Tim Wagner pointed out that “at some point the CloudFormation outweighs the code, and skinny lambdas.” Tim Bray brought up the same issue as it applies to Step Functions, and wrote “if there are a bunch of different tasks to be done, do you make a new Lambda for each, or do you have just one function that examines the data and figures out what to do?” This is a valid question, especially given the level of complexity your organization is comfortable with. He also said that  he’s “seen people go both ways and [hasn’t] been able to say that either is wrong.”

There were also others, like Gojko Adzic, who commented that he does neither. Instead, he “tend[s] to group events by security needs.” Heitor Lessa agreed, and gave a “+1 to that”, noting the importance of “Domain + Security context.” There were several others that simply touted the management simplicity of the #fatlambda, but plenty more, like Chris Munns, who argued the opposite and said he “think[s] longer term the flexibility will be non-blocking towards advancing certain capabilities, where as the fat func might eventually become overly complex and lead to you having to refactor too many bits at once.”

So who’s right? Team #fatfunction or Team #skinnyfunction? 🤷‍♂️ I hate to use the “it depends” answer, but there are certainly many factors that go into this decision when building serverless applications. The clear lesson here is that the community is still split on this. So, for now, whichever approach works best for you, is probably the right one.

Serverless News & Product Announcements 📣

Meet the newest AWS Heroes, including the first Data Heroes!
Yay! AWS has announced some new members to their Heroes program, and there are some great additions. Some of my favorites include Alex DeBrie, Farrah Campbell, Brian Leroux, and Gillian Armstrong! Congrats to all of you. 🎉

Tencent Cloud and Serverless Join Forces to Bring the Serverless Movement to China
Wow, this is pretty big. Tencent is the largest internet company in Asia and powers products like QQ, WeChat, Riot Games and League of Legends. Serverless, Inc. has made them a Premiere Partner, and you can already deploy an Express.js component to their Serverless Cloud Functions.

Fastly Announces Beta Version of the Compute@Edge Solution
Fastly is bringing serverless compute to the edge. According to their site, this is 100x faster than anything on the market. 🏎

Azure serves up pre-warmed Functions to offset cold starts
The Microsoft Ignite conference announced (or summarized) a few new features in their serverless arsenal, including their Premium plan supporting up to 60 minute execution times and pre-warmed instances. There’s also the ability to integrate secrets without needing to add any code.

Serverless Stories 📖

Architecting a Low-Cost Web Content Publishing System
Interesting story of how the University of St. Thomas used serverless technologies combined with legacy constraints to find a hybrid approach that solved their website speed, reliability, and management problems.

Serverless on Amazon: Is 2019 The Last Year I’ll Have to Deal With VMs?
Colin Dellow recounts three different types of projects he built using serverless and grades their overall performance. His approach to building an API and a website should make it obvious that more standard practices need to be followed (or maybe exist in the first place).

Top serverless plugins we are using
Hoang Le lists the Serverless Framework plugins that his team uses when developing serverless applications. It’s a long list, which probably shouldn’t be necessary, but there’s a nice serverless-mysql reference in there. 😉

Serverless Use Cases 🗺

An AWS Lambda ML Model Deployment
More interesting insights into deploying machines learning models on Lambda from Brian Schmidt.

Convert Radio Waves to Alerts using SDR, AWS Lambda and Amazon Transcribe
This is actually quite a fascinating use case (well, to me anyways 🤓). Using Software-defined Radio to capture audio from HAM radio frequencies, convert them to text using Amazon Transcribe, and then send alerts if certain things are said. That’s very cool.

Unlimited Email Sign-Ups for Free
Riccardo Giorato shows you a simple serverless use case, collecting email addresses from a web form with a serverless backend.

Processing High Volume Big Data Concurrently with No Duplicates using AWS SQS
Interesting post on how to use SQS polling and Lambda to reduce duplicate events. The problem I see here is that they’re using a Maximum Receive Count of “1” for an entire batch. This means that several messages in a batch could get processed successfully and still fail the Lambda, moving the entire batch to the DLQ. If you tried to replay those items, you’d likely reprocess some that already succeeded the first time.

Serverless Concepts 🏗

Containers vs/and Serverless
Emrah Samdan gives you the background on the difference between containers and serverless, what they’re good for, and when you might want to choose one over the other.

“Effortless” Serverless
Anjul Garg has a nice post that outlines some “best practices” for running serverless in production. Lots of good tips in here for people with all levels of experience running Lambda.

DynamoDB Importer
Awesome post by Rehan van der Merwe that shows you just how incredibly scalable DynamoDB is. He demonstrates the high throughput rate that DynamoDB can handle by writing 1 million records in 60 seconds with a single Lambda (~17k writes per second).

AWS S3 – 7 Useful CLI Commands
Quick little tutorial on using some of the standard S3 commands from the AWS-CLI. Never hurts to get a refresher.

The Resilient Architecture Collection
Adrian Hornsby has an excellent series on resilient architectures. If you haven’t looked at these yet, do yourself a favor, and find some time.

Serverless Tutorials 🏗

PHP and the AWS Lambda Custom Runtime (Part -1)
If you have any interest in compiling your own custom runtime for Lambda, this post by Mike McGrath might get you pointed in the right direction.

A Quickstart Guide to AWS Cognito, Lambda and SES
Ran Ribenzaft has a great post on getting started with AWS Cognito and integrating it into your serverless application. Writing your own authentication workflows is a lot of work (trust me, I know), so anytime you can use an out-of-the-box solution, it’s worth taking a look.

Writing an Alexa Skill with CRUD functionality using Dynamo DB, Serverless and AWS Lambdas
I honestly don’t think we utilize voice commands enough. Hamza Busuri has a full tutorial that shows you how to build a serverless Alexa skill.

Reduce bloat of your Lambdas
I’m always interested in optimizing my Lambda functions. Sérgio Ramos shows us an example of using the serverless-plugin-ncc to dramatically reduce package sizes and compilation time.

Serverless Security 🔒

Applying best practices for securing sensitive data in Amazon DynamoDB
You might want to grab your pocket protector for this one, because it’s for all us security nerds out there. 🤓 Really in-depth post with lots of really good pointers for securing your data in DynamoDB.

This Is How We Use Snyk to Protect Our Open-Source Projects from *Evil* Dependencies
Not specific to serverless, but third-party dependencies in our serverless apps are potential security threats as well. In this post, the team at Appwrite explains how they integrate Snyk into Github to scan for vulnerabilities on every pull request.

Serverless Reads 🤓

Why Serverless May Become a Dominant Programming Paradigm
I only included this because they say “Serverless is the new black”, which I thought was funny. There are some other good points in here too, but I think most of y’all already know them.

Going Serverless with AWS: Part 2
Eddie Jauregui has a simple explanation of a serverfull versus serverless workflow in an application that handles processing and resizing images. If you’re new to serverless, you might need to read the “Stateless and Event-Based Flow” section very carefully, but it’s a great explanation.

Most technology containers live less than five minutes, and lifespans are getting even shorter
According to this study, 63% of containers run for 10 minutes or less. Are these jobs that could be handled by FaaS?

Embrace the Creative Disruption of Cloud Native / Serverless
There’s a great tone to this post by John Gilbert. I liked how things used to work as well, but the cloud native / serverless approach, is just… better. Is it a paradigm shift? Yes. Is it “easier” to use? Not at first. But eventually it becomes just as easy as “the old days”, with all the benefits that come with this new approach.

For those that prefer the A/V club… 🎥

The Serverless Chats Podcast – Episode #22: Serverless Containers on Google Cloud Run with Bret McGowen
In this episode, I chat with Bret McGowen about how Google Cloud Run can make containers serverless, why Knative is important for building developer platforms, and what the future of serverless looks like at Google Cloud.

Building Resilient Serverless Systems with “Non-Serverless” Components
If you missed Serverlessconf NYC 2019, there are a lot of great talks to see. If you’re interested in watching mine, here it is. 😉

API Gateway Cognito User Pool Authorizer – Serverless Security
In her latest video, Marcia Villalba shows you how to set up a Cognito user pool authorizer for your API Gateway using AWS SAM.

Rethinking How You Do Development: Serverless at the Forefront
Danilo Poccia chats with serverless experts Austen Collins and Matt Weagle to discuss what it means to adopt a serverless mindset, how they got started, and some advice they have for anyone looking to get started with serverless.

Why Serverless is Cloud 2.0 – Serverless Computing London Keynote
Paul Johnston gave the keynote at Serverless Computing London and posted all 172 slides. There are lots of good points in here, and definitely worth taking a look.

When you’re curious what AWS is building… 👷‍♀️

NoSQL Workbench for Amazon DynamoDB adds support for DynamoDB local
You can now connect to your DynamoDB local instance using NoSQL Workbench.

Amazon QuickSight goes Mobile, launches Cross Source Join and More
Access to Amazon QuickSight from my mobile device? What is this, 2010? But seriously, this could prove to be very useful.

Amazon Comprehend Adds Six New Languages
Wow, this is big news for anyone doing NLP. Amazon Comprehend added six new languages including Chinese (Traditional), Chinese (Simplified), Korean, Hindi, Japanese, and Arabic.

15 Years of AWS Blogging!
Major props and thanks to Jeff Barr for all the work he’s done over the years sharing AWS knowledge. Crazy to think it’s been 15 years since he started blogging for AWS.

Introducing the AWS Step Functions Data Science SDK for Amazon SageMaker
This new open-source library allows you to easily create workflows that pre-processes data and then trains and publishes machine learning models using Amazon SageMaker and AWS Step Functions.

AWS CodeBuild Adds Support for AWS Secrets Manager
This is awesome. Much easier to pull secrets directly from Secrets Manager than to add them as environment variables referencing the Parameter Store.

Amazon CloudWatch launches cross-account cross-region dashboards
This might be one of the greatest features AWS has launched in a while. Accessing CloudWatch data across multiple regions and accounts will make monitoring and troubleshooting infinitely more simple. More details here.

Introducing notifications for AWS CodeCommit, AWS CodeBuild, AWS CodeDeploy, and AWS CodePipeline
I love the idea of notifications from these services, I just wish they were sent to EventBridge instead of SNS.

Thoughts from Twitter 🐦

That feeling when you’ve finally added proper DynamoDB indexes, refactored all Scans to Queries, cleaned up some old and unused stuff and your monthly AWS bill went down from ~$320 to ~$20. ~ Yury Smykalov
DynamoDB is an amazing tool, but as I (and hundreds of others have said), it’s not easy to learn. I love hearing stories like this that show the potential once people figure out some of the best/better practices.

CloudFormation: The Resource Import feature enables customers to import existing AWS resources into new or existing CloudFormation Stacks. ~ Jared Short
Jared’s keen changelog monitoring skills picked up on this gem. It’ll be interesting to see what sort of magical powers this will actually give us.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

November 14, 2019 – 5 steps to improving your software engineering team’s usage of serverless (Webinar)

November 19, 2019 – Thundra | Serverless at AWS re:Invent (Webinar) (I’ll be on this one)

November 22, 2019 – Is Serverless SecureLess? (Webinar)

December 2-6, 2019 – AWS re:Invent 2019

December 14, 2019 – ServerlessDays Fukuoka Japan

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Gillian Armstrong (@virtualgill). Gillian is a Solutions Architect at Liberty IT and a recently named AWS Machine Learning Hero. Gillian has been helping to bring machine learning and serverless into the enterprise by spearheading a number of internal projects, and then sharing her experiences and expertise with the community at large. Even though she is a “machine learning” hero, you’ll find her dedication to the advancement of serverless to be just as compelling. Thanks for all you do, Gillian!  🙌

Final Thoughts 🤔

There are so many new serverless things each week, that I find myself whittling through hundreds of posts, sometimes cutting out some really interesting takes. For those of you that are writing articles and sharing your experiences, first of all, thank you! The quality and quantity of serverless content out there is inspiring. And, secondly, if I don’t include your article, please feel free to reach out to me so that I can follow your work. One of the main reasons why I started this newsletter was to help others share their serverless ideas. If I can help you share yours, just let me know.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Until next week,

Off-by-none: Issue #62

Investing in Serverless…

Welcome to Issue #62 of Off-by-none. Thanks for joining us! 👍

Last week, we looked at the new “yumda” project from Michael Hart. This week, we’ve got some exciting serverless investment/acquisition news, plus lots of great content from the serverless community.

If you’re planning on going to AWS re:Invent this year, don’t forget to register for the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. We’re adding names to the waiting list, but we’d still love for you to join us and our awesome guests! We have some amazing sponsors (StackeryThundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless) that are making this event possible. I hope you’ll be able to join us!

Plenty to get to this week, so let’s get right to it! 🚀

Serverless News & Product Announcements 📣

New Relic snags early stage serverless monitoring startup IOpipe
This is exciting news for the team over at IOpipe. They will be joining New Relic to form New Relic Serverless, bringing all their expertise along with them. Read the founders’ note here.

Webiny announces $347K seed to build open-source serverless CMS
I think this is exciting news for serverless. A self-hosted serverless CMS that rivals WordPress is desperately needed. I don’t know if Webiny is the answer to that, but a lot of work has been done on this project and it definitely has promise.

Rackspace to Acquire Onica, a Cloud-Native Consulting and Managed Services Company
This is some interesting news for the cloud. Rackspace is going all in on multi-cloud, and using Onica as a way to help customer leverage AWS’s managed services as part of their cloud migration strategies could uniquely position Rackspace.

Moving to a Self-Hosted Model With Serverless Framework
On top of that seed round, Webiny is now able to be self-hosted by deploying it with the Serverless Framework.

Serverless Stories 📖

My first 14hrs with AWS Lambda
First impressions are important, and this post by Dennis Machu captures his experience as he started to learn serverless. Not uncommon to hear that CloudWatch Logs offer a suboptimal experience. I love hearing stories like this.

Scribble : Notes everywhere!!!
Shaurya Chauhan is working on a completely serverless note-taking app built with AWS Lambda. His three reasons for building with serverless: low maintenance, low cost, and easily scalable. Hard to argue with that for startups and side projects.

Serverless Use Cases 🗺

How to FaaS like a pro: 12 uncommon ways to invoke your serverless functions on AWS [Part 3]
Part 3 of Alex Casalboni’s excellent series on different ways to invoke serverless functions.

Using Google Cloud’s AutoML in AWS Lambda with API Gateway [Part 1]
This use case is the type of “multi-cloud” approach that I think works really well. Rahul Raman is calling Google’s AutoML API from a Lambda function.

When you’re looking for some advanced serverless concepts… 🏗

Check-list for going live with API Gateway and Lambda
Great post by Yan Cui that outlines (several) steps you should take to ensure that your API Gateway/Lambda project has all its bases covered. As he says, not everything on this list is absolutely necessary, but it should give you a good idea of the things you should be thinking about.

Serverless chaos engineering – interview with Emrah Samdan
Chaos engineering is fascinating, and with serverless, distributed systems, knowing how to handle failure at all levels is a must. Emrah Samdan sat down for an interview and lays out some of the basics.

An Introduction to AWS CDK
Excellent post by Efi Merdler-Kravitz that outlines the benefits of using the AWS CDK (Cloud Development Kit) instead of relying on other solutions to manage your Infrastructure-as-Code configurations.

How Do You Manage So Many Serverless Functions?
John Gilbert explains the common service patterns that he uses in serverless applications and how that helps him maintain a smaller number of functions, but still enforce single responsibility. I’ve switched to adding more functions for greater isolation, but there is definitely a management tradeoff.

Structuring a Real-World Serverless App
Frank Wong has a very detailed post outlining his suggestions for structuring a serverless app. There is a lot of good information in here, but personally, I prefer the multi-repo approach and using custom domain mappings over shared API Gateways. I wouldn’t label some of these things best practices, but rather “good practices” based on your use case.

Serverless Tutorials 🏗

Introduction to Amazon DynamoDB for Cassandra developers
This blog post introduces Amazon DynamoDB to Cassandra developers and helps them get started with DynamoDB by showing some basic operations in Cassandra and how to perform the same operations in DynamoDB.

Developing Complete Authorization Modules with AWS Lambda & Go & PostgreSQL & Event-Driven SQS Queue
Yunus Kılıç outlines a complete sample project in this post that shows you how to develop a completely serverless signup workflow using Go.

Using Serverless Framework & Localstack to test your AWS applications locally
Localstack is another option for testing your AWS serverless applications locally. This post by Antonio Reyes will give you the details on getting up and running with it.

Deploy Your Serverless Server-side-rendering (SSR)Angular App on AWS Lambda
Server-side rendering with AWS Lambda (or even Lambda@Edge) is becoming more and more popular, and I think with CloudFront on top, it’s a pretty slick solution. In this post, Neo Liu shows you how you can use Lambda for SSR with Angular.

How Do We Host Our Static Website
Hoang Le shows you how to host a serverless static website on AWS by defining a CI/CD pipeline to automate your deployment process.

Building A Serverless Backend API Part 2
Part 2 of Serkan Özal detailed tutorial on building out a serverless API.

How to create an OpenJS Architect serverless app with TypeScript
Architect is a pretty slick framework for building serverless applications. Brian Leroux shows you how to create your first serverless app built with TypeScript and Architect.

Serverless Security 🔒

State of Serverless and Security
Protego recently did a survey on the state of serverless and security. Big take away here is that 68% of respondents are not confident with their serverless security posture. Is this just normal hyper vigilance, or are cloud providers not doing enough?

Cloud Native Application Security – What You Need To Know
Cloud security is a huge topic (remember AWS’s re:Inforce conference?) that requires a lot of careful thought. This post contains some of the basics to get you thinking in the right direction.

Serverless Reads 🤓

Rethinking Serverless Architectures With EventBridge
I’m a huge fan of EventBridge and the power it adds to distributed messaging. Sarjeel Yusuf outlines why EventBridge is important, and why you should start rethinking the way you’re building serverless applications.

Late to the party? No problem. Meet serverless…
Tristan Strathearn compares serverless to other more traditional approaches using a number of factors to highlight the differences.

Forrester: The 5 ways cloud computing will change in 2020
There’s some good predictions in this post. I totally agree that hyperscale cloud alliances with SaaS companies will be huge to help move the focus back to app development. And, cloud security will definitely be at the forefront this coming year.

What to expect from serverless tech in 2020?
John Demian also has some thoughts on what to expect from serverless in 2020. He says education is sorely needed, and even though there is a lot out there, I totally agree.

Serverless: Is It The Kubernetes Killer?
Kendall Miller says “Serverless isn’t the future. But serverless is definitely a key part of the future.” I think it depends on how you define serverless. 😉

How Serverless Applications Will Change Your Business
Dorian Martin writes about six ways that adopting serverless will fundamentally change how you build and grow your business.

For the audiophile… 🎧

Serverless Chats Podcast – Episode #21: Getting Started with Serverless (Special Episode)
In this special episode, I outline a number of topics for people learning serverless, and let my guests from the first 20 episodes explain the details of each, and why they’re important.

When you’re curious what AWS is building… 👷‍♀️

AWS for WordPress plugin now available and with new Amazon CloudFront workflow
If you’re still using WordPress and would like to speed up page loads and reduce the traffic to your server, this new update to the AWS for WordPress plugin might help you. It will now manage a CloudFront distribution for you and use your WP site as the origin. I tried installing this and had some strange issues, but I have an atypical set up.

Create serverless applications with an automated deployment pipeline from the AWS Lambda console
A beta version of this feature was released early last month, but it’s not available in a bunch of regions. I really like how this bootstraps a project for you, but I wish it would generate the CloudFormation for the CI/CD process too.

Manage your Amazon API Gateway limits with AWS Service Quotas
If you haven’t looked at AWS Service Quotas yet, do yourself a favor and check it out. The AWS documentation is often a bit difficult to sift through, and this tool not only lets you see every service, what the limits are, and whether or not they are adjustable, but you can manage your quota change requests as well. Super handy.

Amplify CLI enables creating Amazon Cognito User Pool Groups, configuring fine-grained permissions on groups, and adding user management capabilities to applications
The Amplify CLI now enables developers to create Amazon Cognito User Pool Groups and configure fine grained permissions on these groups for accessing underlying backend resources such as S3, API Gateway, and AppSync.

AWS Secrets Manager now supports larger size for secrets and resource polices and higher request rate for GetSecretValue API
The GetSecretValue API call now allows up to 1,500 requests per second, plus you can store secrets that are up to 10 kb.

Serverless Tools 🛠

Designing a more approachable Serverless experience
The team over at the Agile Monkeys announced “Booster” at Serverlessconf New York. I think they are right about the learning curve, and more higher level abstractions are definitely needed. It’ll be interesting to see where this project goes and whether or not this opinionated approach will make it easier.

15 hours writing CloudFormation reduced to 15 minutes with Stackery
This is an all too familiar story for those of us who work with the beast that is CloudFormation. In many cases, writing CloudFormation by hand (even with some copy and pasting) is untenable. There are lots of abstractions for this, but Stackery’s visual builder is a pretty cool tool.

Thoughts from Twitter 🐦

Looks like #Serverless monitoring tools are giving up the focus on #AWS Lambda one by one. To me, this validates the obvious: AWS Lambda is still not as widely adopted as people expected after 5 years. What do you think? Does AWS Lambda still need time? ~ Serhat Can
It looks like Serhan is trying to stir some stuff up on Twitter. 😀 But he does make an astute observation about some monitoring tools shifting to a broader strategy. It’s a great thread that’s worth checking out.

Why I love #Serverless. Commerce event brought more visitors. Containers needed bit of TLC (even with the auto…) and serverless services? No one noticed they existed! That feeling, as in the MC advert, is priceless. ~ Sheen Brisals
Sheen posted about a recent traffic spike at LEGO, noting that the Lambda functions needed no special attention to handle the increased traffic.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

November 6-8, 2019 – Serverless Computing London

November 14, 2019 – 5 steps to improving your software engineering team’s usage of serverless (Webinar)

December 2-6, 2019 – AWS re:Invent 2019

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Sven Al Hamad (@SvenAlHamad). Sven is the CEO and co-founder of Webiny, an open source, serverless CMS. This project was started almost two years ago, and Sven and his team’s continued dedication has resulted in a seed round to help push this project even further. Building and maintaining a project like this isn’t easy, but thanks to people like Sven, the serverless community continues to benefit from great tools like Webiny! Thanks, Sven. Keep up the great work! 🙌

Final Thoughts 🤔

Congrats again to the teams at IOpipe and Webiny. It’s great to see investments like this into serverless projects and technologies. I’m excited to see where this leads for both of them.

I also want to thank all the #ServerlessForEveryone Community Party sponsors again. Stackery, Thundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless are the ones that have made this whole thing possible. Thank you all. It’s going to be an absolute blast!

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.


Off-by-none: Issue #61

Yup, it’s “yum” for Lambda… 👨🏻‍💻

Welcome to Issue #61 of Off-by-none. I’m so happy you’re here! 😀

Last week, we looked at alternatives for function composition and introduced some new serverless development platforms. This week, we’re going to take a quick look at the new “yumda” project, plus we’ve got plenty of great content from the community.

BTW, are you going to AWS re:Invent this year? If so, don’t forget to register for the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. We’ve got plenty of awesome guests and some amazing sponsors (Stackery, Thundra, AWS, CloudZero, LumigoProtego, and Serverless, Inc) that’ll make this a great event. I hope you’ll join us!

Lots to get to this week, so let’s get started! 🏎

Native dependencies in Lambda never tasted so good… 🍭

Last week on the Serverless Chats Podcast, I spoke with Michael Hart about an amazing new project he was working on called “yumda”. This project lets you easily install binary packages into your Lambda environments and then publish them as Lambda Layers to use in your serverless applications.

The project has been released to the public and is now available on GitHub. 🎉

This is a really great project that will make working with native binaries so much simpler. I think it will open up tons of possibilities, especially for those new to Lambda and serverless. Give it a look, and be sure to send your feedback.

Serverless News & Product Announcements 🚀

CNCF Serverless WG gets CloudEvents into incubator, approves 1.0 release
The CloudEvents spec from the CNCF is maturing. Several projects, including Microsoft’s EventGrid, Adobe I/O Events, Oracle Fn, and Knative have adopted the standard.

AWS Lambda Performance Optimization & Monitoring with Tracing & Spans (Serverless Framework Pro)
The Serverless Framework is adding feature after feature to give you complete visibility into your serverless applications. This new Tracing and Spans feature is another really useful tool.

Seamless Monitoring Simplified!
Thundra’s new update lets customers instrument their functions directly from the console rather than needing to modify YAML files or add additional instrumentation.

Serverless Stories 📖

Dirty Old Code
Pierre Bails and the team at Precogs migrated their monolithic Ruby on Rails app to a serverless environment. Check out how they migrated their application in pieces, slowly reducing the dependency on the legacy system.

Canary (B/G) Deployment for AWS Serverless Stacks (Immutable IaC)
Dinesh Dandu walks us through how his team built their CI/CD pipeline to deploy their serverless application.

My key takeaways from ServerlessDays Stockholm
ServerlessDays events are awesome. They give people a chance to see and hear speakers from around the world. Tuomo Varis was at ServerlessDays Stockholm, and has put together a nice recap of the event.

Serverless Use Cases 🗺

Creating, monitoring, and testing cron jobs on AWS
Running automated tasks is one of the most basic, yet powerful use cases for serverless. Gareth McCumskey has a very detailed post that will show you how to take full advantage of serverless cron jobs.

Serverless Event Scheduling
Ben Ellerby outlines a variation on the Step Functions as an email scheduler pattern. He adds an SQS queue for some additional durability and retries. I really like this Step Functions approach for scheduling ad hoc events.

The Serverless Vector Map Stack Lives!
Great post by Alexander Rolek that details how the release of AWS Aurora Serverless Postgres (with PostGIS support) makes building an entirely serverless map stack now possible.

When you’re looking for some advanced serverless concepts… 🏗

Serverless – AWS Lambda Python Dependencies
Dorian Machado gives you the basic steps to install Python dependencies for use in your Lambda functions.

.NET Core 3.0 AWS Lambda Benchmarks and Recommendations
Zac Charles has been keeping up with the performance of .NET Core 3.0 on AWS Lambda and shares his latest benchmarks and recommendations with us.

AWS Lambda – The Ultimate Guide
Pretty much everything you would ever want to know about Lambda is in this article from the team at Serverless, Inc.

A simple introduction to Step Functions
If you’ve yet to dive into AWS Step Functions, or just want a quick refresher, take a look at Renato Byrro’s introductory post.

Don’t wait for Functionless. Write less Functions instead
Sheen Brisals has a great post that gets you thinking about minimizing your Lambda functions in favor of using built in cloud integrations. We’re not “functionless” yet, but as Sheen points out, there are a lot of things we can do without them.

Serverless Tutorials 🏗

Serverless Data Visualizations
This tutorial will show you how to create data visualizations with AWS Lambda using Python’s Pandas, Numpy, and Matplotlib libraries.

How to get started and debug Lambda functions
Ankita Kulkarni shows you how to build and test Lambda functions with Netlify.

Developing Serverless Applications on AWS using AWS Serverless Application Model (SAM)
This is a thorough document that consolidates a lot of information into a single post. If you’re working with AWS SAM, this could be a good reference manual for you.

Go serverless in Kubernetes with Knative
If you’re thinking about going down the Knative route, Paulo Kieffer has a short and sweet intro for you.

Building A Serverless Backend API Part 1
Here’s a very detailed tutorial from Serkan Özal on how to create a serverless API. This is great for beginners.

Setting up CI/CD for Lambda Functions using AWS CodePipeline
Lots of posts on CI/CD for serverless projects, and everyone’s process is different! Here is another example by Maarten Thoelen.

Serverless CI/CD Using Github Actions
And here’s another CI/CD workflow for your serverless applications. This one uses the new Github Actions, which is kind of cool.

Serverless Security 🔒

Cloud Native Security: Focus On Your Domain
John Gilbert shares his thoughts on where the shared responsibility model ends with serverless applications, and where you must step in to ensure your users’ sensitive data is protected.

Cloud Native Security: What it Means
Tal Melamed explains how serverless and cloud native change the security landscape, and offers some recommendations for making sure your applications are secure.

Serverless Reads 🤓

Serverless and multi-cloud: Hype or reality?
Interesting post by Sebastien Goasguen from TriggerMesh. In my mind, there is a big difference between being “multi-cloud” and using services from “multiple” clouds. While we’ll definitely continue to see enterprises adopting the multi-cloud strategy, I still believe this has the opposite effect on choosing best-of-breed services.

Hey AWS – it’s time to pay OSS developers
Forrest Brazeal’s latest Cloud Irregular newsletter points out the lack luster performance of AWS’s Serverless Application Repository (SAR). Could adding some incentives for developers make it more popular?

What is Serverless? The “2020” edition
Paul Johnston can’t stand the word “serverless” any more, but that hasn’t stopped him from embracing the value it provides. In his latest post, he articulates a few major positions on what serverless is, and why it’s important for the future of the cloud.

The Serverless Supercomputer
If you’re curious about the future of serverless, and what’s truly possible, take a few minutes to read Tim Wagner’s latest post.

For the those that want an audio/visual experience… 📽

Episode #20: The Serverless Journey of with Sheen Brisals
In this episode, I chat with Sheen Brisals about the problems the LEGO Group was trying to solve with serverless, what they learned from their journey, and what’s the idea behind “functionless.

Serverlessconf NYC 2019 Videos
A Cloud Guru posted several of the talks from this year’s Serverlessconf in NYC. Lots of really interesting presentations, so be sure to grab some popcorn and a notebook.

Building CI/CD Pipelines for Serverless Applications
More serverless CI/CD goodness. This time from the Serverless Toronto meetup.

When you’re wondering what the devs at AWS are up to… 👷‍♀️

Amazon Transcribe Now Supports Australian English Speech-to-Text in Real Time
I know AWS keeps adding dialect support, but I’ve been using it with US English for quite some time and the accuracy can be terrible. Maybe they just need to add Boston English as another supported language. 🤷‍♂️

AWS Amplify Console announces Pull-Request Previews for Fullstack Serverless Applications
This is a pretty cool feature. A pull-request preview deploys every pull request made to your GitHub repository to a unique preview URL. It even spins up and tears down ephemeral backends for testing.

200 Amazon CloudFront Points of Presence + Price Reduction
Wow, 200 total POPs. I have some extra room in my basement if AWS is looking for some more space.

Serverless Tools 🛠

This is a Serverless framework plugin that simplifies the configuration of SNS to SQS to Lambda.

Edit serverless YAML templates in VSCode like a pro | Serverless IDE
I think we’ve mentioned this tool before, but Pavel Vlasov’s Serverless IDE tool is a nice little add-on for your serverless development workflow.

Thoughts from Twitter 🐦

10 reasons why I love AWS Step Functions so much, illustrated using a JavaScript Promise chain: step = json => service(params) .then(flakyService()) .then(longService()) .then(({ id }) => Promise.all([send(id), audit(id)])) ~ Jesse Warden
Jesse Warden has a clever thread that demonstrates the usefulness of AWS Step Functions.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

October 30, 2019 – Serverless Transformation Webinar – How Can Serverless Improve (I’ll be on this with Ran Ribenzaft and Erica Windisch)

October 30, 2019 – Retail at the Scale of Serverless with AWS – Webinar

October 30, 2019 – Managing Serverless Applications with SAM Templates

November 6-8, 2019 – Serverless Computing London

December 2-6, 2019 – AWS re:Invent 2019

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Daniel ZivKovic (@DanielBookmarks). Daniel is a Cloud Solutions Architect at Onica, and the organizer of the Serverless Toronto User Group. He’s been organizing and running these events since May of last year, and has brought in a number of amazing serverless speakers to share their knowledge with the Toronto developer community. I’ve met Daniel a number of times, and his enthusiasm for serverless is infectious. The work he does helps bring new people into the serverless community everyday, and that is something to cheer for. Thanks for all you do, Daniel! 🙌

Final Thoughts 🤔

It’s been another heads down week for me trying to wrap up a big serverless project I’m working on, but I did get a chance to chat with some AWS PMs and partners. Lots of super cool stuff coming down the pike for Lambda and serverless at AWS. We’re just over a month away from re:Invent, and I’m ridiculously excited about it. It’s going to be a very, very good end-of-year for serverless.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Until next week,