Product Guy, Serverless Advocate & Startup Veteran

My name is Jeremy Daly. I appreciate the visit. 👍 I’ve been managing the development of complex web and mobile applications for businesses across the globe for over 20 years. I’m currently the Chief Technology Officer at AlertMe, but I always keep myself busy with several side projects and consulting clients. I’m also an AWS Serverless Hero.

I write a lot about serverless and I like to share thoughts and stories about programmingproduct managemententrepreneurship and productivity. Sometimes I’ll write reviews or have random thoughts that I need to get out of my head. I also like to post a how-to article every once in a while. Feel free to jump in to one of the categories above, view all my posts, or view my latest posts below.

If you want to keep up with what I’m working on, you can follow me on TwitterGithub and Facebook.

Also, if you’re interested in serverless, please subscribe to Off-by-none, a weekly newsletter that focuses on all things serverless, and be sure to listen to Serverless Chats, a weekly podcast that discusses all things serverless.


My Latest Posts:

Takeaways from AWS re:Invent 2019’s Amazon DynamoDB Deep Dive: Advanced Design Patterns (DAT403)

AWS re:Invent 2019 is a wrap, but now the real work begins! There are hundreds of session videos now available on YouTube. So when you have a few days (or weeks) of downtime, you can dig in to these amazing talks and learn about whatever AWS topics you fancy.

I was only able to attend a few talks this year, but one that I knew I couldn’t miss in person, was Rick Houlihan’s DAT403: Amazon DynamoDB deep dive: Advanced design patterns. At the last two re:Invents, he gave similar talks that explored how to use single-table designs in DynamoDB… and they blew my mind! 🤯 These videos were so mind-bending, that they inspired me to immerse myself in NoSQL design and write my How to switch from RDBMS to DynamoDB in 20 easy steps post. I was hoping to have a similar experience with this year’s edition, and I WAS NOT DISAPPOINTED.

As expected, it was a 60 minute firehose of #NoSQL knowledge bombs. There was A LOT to take away from this, so after the session, I wrote a Twitter thread that included some really interesting lessons that stuck out to me. The video has been posted, so definitely watch it (maybe like 10 times 🤷‍♂️), and use it to get started (or continue on) your DynamoDB journey.

Continue Reading…

🚀 Project Update:

Lambda API: v0.10.3 Released

Lambda API v0.10.3 has been released to fix a few minor bugs around route prefixing and base64Encoding (thanks @Sleavely and @btakita), and to add a whitelist for headers on error cases (@hussfelt). Read More...

The Dynamic Composer (an AWS serverless pattern)

I’m a big fan of following the Single Responsibility Principle when creating Lambda functions in my serverless applications. The idea of each function doing “one thing well” allows you to easily separate discrete pieces of business logic into reusable components. In addition, the Lambda concurrency model, along with the ability to add fine-grained IAM permissions per function, gives you a tremendous amount of control over the security, scalability, and cost of each part of your application.

However, there are several drawbacks with this approach that often attract criticism. These include things like increased complexity, higher likelihood of cold starts, separation of log files, and the inability to easily compose functions. I think there is merit to these criticisms, but I have personally found the benefits to far outweigh any of the negatives. A little bit of googling should help you find ways to mitigate many of these concerns, but I want to focus on the one that seems to trip most people up: function composition.

Continue Reading…

🚀 Project Update:

Lambda API: v0.10.2 Released

Lambda API v0.10.2 has been released to add additional TypeScript support (@maxmellen), fix an issue with multiValueHeaders containing a null value (@stawecki), and support for overriding console.log for logging output (@Sleavely). Read More...
🚀 Project Update:

Serverless MySQL: v1.5.0 released

Serverless MySQL v1.5.0 is a maintenance/security update that also adds a new feature to transaction handling allowing you to return null queries to simplify conditional transaction workflows. Read More...

How to switch from RDBMS to DynamoDB in 20 easy steps…

I posted a thread on Twitter with some thoughts on how to how to switch from RDBMS to DynamoDB. Some people have asked me to turn it into a blog post to make it easier to follow. So here it is… with some bonus steps at the end. Enjoy! 😁

Continue Reading…

🚀 Project Update:

Serverless MySQL: v1.4.0 Released

Thanks to contributions from the community, Serverless MySQL v1.4.0 adds the ability to bring your own MySQL and Promise libraries, an enhancement that lets you use multiple instances, and some more TypeScript updates. Read More...

Developing Serverless Applications Locally with the “serverless-cloudside-plugin”

Developing and testing serverless applications locally can be a challenge. Even with tools like SAM and the Serverless Framework, you often end up mocking your cloud resources, or resorting to tricks (like using pseudo-variables) to build ARNs and service endpoint URLs manually. While these workarounds may have the desired result, they also complicate our configuration files with (potentially brittle) user-constructed strings, which duplicates information already available to CloudFormation.

This is a common problem for me and other serverless developers I know. So I decided to come up with a solution.

Continue Reading…

🚀 Project Update:

Lambda API: v0.10.1 Released

Lambda API v0.10.1 has been released to fix an issue with the "statusCode" reporting incorrectly in error logs. The arity requirement for handler functions has also been relaxed since it's possible that they may not be needed within a route definition. Read More...

Throttling Third-Party API calls with AWS Lambda

In the serverless world, we often get the impression that our applications can scale without limits. With the right design (and enough money), this is theoretically possible. But in reality, many components of our serverless applications DO have limits. Whether these are physical limits, like network throughput or CPU capacity, or soft limits, like AWS Account Limits or third-party API quotas, our serverless applications still need to be able to handle periods of high load. And more importantly, our end users should experience minimal, if any, negative effects when we reach these thresholds.

There are many ways to add resiliency to our serverless applications, but this post is going to focus on dealing specifically with quotas in third-party APIs. We’ll look at how we can use a combination of SQS, CloudWatch Events, and Lambda functions to implement a precisely controlled throttling system. We’ll also discuss how you can implement (almost) guaranteed ordering, state management (for multi-tiered quotas), and how to plan for failure. Let’s get started!

Continue Reading…

How To: Use SNS and SQS to Distribute and Throttle Events

An extremely useful AWS serverless microservice pattern is to distribute an event to one or more SQS queues using SNS. This gives us the ability to use multiple SQS queues to “buffer” events so that we can throttle queue processing to alleviate pressure on downstream resources. For example, if we have an event that needs to write information to a relational database AND trigger another process that needs to call a third-party API, this pattern would be a great fit.

This is a variation of the Distributed Trigger Pattern, but in this example, the SNS topic AND the SQS queues are contained within a single microservice. It is certainly possible to subscribe other microservices to this SNS topic as well, but we’ll stick with intra-service subscriptions for now. The diagram below represents a high-level view of how we might trigger an SNS topic (API Gateway → Lambda → SNS), with SNS then distributing the message to the SQS queues. Let’s call it the Distributed Queue Pattern.

Distributed Queue Pattern

This post assumes you know the basics of setting up a serverless application, and will focus on just the SNS topic subscriptions, permissions, and implementation best practices. Let’s get started!

Continue Reading…

🚀 Project Update:

Lambda API: v0.10 Released

Lambda API v0.10 adds the ability for you to seamlessly switch your Lambdas between API Gateway and Application Load Balancers. New execution stacks enables method-based middleware and more wildcard functionality. Plus full support for multi-value headers and multi-value query string parameters. Read More...

Stop Calling Everything Serverless!

I’ve been building serverless applications since AWS Lambda went GA in early 2015. I’m not saying that makes me an expert on the subject, but as I’ve watched the ecosystem mature and the community expand, I have formed some opinions around what it means exactly to be “serverless.” I often see tweets or articles that talk about serverless in a way that’s, let’s say, incompatible with my interpretation. This sometimes makes my blood boil, because I believe that “serverless” isn’t a buzzword, and that it actually stands for something important.

I’m sure that many people believe that this is just a semantic argument, but I disagree. When we refer to something as being “serverless”, there should be an agreed upon understanding of not only what that means, but also what it empowers you to do. If we continue to let marketers hijack the term, then it will become a buzzword with absolutely no discernible meaning whatsoever. In this post, we’ll look at how some leaders in the serverless space have defined it, I’ll add some of my thoughts, and then offer my own definition at the end.

Continue Reading…

Serverless Tip: Don’t overpay when waiting on remote API calls

Our serverless applications become a lot more interesting when they interact with third-party APIs like Twilio, SendGrid, Twitter, MailChimp, Stripe, IBM Watson and others. Most of these APIs respond relatively quickly (within a few hundred milliseconds or so), allowing us to include them in the execution of synchronous workflows (like our own API calls).  Sometimes we run these calls asynchronously as background tasks completely disconnected from any type of front end user experience.

Regardless how they’re executed, the Lambda functions calling them need to stay running while they wait for a response. Unfortunately, Step Functions don’t have a way to create HTTP requests and wait for a response. And even if they did, you’d at least have to pay for the cost of the transition, which can get a bit expensive at scale. This may not seem like a big deal on the surface, but depending on your memory configuration, the cost can really start to add up.

In this post we’ll look at the impact of memory configuration on the performance of remote API calls, run a cost analysis, and explore ways to optimize our Lambda functions to minimize cost and execution time when dealing with third-party APIs.

Continue Reading…

re:Capping re:Invent: AWS goes all-in on Serverless

Last week I spent six incredibly exhausting days in Las Vegas at the AWS re:Invent conference. More than 50,000 developers, partners, customers, and cloud enthusiasts came together to experience this annual event that continues to grow year after year. This was my first time attending, and while I wasn’t quite sure what to expect, I left with not just the feeling that I got my money’s worth, but that AWS is doing everything in their power to help customers like me succeed.

There have already been some really good wrap-up posts about the event. Take a look at James Beswick’s What I learned from AWS re:Invent 2018, Paul Swail’s What new use cases do the re:Invent 2018 serverless announcements open up?, and All the Serverless announcements at re:Invent 2018 from the Serverless, Inc. blog. There’s a lot of good analysis in these posts, so rather than simply rehash everything, I figured I touch on a few of the announcements that I think really matter. We’ll get to that in a minute, but first I want to point out a few things about Amazon Web Services that I learned this past week.

Continue Reading…

Aurora Serverless Data API: An (updated) First Look

Update June 5, 2019: The Data API team has released another update that adds improvements to the JSON serialization of the responses. Any unused type fields will be removed, which makes the response size 80+% smaller.

Update June 4, 2019: After playing around with the updated Data API, I found myself writing a few wrappers to handle parameter formation, transaction management, and response formatting. I ended up writing a full-blown client library for it. I call it the “Data API Client“, and it’s available now on GitHub and NPM.

Update May 31, 2019: AWS has released an updated version of the Data API (see here). There have been a number of improvements (especially to the speed, security, and transaction handling). I’ve updated this post to reflect the new changes/improvements.

On Tuesday, November 20, 2018, AWS announced the release of the new Aurora Serverless Data API. This has been a long awaited feature and has been at the top of many a person’s #awswishlist. As you can imagine, there was quite a bit of fanfare over this on Twitter.

Obviously, I too was excited. The prospect of not needing to use VPCs with Lambda functions to access an RDS database is pretty compelling. Think about all those cold start savings. Plus, connection management with serverless and RDBMS has been quite tricky. I even wrote an NPM package to help deal with the max_connections issue and the inevitable zombies 🧟‍♂️ roaming around your RDS cluster. So AWS’s RDS via HTTP seems like the perfect solution, right? Well, not so fast. 😞 (Update May 31, 2019: There have been a ton of improvements, so read the full post.)

Continue Reading…

🚀 Project Update:

Lambda API: v0.9.2 Released

Lambda API v0.9.2 has been released and contains additional updates and fixes for the index.d.ts TypeScript declarations file. Thanks again to @hassankhan and @Wintereise for submitting the changes. The release is immediately available via NPM. Read More...
🚀 Project Update:

Lambda API: v0.9.1 Released

Lambda API v0.9.1 has been released to include the index.d.ts TypeScript declarations file in the NPM package (thanks again, @hassankhan). The release is immediately available via NPM. Read More...
🚀 Project Update:

Lambda API: v0.9 Released

v0.9 adds new features to give developers better control over error handling and serialization. A TypeScript declaration file has also been added along with some additional API Gateway inputs that are now available in the REQUEST object. Read More...

Takeaways from ServerlessNYC 2018

I had the opportunity to attend ServerlessNYC this week (a ServerlessDays community conference) and had an absolutely amazing time. The conference was really well-organized (thanks Iguazio), the speakers were great, and I was able to have some very interesting (and enlightening) conversations with many attendees and presenters. In this post I’ve summarized some of the key takeaways from the event as well as provided some of my own thoughts.

Note: There were several talks that were focused on a specific product or service. While I found these talks to be very interesting, I didn’t include them in this post. I tried to cover the topics and lessons that can be applied to serverless in general.

Update November 16, 2018: Some videos have been posted, so I’ve provided the links to them.

Audio Version:

Continue Reading…

What 15 Minute Lambda Functions Tells Us About the Future of Serverless

Amazon Web Services recently announced that they increased the maximum execution time of Lambda functions from 5 to 15 minutes. In addition to this, they also introduced the new “Applications” menu in the Lambda Console, a tool that aggregates functions, resources, event sources and metrics based on services defined by SAM or CloudFormation templates. With AWS re:Invent just around the corner, I’m sure these announcements are just the tip of the iceberg with regards to AWS’s plans for Lambda and its suite of complementary managed services.

While these may seem like incremental improvements to the casual observer, they actually give us an interesting glimpse into the future of serverless computing. Cloud providers, especially AWS, continue to push the limits of what serverless can and should be. In this post, we’ll discuss why these two announcements represent significant progress into serverless becoming the dominant force in cloud computing.

Continue Reading…

🚀 Project Update:

Lambda API: v0.8.1 Released

Lambda API v0.8.1 has been released to patch an issue with middleware responses and a path prefixing options bug. The release is immediately available via NPM. Read More...

An Introduction to Serverless Microservices

Thinking about microservices, especially their communication patterns, can be a bit of a mind-bending experience for developers. The idea of splitting an application into several (if not hundreds of) independent services, can leave even the most experienced developer scratching their head and questioning their choices. Add serverless event-driven architecture into the mix, eliminating the idea of state between invocations, and introducing a new per function concurrency model that supports near limitless scaling, it’s not surprising that many developers find this confusing. 😕 But it doesn’t have to be. 😀

In this post, we’ll outline a few principles of microservices and then discuss how we might implement them using serverless. If you are familiar with microservices and how they communicate, this post should highlight how these patterns are adapted to fit a serverless model. If you’re new to microservices, hopefully you’ll get enough of the basics to start you on your serverless microservices journey. We’ll also touch on the idea of orchestration versus choreography and when one might be a better choice than the other with serverless architectures. I hope you’ll walk away from this realizing both the power of the serverless microservices approach and that the basic fundamentals are actually quite simple.  👊

Audio Version:

Continue Reading…

🚀 Project Update:

Serverless MySQL: v1.1.0 Released

Serverless MySQL v1.1.0 adds additional transaction support capabilities to allow users to retrieve interim query results for use with future queries. This is useful for getting the insertId from previous queries when performing transactions. Read More...

Serverless Security: Locking Down Your Apps with FunctionShield

I’ve written quite extensively about serverless security, and while you don’t need to be an expert on the matter, there are a number of common sense principles that every developer should know. Serverless infrastructures (specifically FaaS and managed services) certainly benefit from an increased security posture given that the cloud provider is handling things like software patching, network security, and to some extent, even DDoS mitigation. But at the end of the day, your application is only as secure as its weakest link, and with serverless, that pretty much always comes down to application layer security.

In this post we’re going to look at ways to mitigate some of these application layer security issues by using some simple strategies as well as a free tool called FunctionShield.

Audio Version:

Continue Reading…

Managing MySQL at Serverless Scale

“What? You can’t use MySQL with serverless functions, you’ll just exhaust all the connections as soon as it starts to scale! And what about zombie connections? Lambda doesn’t clean those up for you, meaning you’ll potentially have hundreds of sleeping threads blocking new connections and throwing errors. It can’t be done!”  ~ Naysayer

I really like DynamoDB and BigTable (even Cosmos DB is pretty cool), and for most of my serverless applications, they would be my first choice as a datastore. But I still have a love for relational databases, especially MySQL. It had always been my goto choice, perfect for building normalized data structures, enforcing declarative constants, providing referential integrity, and enabling ACID-compliant transactions. Plus the elegance of SQL (structured query language) makes organizing, retrieving and updating your data drop dead simple.

But now we have SERVERLESS. And Serverless functions (like AWS Lambda, Google Cloud Functions, and Azure Functions) scale almost infinitely by creating separate instances for each concurrent user. This is a MAJOR PROBLEM for RDBS solutions like MySQL, because available connections can be quickly maxed out by concurrent functions competing for access. Reusing database connections doesn’t help, and even the release of Aurora Serverless doesn’t solve the max_connections problem. Sure there are some tricks we can use to mitigate the problem, but ultimately, using MySQL with serverless is a massive headache.

Well, maybe not anymore. 😀 I’ve been dealing with MySQL scaling issues and serverless functions for years now, and I’ve finally incorporated all of my learning into a simple, easy to use NPM module that (I hope) will solve your Serverless MySQL problems.

Continue Reading…

Jeremy goes to AWS re:Invent 2018

It’s official! I’m going to AWS re:Invent 2018. 🙌

My goal from this trip is to learn, learn, learn… and then share, share, share.   There are over 30 sessions that talk about serverless, plus 40,000 other people there to meet and learn from! I’m so excited. 🙃

I know that many of you will be there, but for those of you who can’t be, I’ll do my best to share insights, tips, how-tos, best practices and more. I’ll even have a drink for you if you’d like 🍺 (no arm twisting necessary)!

Continue Reading…

Serverless Microservice Patterns for AWS

I’m a huge fan of building microservices with serverless systems. Serverless gives us the power to focus on just the code and our data without worrying about the maintenance and configuration of the underlying compute resources. Cloud providers (like AWS), also give us a huge number of managed services that we can stitch together to create incredibly powerful, and massively scalable serverless microservices.

I’ve read a lot of posts that mention serverless microservices, but they often don’t go into much detail. I feel like that can leave people confused and make it harder for them to implement their own solutions. Since I work with serverless microservices all the time, I figured I’d compile a list of design patterns and how to implement them in AWS. I came up with 19 of them, though I’m sure there are plenty more.

In this post we’ll look at all 19 in detail so that you can use them as templates to start designing your own serverless microservices.

Audio Version:

Continue Reading…

🚀 Project Update:

Lambda API: v0.8 Released

Lambda v0.8 is finally here and was well worth the wait! New features include allowing middleware to accept multiple handlers, new convenience methods for cache control and signing S3 URLs, and async/await support for the main function handler. And best of all, new LOGGING and SAMPLING support for you to add more observability into your APIs and web applications. Read More...

Aurora Serverless: The Good, the Bad and the Scalable

Amazon announced the General Availability of Aurora Serverless on August 9, 2018. I have been playing around with the preview of Aurora Serverless for a few months, and I must say that overall, I’m very impressed. There are A LOT of limitations with this first release, but I believe that Amazon will do what Amazon does best, and keep iterating until this thing is rock solid.

The announcement gives a great overview and the official User Guide is chock full of interesting and useful information, so I definitely suggest giving those a read. In this post, I want to dive a little bit deeper and discuss the pros and cons of Aurora Serverless. I also want to dig into some of the technical details, pricing comparisons, and look more closely at the limitations.

Audio Version

Continue Reading…

A Tale of Two Teams

Audio Version:

It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness… ~ A Tale of Two Cities by Charles Dickens

There is a revolution happening in the tech world. An emerging paradigm that’s letting development teams focus on business value instead of technical orchestration. It is helping teams create and iterate faster, without worrying about the limits or configurations of an underlying infrastructure. It is enabling the emergence of new tools and services that foster greater developer freedom. Freedom to experiment. Freedom to do more with less. Freedom to immediately create value by publishing their work without the traditional barriers created by operational limits.

Continue Reading…

How To: Add Test Coverage to your Serverless Applications

Writing serverless functions brings developers closer and closer to the stack that runs their code. While this gives them a tremendous amount of freedom, it also adds additional responsibility. Serverless applications require developers to think more about security and optimizations, as well as perform other tasks that were traditionally assigned to operations teams. And of course, code quality and proper testing continue to be at the top of the list for production-level applications. In this post, we’ll look at how to add test coverage to our Node.js applications and how we can apply it to our Serverless framework projects. ⚡️

Continue Reading…

Serverless Peeps You Need To Follow

In my never ending quest to consume all things serverless, I often find myself scouring the Interwebs for new and interesting serverless articles, blog posts, videos, and podcasts. There are more and more people doing fascinating things with serverless every day, so finding content is becoming easier and easier. However, this increase in content comes with an increase in noise as well. Cutting through that noise isn’t always easy. 🙉

Great content with valuable insights

I personally love reading articles that introduce new use cases or optimizations for serverless. Stories about companies using serverless in production and how their architectures are set up are also extremely interesting. I’ve been working in the serverless space for several years now, and have come across a number of people who produce and/or share really great content. I’ve put together a list of people that I follow and enjoy their content regularly. Hopefully these people will help you learn to love serverless as much as I do. ❤️⚡️

Continue Reading…

How To: Tag Your Lambda Functions for Smarter Serverless Applications

As our serverless applications start to grow in complexity and scope, we often find ourselves publishing dozens if not hundreds of functions to handle our expanding workloads. It’s no secret that serverless development workflows have been a challenge for a lot of organizations. Some best practices are starting to emerge, but many development teams are simply mixing their existing workflows with frameworks like Serverless and AWS SAM to build, test and deploy their serverless applications.

Beyond workflows, another challenge serverless developers encounter as their applications expand, is simply trying to keep all of their functions organized. You may have several functions and resources as part of a microservice contained in their own git repo. Or you might simply put all your functions in a single repository for better common library sharing. Regardless of how code is organized locally, much of that is lost when all your functions end up in a big long list in the AWS Lambda console. In this post we’ll look at how we can use AWS’s resource tagging as a way to apply structure to our deployed functions. This not only give us more insight into our applications, but can be used to apply Cost-Allocation Tags to our billing reports as well. 👍

Continue Reading…

Thinking Serverless (Big and Small)

I’ve been reading and writing a lot of about serverless lately, and one of the things I realized, is that most articles talk about how SCALABLE serverless architectures are. This, of course, is one of the major benefits of using serverless to build your applications. The ability to scale to thousands of concurrent requests per second without needing to manage your own servers, is simply amazing. 🙌

However, not needing to manage any servers has other benefits beyond the capabilities to achieve web scale. Having on-demand compute space also make serverless the perfect candidate for smaller workloads. In this post, let’s discuss how we can utilize serverless to handle our “less than unicorn 🦄” services and the benefits this can bring.

Continue Reading…

Lambda Warmer: Optimize AWS Lambda Function Cold Starts

At a recent AWS Startup Day event in Boston, MA, Chris Munns, the Senior Developer Advocate for Serverless at AWS, discussed Lambda cold starts and how to mitigate them. According to Chris (although he acknowledge that it is a “hack”) using the CloudWatch Events “ping” method is really the only way to do it right now. He gave a number of really good tips to pre-warm your functions “correctly”:

  • Don’t ping more often than every 5 minutes
  • Invoke the function directly (i.e. don’t use API Gateway to invoke it)
  • Pass in a test payload that can be identified as such
  • Create handler logic that replies accordingly without running the whole function

Continue Reading…

15 Key Takeaways from the Serverless Talk at AWS Startup Day

I love learning about the capabilities of AWS Lambda functions, and typically consume any article or piece of documentation I come across on the subject. When I heard that Chris Munns, Senior Developer Advocate for Serverless at AWS, was going to be speaking at AWS Startup Day in Boston, I was excited. I was able to attend his talk, The Best Practices and Hard Lessons Learned of Serverless Applications, and it was well worth it.

Chris said during his talk that all of the information he presented is on the AWS Serverless site. However, there is A LOT of information out there, so it was nice to have him consolidate it down for us into a 45 minute talk. There was some really insightful information shared and lots of great questions. I was aware of many of the topics discussed, but there were several clarifications and explanations (especially around the inner workings of Lambda) that were really helpful. 👍

Continue Reading…

Mixing VPC and Non-VPC Lambda Functions for Higher Performing Microservices

I came across a post the in the Serverless forums that asked how to disable the VPC for a single function within a Serverless project. This got me thinking about how other people structure their serverless microservices, so I wanted to throw out some ideas. I often mix my Lambda functions between VPC and non-VPC depending on their use and data requirements. In this post, I’ll outline some ways you can structure your Lambda microservices to isolate services, make execution faster, and maybe even save you some money. ⚡️💰

Continue Reading…

5 Reasons Why Your Serverless Application Might Be A Security Risk

There has been a lot of buzz lately about serverless security. People are certainly talking about it more and sharing great articles on the topic, but many serverless developers (especially new ones) are still making the same critical mistakes. Every time a serverless function is deployed, its unique security challenges need to be addressed. Every time. I’ve researched and written extensively about serverless security (see Securing Serverless: A Newbie’s Guide). I’ve read countless articles on the subject. And while there is no shortage of information available, let’s be honest: developers are busy building applications, not pouring through hundreds of articles.

I know, it sounds boring, but I would encourage you to do your research on serverless security. Serverless applications are different than traditional, server-hosted applications. Much of the security responsibility falls on the developer, and not following best practices opens you (or your company) up to an attack. But I know you’re busy. I totally get it. So rather than forcing you to read a bunch of long articles 😴 or watch a plethora of videos 🙈, I’ve whittled it all down to the five biggest serverless security risks for you. Sure, there are a lot of other things to consider, but IMO, these are the most important ones. Nothing here hasn’t been said before. But If you do nothing more than follow these principles, your serverless applications will be much more secure. 🔒

Continue Reading…

Serverless Consumers with Lambda and SQS Triggers

Update November 19, 2019: AWS announced support for SQS FIFO queues as a Lambda event source (announcement here). FIFO queues guarantee message order, which means only one Lambda function is invoked per MessageGroupId.

On Wednesday, June 27, 2018, Amazon Web Services released SQS triggers for Lambda functions. Those of you who have been building serverless applications with AWS Lambda probably know how big of a deal this is. Until now, the AWS Simple Queue Service (SQS) was generally a pain to deal with for serverless applications. Communicating with SQS is simple and straightforward, but there was no way to automatically consume messages without implementing a series of hacks. In general, these hacks “worked” and were fairly manageable. However, as your services became more complex, dealing with concurrency and managing fan out made your applications brittle and error prone. SQS triggers solve all of these problems. 👊

Update December 6, 2018: At some point over the last few months AWS fixed the issue with the concurrency limits and the redrive policy. See Additional experiments with concurrency and redrive polices below.

Audio Version (please note that this audio version is out of date given the new updates)

Continue Reading…

Event Injection: Protecting your Serverless Applications

Updated January 25, 2019: This post was updated based on feedback from the community.

The shared security model of cloud providers extends much further with serverless offerings, but application security is still the developer’s responsibility. Many traditional web applications are front-ended with WAFs (web application firewalls), RASPs (runtime application self-protection), EPPs (endpoint protection platforms) and WSGs (web security gateways) that inspect incoming and outgoing traffic. These extra layers of protection can save developers from themselves when making common programming mistakes that would otherwise leave their applications vulnerable. If you’re invoking serverless functions from sources other than API Gateway, you no longer have the ability to use the protection of a WAF. 

Continue Reading…

10 Things You Need To Know When Building Serverless Applications

I am a HUGE fan of serverless architectures. This new type of compute not only opens up more possibilities for developers, but can support highly-scalable, complex applications for a fraction of the cost compared to provisioning virtual servers. My first question when planning a new application is always, “Can I build this with serverless?” Spoiler alert, the answer is almost always YES!

I’ve been building serverless applications since the release of AWS Lambda in 2015, so answering the question above is pretty easy for me. However, a lot of people I talk to who are new to serverless often have many questions (and misconceptions). I want you to be successful, so below I’ve create a list of 10 things you need to know when building a serverless application. These are things I wish I knew when I started, so hopefully they’ll help you get up to speed a faster and start building some amazing applications.

Continue Reading…

🚀 Project Update:

Lambda API: v0.7 Released

v0.7 adds new features to control middleware execution based on path, plus additional parsing of the AWS Lambda context object. ESLint and coverage reports using Istanbul and Coveralls were also added to ensure code quality and adequate test coverage. Read More...

How To: Optimize the Serverless Optimizer Plugin

I’m sure you’re already well aware of how awesome the ⚡ Serverless Framework is for managing and deploying your serverless applications. And you’re probably aware that there are several great plugins available that make Serverless even better. But did you know that there was a plugin to optimize your functions and reduce the size of your deployment packages? Or are you already using this plugin to optimize your functions, but hate how it takes too long to optimize locally run functions? In this post I’ll share some quick tips to help you optimize your Serverless Optimizer experience.

Continue Reading…

Transducers: Supercharge your functional JavaScript

This is the first in a series of posts on functional programming in JavaScript. My goal is to make these ideas more accessible to all levels of programmers. Feedback about style, content, etc., would all be greatly appreciated.

One thing that perplexed me early on in my functional programming days was the concept of transducers. I spent a lot of time Googling and found some great articles that went deep into the theory and the underlying mechanics. However, the practical use of them still seemed a bit out of reach. In this post I’ll attempt to explain transducers in a more understandable way and hopefully give you the confidence to use them in your functional JavaScript. While this article attempts to make transducers more accessible, you will need to have some basic knowledge of functional programming in JavaScript. Specifically, you should know about function composition and iterator functions like .map(), .filter(), and most importantly, .reduce(). If you are unfamiliar with these concepts, go get a grasp on them first.

Continue Reading…

🚀 Project Update:

Lambda API: v0.6 Released

v0.6 is all about making the serverless developer's life easier! New support for both callback-style and async-await in route functions and middleware, new HTTP method routing features, and route debugging tools. Plus Etag support and automatic authorization parsing. Read More...

Solving the Cold Start Problem

Dear AWS Lambda Team,

I have a serious problem: I love AWS Lambda! In fact, I love it so much that I’ve pretty much gone all in on this whole #serverless thing. I use Lambda for almost everything now. I use it to build backend data processing pipelines, distribute long running tasks, and respond to API requests. Heck, I even built an Alexa app just for fun. I found myself building so many RESTful APIs using Lambda and API Gateway that I went ahead and created the open source Lambda API web framework to allow users to more efficiently route and respond to API Gateway requests.

Serverless technologies, like Lambda, have revolutionized how developers think about building applications. Abstracting away the underlying compute layer and replacing it with on-demand, near-infinitely scalable function containers is brilliant. As we would say out here in Boston, “you guys are wicked smaht.” But I think you missed something very important. In your efforts to conform to the “pay only for the compute time you consume” promise of serverless, you inadvertently handicapped the service. My biggest complaint, and the number one objection that I hear from most of the “serverless-is-not-ready-for-primetime” naysayers, are Cold Starts.

Continue Reading…

How To: Manage Serverless Environment Variables Per Stage

I often find myself creating four separate stages for each ⚡ Serverless Framework project I work on: dev, staging, prod, and local. Obviously the first three are meant to be deployed to the cloud, but the last one, local, is meant to run and test interactions with local resources. It’s also great to have an offline version (like when you’re on a plane ✈ or have terrible wifi somewhere). Plus, development is much faster because you’re not waiting for round trips to the server. 😉

A really great feature of Serverless is the ability to configure ENVIRONMENT variables in the serverless.yml file. This lets us store important global information like database names, service endpoints and more. We can even reference passwords securely using AWS’s Service Manager Parameter Store and decode encrypted secrets on deployment, keeping them safe from developers and source repositories alike. 😬 Just reference the variable with ${ssm:/myapp/my-secure-value~true} in your configuration file.

Continue Reading…

🚀 Project Update:

Lambda API: v0.5 Released

v0.5 takes advantage of AWS Lambda's recently released support for Node v8.10 and has removed its Bluebird promise dependency in favor of async/await. Lambda API is now faster and adds built-in CORS support, additional wildcard features, new HTTP header management methods and more. Read More...

How To: Stub “.promise()” in AWS-SDK Node.js

Since AWS released support for Node v8.10 in Lambda, I was able to refactor Lambda API to use async/await instead of Bluebird promises. The code is not only much cleaner now, but I was able to remove a lot of unnecessary overhead as well. As part of the refactoring, I decided to use AWS-SDK’s native promise implementation by appending .promise() to the end of an S3 getObject call. This works perfectly in production and the code is super compact and simple:

The issue came with stubbing the call using Sinon.js. With the old promise method, I was using promisifyAll() to wrap new AWS.S3() and then stubbing the getObjectAsync method. If you’re not familiar with stubbing AWS services, read my post: How To: Stub AWS Services in Lambda Functions using Serverless, Sinon.JS and Promises.

Continue Reading…

How To: Manage RDS Connections from AWS Lambda Serverless Functions

Someone asked a great question on my How To: Reuse Database Connections in AWS Lambda post about how to end the unused connections left over by expired Lambda functions:

I’m playing around with AWS lambda and connections to an RDS database and am finding that for the containers that are not reused the connection remains. I found before that sometimes the connections would just die eventually. I was wondering, is there some way to manage and/or end the connections without needing to wait for them to end on their own? The main issue I’m worried about is that these unused connections would remain for an excessive amount of time and prevent new connections that will actually be used from being made due to the limit on the number of connections.

🧟‍♂️ Zombie RDS connections leftover on container expiration can become a problem when you start to reach a high number of concurrent Lambda executions. My guess is that this is why AWS is launching Aurora Serverless, to deal with relational databases at scale. At the time of this writing it is still in preview mode.

Update September 2, 2018: I wrote an NPM module that manages MySQL connections for you in serverless environments. Check it out here.

Update August 9, 2018: Aurora Serverless is now Generally Available!

Overall, I’ve found that Lambda is pretty good about closing database connections when the container expires, but even if it does it reliably, it still doesn’t solve the MAX CONNECTIONS problem. Here are several strategies that I’ve used to deal with this issue.

Continue Reading…

Is Code Really Self-Documenting?

In my 20+ years of programming, I’ve encountered a near endless amount of opinions on everything from coding styles to programming paradigms to the great whitespace debate. Obviously, I have strong opinions on a number of these. But for me, the one that bothers me the most is this notion that “code is self-documenting.” 😾

I know what you’re probably thinking: “of course not all code is self-documenting, only well-written code is.” I don’t entirely disagree. I can generally look at someone else’s code and understand exactly WHAT it is doing. However, often it’s not obvious WHY they did it that way, or even why they did it in the first place. In my opinion, the programmer’s intent (the WHY) is just as important as the HOW when it comes to properly documenting software.

So whether you agree with me or not, let’s explore how to better document our software by writing cleaner code, following some general commenting etiquette, and commenting more effectively to make you and your team more productive. 👍

Continue Reading…

Securing Serverless: A Newbie’s Guide

So you’ve decided to build a serverless application. That’s awesome! May I be the first to welcome you to the future. 🤖 I bet you’ve done a lot of research. You’ve probably even deployed a few test functions to AWS Lambda or Google Cloud Functions and you’re ready to actually build something useful. You probably still have a bunch of unanswered questions, and that’s cool. We can still build some really great applications even if we only know the basics. However, when we start working with new things we typically make a bunch of dumb mistakes. While some are relatively innocuous, security mistakes can cause some serious damage.

I’ve been working with serverless applications since AWS launched Lambda in early 2015. Over the last few years I’ve developed many serverless applications covering a wide range of use cases. The most important thing I’ve learned: SECURE YOUR FUNCTIONS! I can tell you from personal experience, getting burned by an attack is no bueno. I’d hate to see it happen to you. 😢

To make sure it doesn’t happen to you, I’ve put together a list of 🔒Serverless Security Best Practices. This is not a comprehensive list, but it covers the things you ABSOLUTELY must do. I also give you some more things to think about as you continue on your serverless journey. 🚀

Continue Reading…

How To: Build a Serverless API with Serverless, AWS Lambda and Lambda API

AWS Lambda and AWS API Gateway have made creating serverless APIs extremely easy. Developers can simply create Lambda functions, configure an API Gateway, and start responding to RESTful endpoint calls. While this all seems pretty straightforward on the surface, there are plenty of pitfalls that can make working with these services frustrating.

There are, for example, lots of confusing and conflicting configurations in API Gateway.  Managing deployments and resources can be tricky, especially when publishing to multiple stages (e.g. dev, staging, prod, etc.). Even structuring your application code and dependencies can be difficult to wrap your head around when working with multiple functions.

In this post I’m going to show you how to setup and deploy a serverless API using the Serverless framework and Lambda API, a lightweight web framework for your serverless applications using AWS Lambda and API Gateway. We’ll create some sample routes, handle CORS, and discuss managing authentication. Let’s get started.

Continue Reading…

Off-by-none: Issue #67

re:Covering from re:Invent… 🤯

Welcome to Issue #67 of Off-by-none. I’m so glad you joined us.

Last week, I was at AWS re:Invent in Las Vegas. It was a five day assault on the senses, but loaded with great announcements and amazing people! In this issue, we’ll look at a few of the big serverless announcements from the big show, and as always, share plenty of content from the serverless community.

Let’s get to it! 👍

When you want someone to catch you up on everything that happened at re:Invent 2019… 🚂

Explore all the new features and services announced at AWS re:Invent 2019
Who better to get all the info from re:Invent from than AWS themselves? Every year AWS puts together a site that wraps up all the major announcements and even makes them searchable. So if you want a source of truth, here it is.

Takeaways from AWS re:Invent 2019’s Amazon DynamoDB Deep Dive: Advanced Design Patterns (DAT403)
Rick Houlihan delivered another mind-bending edition of his Advanced Design Patterns for DynamoDB talk at re:Invent this year. I had some thoughts on it and provided 12 key takeaways.

AWS re:Invent 2019 – Serverless Announcements Recap
Renato Byrro covers all the major serverless announcements in this post. It also includes some links to more detailed posts about some of the items.

Amazon RDS Proxy makes it easier to use SQL in Serverless
Alex DeBrie from Serverless, Inc. shares why the Amazon RDS Proxy is an important improvement for lots of serverless users. He argues (rightfully so, IMO) that RDBMS starts to lose a bit of the serverless feel, and maybe DynamoDB is still a better first choice.

re:Invent 2019 – AWS API Gateway v2 for HTTP
This was another release that had several people talking. Fernando Medina Corey explains what is changing with resources, authorizers and CORS.

When you really want Provisioned Concurrency… ⏱

There was a ton of buzz around the release of Provisioned Concurrency. So much so, that I think everyone wrote something about it. Yan Cui’s post, It’s the end of the cold start as we know it, is definitely my favorite. There were several others as well, all with varying perspectives:

Thinking outside the box, as he tends to do, Michael Hart figured out some insane trickery to use provisioned currency and the 10 seconds of free init time to save you a ton of money. Even though this works, Michael suggests not using it for production. It’s worth the read: Shave 99.93% off your Lambda bill with this one weird trick.

Serverless News & Announcements 📣

Launching Webiny Serverless Form Builder
The team at Webiny continues to add improvements to their platform. The latest enhancement brings us a serverless form builder that supports everything from simple contact forms, to more complex lead generation forms.

Azure Functions 3.0 go-live release is now available
The go-live release for Azure Functions 3.0 is now available, so it’s now possible to build and deploy 3.0 functions in production. Functions 3.0 brings new capabilities including the ability to target .NET Core 3.1 and Node 12.

Browser Functions: A new serverless platform using Web Browser execution engines
This is pretty cool. The project still looks a bit early, and there are several limitations, but the concept is very interesting.

Serverless Use Cases 🗺

Halving our AWS Lambda bill with parallel processing in Python
There are a lot of use cases like this where sometimes little tricks can end up saving a lot of time and money. In a perfect world, these Lambda functions could just be run as separate parallel processes, but given the current billing model, I think “hacks” like this work just fine.

Bringing our Comic Relief Values to life with IoT, Amplify and Serverless
I love the idea of this use case. Anything that gives you a chance to play with IoT toys and serverless has got to be worth it, right? I’m sure there are ways to adapt this use case using just the buttons, but with more sophisticate IoT devices, the possibilities are endless.

Serverless Concepts 🏗

Serverless Microfrontends in AWS
I keep hearing more and more about “microfrontends”, and in this article, Ashan Fernando specifies some ways to host them using serverless AWS technology.

Fine-tuning your Lambdas
Catherine Shen has a great post that outlines some ways to troubleshoot and reduce the iterator age of your Kinesis streams.

Migration to Serverless – microservices use case
Serkan Özal walks you through a hypothetical migration to serverless. I always like to see how people envision serverless representations of microservices.

Awesome DynamoDB ?
Interested in all things DynamoDB? Alex DeBrie keeps trying to one up me and win Rick Houlihan’s favor. This time he created an Awesome DynamoDB repo on GitHub with links to all the greatest DynamoDB resources. There are a lot of very useful links in here.

How Do You Test Serverless Functions?
Lots of things stay the same when testing serverless applications, but it’s also possible to do some pretty amazing things. John Gilbert outlines the tools and processes he uses to test his serverless applications.

Serverless Tutorials 🏗

Microservices with Serverless & AWS Lambda Layers
Markus Wagner shows you how to build a microservice using Lambda Layers to store and share your dependencies. I think Lambda Layers work really well within a single microservice, but not so well when you start sharing them across microservices. This is mainly due to the way they’re versioned.

Building a serverless GraphQL API with Node.js, AWS Lambda and Apollo
Not everyone wants to use AppSync (although you should definitely look at it if you haven’t yet), so sometimes building your own serverless Apollo server makes sense. Thomas Maximini gives you all the details in his post.

Using system packages and custom binaries in Google Cloud Functions.
Google Cloud Functions and Lambda share some similarities, one of them being that they are just running on top of an OS that can execute custom binaries and system packages. We’ve seen a lot of posts about this for Lambda, but this is the first one that caught my eye on GCP.

How to model Amazon DynamoDB databases with NoSQL Workbench
Ratul Saha uses the example given in Alex DeBrie’s excellent talk from re:Invent to show us how to use the new NoSQL Workbench for DynamoDB.

Serverless Reads 🤓

Why Container Devs Need to Care about re:Invent 2019’s Serverless Launches
Strap in for Tim Wagner’s take on what all the serverless launches at re:Invent mean, why they’re important to the adoption of serverless, and what was missing.

Five Things to Take Away From re:Invent 2019
Nothing really about serverless in here, but an important read nonetheless.

The M-shaped software
If you haven’t been paying attention to Web Assembly, now might be the time. Michael Yuan explains why WASM could be the middleware that connects software with specialized hardware.

What I’ve Learned in 4 years at AWS re:Invent
Thinking about going to re:Invent next year? This piece from Nick Gottlieb sums up re:Invent perfectly, so it might convince you, or change your mind.

Lambda Destinations for synchronous invocations · Issue #1 · stojanovic/random
Slobodan Stojanovic put together a compelling argument for letting Lambda Destinations handle “synchronous” invocations as well. Lots of interesting use cases in here with some insights as to how they solve them now.

Reflecting on re:Invent — The Human Side of Serverless
If you’ve never met Farrah Campbell from Stackery, make a point of it. She’s a wonderful human being that is doing a world of good for the serverless community. She also makes a great co-host for serverless parties at re:Invent. 😉

When you prefer a multimedia experience… 📽

Serverless Chats – Episode #26: re:Inventing Serverless with Chris Munns
In this episode, I chat with Chris Munns about all the new serverless product releases from AWS re:Invent 2019, the ongoing feature improvements AWS continues to make, and how his team plans to bring serverless to everyone in 2020.

Where Does Serverless Fit in Cloud Native?
Shayne Boyer talks with Jeff Hollan about how serverless fits into this whole cloud native thing. There is a healthy mix of skepticism from Shayne, but Jeff does a great job laying out the benefits and discussing the tools that are available through Azure to make it easier.

When the teams at AWS can finally get some sleep… 🛏

Amazon API Gateway Offers Faster, Cheaper, Simpler APIs Using HTTP APIs (Preview)
HTTP APIs are BIG! Besides the massive cost reduction, the latency is much lower. There are limitations with this new product, but I think we’ll see those eased over time. And for the majority of use cases, this will work just fine.

AWS Lambda announces Provisioned Concurrency
Here is another big one. I think it’s a stop gap, trying to provide counter-arguments to the serverless naysayers, but there will be use cases for this that could open up a whole new set of low-latency, synchronous applications.

Introducing Amazon RDS Proxy (Preview)
I want my, I want my, I want MySQL. I still love RDBMS, as do millions of developers around the world. So the ability to use connection pooling from Lambda functions (so we don’t flood our databases with unnecessary connections) is a game-changer. Expect to see lots of other RDS-compliant database engines working with this soon.

Introducing AWS Step Functions Express Workflows
I’ve shied away from Step Functions for many use cases simply because of the cost and throughput limitations. Express Workflows changes that (with some caveats, of course), making orchestration, function composition, and other state machine capabilities much more accessible.

Introducing the Amplify DataStore, a persistent storage engine that synchronizes data between apps and the cloud
A queryable, on-device data store for web, IoT, and mobile developers using iOS, Android, and React Native. Plus it has some amazingly smart cloud synching.

Introducing Amazon SageMaker Studio – the first integrated development environment (IDE) for machine learning
If you’ve been puzzled by machine learning, this new SageMaker Studio could be your saving grace. I haven’t tried it yet, but the docs make it look incredibly easy. Just be careful not to accidentally build Skynet with it. 🤖

AWS announces UltraWarm (preview) for Amazon Elasticsearch Service
It’s still not serverless Elasticsearch, but it is a big improvement to the daunting costs that was the typical hot storage. The analysis capabilities on ES are great, so it might be worth taking another look if you dismissed it in the past.

Introducing The Amazon Builders’ Library
No excuses. Drop everything that you are doing and go read these documents. Your future self will thank you.

AWS launches Fargate Spot, save up to 70% for fault tolerant applications
This might be as important an announcement as spot instances were for EC2. You need to orchestrate them with ECS, but for the right kinds of workloads, this could be very, very useful.

Thoughts from Twitter 🐦

Here’s a YouTube playlist of DynamoDB-related sessions from AWS re:Invent 2019… And we’re waiting for a few more videos to be uploaded. ~ @DynamoDB
There were a bunch of great session on DynamoDB at re:Invent this year. If you love DynamoDB (or want to), this YouTube playlist will give you a ton of insight.

If you’re looking to get started with #serverless, I’ve put together a body of work over the past year that starts with functions & expands much deeper, going into philosophy & building serverless apps leveraging things like auth, db, & storage. Check out these resources… ~Nader Dabit
Nader Dabit had a good thread that links to all the resources he’s worked on over the years about serverless. Lots of really great stuff in here.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

December 13, 2019 – AWS re:Invent Recap – Serverless Edition (Webinar)

December 14, 2019 – ServerlessDays Fukuoka Japan

December 20, 2019 – Using Relational Databases with AWS Lambda – Easy Connection Pooling (Webinar)

January 24, 2020 – ServerlessDays Belfast

February 13, 2020 – ServerlessDays Cardiff

February 21, 2020 – ServerlessDays Rome

February 27, 2020 – ServerlessDays Nashville

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Eric Hammond (@esh). Eric is one of the OG AWS Serverless Heroes and a legend among early AWS adopters with his work on community AMIs and the creation of user-data scripts for EC2. Way back when serverless first became a thing, Eric started writing and sharing serverless tips and insights via his blog, GitHub and Twitter. Five years later, he’s still front and center at all the most important serverless re:Invent talks, live tweeting and sharing his knowledge with the community. But the thing I really love about Eric’s work, is just how thoughtful his posts and Tweets are. He constantly shares other people’s work and builds people up. Thank you for all you do, Eric. We need more of this in the world! 🙌

Final Thoughts 🤔

This was only my second re:Invent, but I somehow ended up being even more exhausted this year! I met with a number of great AWS teams and am now even more excited about the future of serverless. EventBridge is becoming an absolute staple, Lambda Destinations is moving us closer to the promise of “only writing business logic”, and new products, like RDS Proxy, Express Workflows, and HTTP APIs, are starting to open up a whole new world of possibilities. There is a lot of complexity being added with all this, but 2020 could be the breakout year for better abstractions.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Take care,
Jeremy

Off-by-none: Issue #66

Live from AWS re:Invent… 🎥

Welcome to Issue #66 of Off-by-none. Thanks for join us! 🙌

Last week, we looked at the firehose of AWS pre:Invent announcements. This week, I’m at re:Invent and still trying to digest all of the recent launches and updates while also trying to keep up with all the new ones. In this issue, I’ve included some of the more exciting new AWS announcements, and included a bunch of content from the serverless community.

I also want to thank the sponsors of the #ServerlessForEveryone Community Party @ AWS re:Invent 2019 again. StackeryThundra, AWS, CloudZero, Lumigo, Protego (now Check Point), Serverless, Inc., Edrans, and New Relic Serverless have made this entire event possible.

Let’s jump in!

When you realize that you’ve been really busy… 🏃‍♂️

I’ve was running around like crazy last week trying to finish up some projects before re:Invent. I did end up publishing a beta version of the new DynamoDB Toolbox that gives you a whole set of tools that makes it much easier to work with data in DynamoDB. Take a look and let me know what you think.

Since I’ve been at re:Invent, I managed to meet up with several people, and share some thoughts on the EventBridge Schema Registry that was announced at Midnight Madness. Lots of interesting possibilities with this.

I also got a chance to attend a few sessions yesterday, including Rick Houlihan’s DynamoDB Advanced Modeling Session. Lots of amazing information in there, so I recapped some important takeaways and lessons in a Twitter thread for you.

Serverless News & Product Announcements 📣

All the Serverless announcements at re:Invent 2019
There are a ton of serverless announcements at re:Invent, so luckily the team over at Serverless, Inc. is compiling them all for us!

Check Point Extends Leadership in Cloud Security with Unmatched Serverless Protection
Check Point Software Technologies has acquired Protego. Congrats to TJ, Hillel, and the rest of the team.

Merry and Bright with Azure Advocates’ 25 Days of Serverless
I like challenges that engage developers like this. Microsoft is running a serverless coding challenge every day for 25 days. Solve it in the programming language of your choice and submit your solution via GitHub. Then they’ll showcase the best solutions every week.

Webiny – November Update
Lots of cool things happening in the serverless CMS world. The team at Webiny published a summary of the highlights and new things they launched in the last few weeks.

Serverless Concepts 🏗

Lessons learned using Single-table design with DynamoDB and GraphQL in production
Great post by Rafal Wilinski that recounts some of the important lessons learned when building single-table designs in DynamoDB.

Our Serverless Blueprint: The step-by-step guide
Ivan Breet outlines how his team at Simply Anvil creates and structures serverless applications.

Couple of Minutes: Serverless Microservices Decomposition
This short post by Loganathan Murugesan highlights an important concept when building microservices with serverless. Using the single responsibility principle, it’s likely that a single Lambda does not provide an entire business capability. Serverless microservices are structured as a collection of Lambdas and other resources that work together to provide the capability.

Fast Cloudfront log queries using AWS Athena and Serverless
Great post by Ben Hoyt that shows the power Athena and a simple Lambda function have to make querying massive amounts of log data fast and inexpensive.

Serverless Reads 🤓

Adrian’s top AWS updates
Adrian Hornsby summarizes some of his favorite announcements from AWS over the last few weeks.

A “Less Server” Data Infrastructure Solution for Ingestion and Transformation Pipelines
Michael Triska outlines how serverless architecture patterns and services like Snowflake, AWS Glue, and AWS Fargate have changed the way we build ETL pipelines.

How DynamoDB Is Gaining Popularity In The Developer Community
Nothing incredibly insightful in here, but this is a trend we’re starting to see. And of course, I think it’s a good thing.

EventBridge Schema Registry — what it is and why it matters for Serverless applications
Alex DeBrie has some thoughts on the new EventBridge Schema Registry.

Microsoft, not Amazon, is going to win the cloud wars
I think there is plenty of market share to spread around in the public cloud business. I like what Microsoft is doing with its serverless offerings, but IMO, the winners of the “cloud wars” will be consumer choice. Multi-cloud is going to mean something a lot different in a few years, and simply supporting deployments, won’t be a winning strategy.

For the audio enthusiast… 🎧

Episode #25: Using Serverless to Transform Careers and Communities with Farrah Campbell and Danielle Heberling
In this episode, I chat with Farrah Campbell and Danielle Heberling about how they found their way into tech, how serverless connected them, and the serverless project they built to help expand the community.

Just a handful of AWS re:Invent announcements…

Introducing Amazon EventBridge schema registry and discovery – In preview
These are new developer tool features, which allow you to automatically find events and their structure, or schema, and store them in a shared central location.

AWS Lambda Now Supports Maximum Event Age and Maximum Retry Attempts for Asynchronous Invocations
AWS Lambda now supports two new features to provide developers additional controls on how to process asynchronous invocations: Maximum Event Age and Maximum Retry Attempts.

Amazon Relational Database Service (RDS) Data API Client Library Supports Java (Preview)
You can use the Amazon Relational Database Service (Amazon RDS) Data API Client Library with support for Java, now available in preview, to quickly and easily build applications for Amazon Aurora Serverless.

Amazon SageMaker Autopilot – Automatically Create High-Quality Machine Learning Models With Full Control And Visibility
Amazon SageMaker Autopilot automatically creates the best classification and regression machine learning models, while allowing full control and visibility.

Introducing Amazon Detective
Amazon Detective is a new service in Preview that makes it easy to analyze, investigate, and quickly identify the root cause of potential security issues or suspicious activities.

Introducing AWS Identity and Access Management (IAM) Access Analyzer
AWS Identity and Access Management (IAM) Access Analyzer is a new feature that makes it simple for security teams and administrators to check that their policies provide only the intended access to resources.

AWS announces Amazon CodeGuru for automated code reviews and application performance recommendations
Amazon CodeGuru is a new machine learning service for development teams who want to automate code reviews, identify the most expensive lines of code in their applications, and receive intelligent recommendations on how to fix or improve their code.

Announcing Amazon Managed Apache Cassandra Service – Now in Preview
Amazon Managed Apache Cassandra Service (MCS) is a scalable, highly available, and managed Apache Cassandra-compatible database service. Amazon MCS is serverless, so you pay for only the resources you use and the service automatically scales tables up and down in response to application traffic.

Introducing Access Analyzer for Amazon S3 to review access policies
Access Analyzer for S3 is a new feature that monitors your access policies, ensuring that the policies provide only the intended access to your S3 resources.

Run Serverless Kubernetes Pods Using Amazon EKS and AWS Fargate
You can now use Amazon Elastic Kubernetes Service (EKS) to run Kubernetes pods on AWS Fargate, the serverless compute engine built for containers on AWS. This makes it easier than ever to build and run your Kubernetes applications in the AWS cloud.

Introducing the Amazon EventBridge Schema Registry – Now In Preview
The Amazon EventBridge schema registry stores event structure – or schema – in a shared central location and maps those schemas to code for Java, Python, and Typescript so it’s easy to use events as objects in your code.

Announcing UltraWarm (Preview) for Amazon Elasticsearch Service
UltraWarm is a fully managed, low-cost, warm storage tier for Amazon Elasticsearch Service.

Thoughts from Twitter 🐦

To be clear: I don’t advocate single-table DDB for nearly the breadth of cases that the AWS docs seem to (and the linked post has some helpful context on why). I think it’s a useful design pattern for a well-defined, narrow-ish workload, but pretty far up the hierarchy of needs. ~ Forrest Brazeal
I appreciate the different perspectives on this, but I’m still sticking with #TeamSingleTable. I agree with Forrest on many points, but as Rick Houlihan has shown us over and over again, this design pattern supports a very broad set of workloads (if you can understanding the modeling behind it, that is).

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

December 3, 2019 – Taco Tuesday with the AWS Heroes

December 4, 2019 – #ServerlessForEveryone Community Party @ AWS re:Invent 2019

December 14, 2019 – ServerlessDays Fukuoka Japan

Final Thoughts 🤔

re:Invent is keeping me extremely busy, so pardon the lightness of this week’s newsletter. Next week we’ll get back to normal.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Find me at re:Invent,
Jeremy

Off-by-none: Issue #65

Serverless is a journey, not an event destination… 🪐

Welcome to Issue #65 of Off-by-none. It’s great to have you here! 😎

Last week, we looked at the first major batch of announcements and feature launches leading up to re:Invent. This week, there were about a thousand more (give or take 😉)! We’ll try and cull that list down for you, plus we still have plenty of content from the serverless community.

Before we jump in, I just want to thank the sponsors of the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. StackeryThundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless have made this entire event possible.

Okay, now prepare for serverless announcement overload! 🤯

Guides to at Serverless re:Invent 🗺

re:Invent is going to be a nonstop onslaught of information, technical sessions, product announcements, parties and more. Here are some guides to help you plan your time. (BTW, if you’re not going to re:Invent, take note of some of these sessions and watch the videos once they’re posted.)

AWS re:Invent Serverless Survival Guide
Trisha Paine provides a list of can’t miss re:Invent sessions for developers, cloud architects, and security folks.

re:Invent Guide: Advanced Serverless
I’m including Eric Hammond’s excellent guide to Advanced Serverless sessions again. Great stuff in here.

Serverless at re:Invent
I had a chat with Farrah Campbell, Chase Douglas, Serhat Can, and Emrah Şamdan about some of our favorite upcoming re:Invent sessions.

re:Invent Guide: Databases
This isn’t exclusively serverless, but there are a few DynamoDB sessions in here that should be super interesting.

Serverless Stories 📖

How Small Startups Move Incredibly Fast With Serverless
Serverless and startups go together like peanut butter and jelly. Dadi Atar recounts why serverless was the right decision for his latest startup journey.

Serverless Big Data: lessons learned building an event ingestion pipeline on Amazon Web Services
Luca Bianchi dives into the details of building a big data ingestion pipeline with serverless.

Story of serverless (Lambda) to move large data from DynamoDB
Always love seeing experiments like this. Bala Dutt and Sachin Maheshwari discuss their experiment that got to processing 7k records per second using DynamoDB and Lambda.

Using CodePipeline to Automate Serverless Applications Deployment
Pedro Lourenço shows you how to build a fully automated CI/CD pipeline with CodePipeline and CodeBuild. He then explains why after using it for six months, his team outgrew it.

Serverless Concepts 🏗

How to access VPC and internet resources from Lambda without paying for a NAT Gateway
Paul Swail points out a workaround for synchronous Lambda workflows needing VPC access and Internet access. Lambdas calling Lambdas, yup, sometimes it just makes sense.

What are Serverless Components?
Ryan Jones has a series of articles on Serverless Components from the makers of the Serverless Framework. These are definitely worth knowing about. Also see: What are the benefits of Serverless Components? and How to use Serverless Components?

Take the legwork out of API Gateway troubleshooting
This is a great new feature in Serverless Framework Pro. Verne Lindner explains why tracking down API Gateway errors can be hard, and how the new request explorer makes corresponding logs easier to find.

The State of AWS Lambda Supported Languages & Runtimes (Updated November 2019)
AWS has released several new Lambda runtimes, Fernando Medina Corey gets you up to speed on all the latest ones.

Data Analytics with Serverless Services
Emrah Samdan outlines the pros and cons of building a serverless analytics pipeline.

Serverless Patterns implemented: using an SQS queue as a DLQ for a SNS topic
Vicenç García has implemented another pattern from my serverless patterns post. This time he tackles the Notifier pattern with the added benefit of the recently released SNS DLQs.

Serverless Tutorials 🏗

Implement the new Stripe checkout workflow with a React Amplify project (serverless and SCA compliant)
Straightforward use case for creating a simple checkout workflow using Stripe and a serverless backend.

Building a serverless data pipeline using Kinesis > Lambda > S3 Glacier pt.2
Part 2 of Serkan Özal’s series covering how to build a serverless data pipeline with Kinesis, Lambda, and AWS S3 Glacier.

Using the AWS CDK to build scheduled Lambda Functions
Maarten Thoelen provides a complete sample for creating Lambda functions and building a CI/CD pipeline using infrastructure as code.

How to build serverless cursor-based pagination service on the AWS Cloud
Andy Nguyen shows an alternate technique to pagination. If you’re using DynamoDB as your database, then this method would be your main option.

How to deploy a Lambda@Edge function with AWS CloudFormation
This is an interesting use case that merges audio files using Lambda@Edge to deliver audio responses to Alexa. Stephane Couzinier explains the whole process (including a CI/CD pipeline) and provides the finished code.

Serverless Security 🔒

How to build secure serverless applications for PCI-DSS compliance?
Thinking about storing data that requires PCI-DSS compliance? My advice: DON’T. But if you need to, this article from Ryan Jones will get you asking the right questions.

Serverless Reads 🤓

New Lambda Features – Pre-re:Invent – Nov. 2019
The number of new serverless announcements are overwhelming. Fernando Medina Corey picks out a few of them and gives us more detail on why they are important.

Is Your CI/CD Pipeline Ready For Chaos?
John Gilbert asks some important questions about your CI/CD process.

The top 30 Amazon products and services tech pros used this year
2nd Watch looked at their customer’s usage of AWS products, and there are some interesting ones at the top, including DynamoDB, SNS, and even Lambda, with 83% of their customer using it.

7 of the Best Benefits of AWS Lambda for Your Cloud Computing System
Trisha Paine outlines seven benefits to AWS Lambda. Hard for me to argue with these.

When you have time during your commute… 🚙

Episode #24: Serverless Application Security with Ory Segal (Part 2)
In this episode, I continue my conversation with Ory Segal about Serverless Application Security. We finish reviewing the CSA Top 12 Most Critical Risks for Serverless Applications, and discuss the future of security for serverless and ephemeral compute.

When the teams at AWS can’t even wait until re:Invent… 🤷‍♂️

Once again, AWS has flooded us with a plethora of announcements regarding new features and products. I haven’t had the time to read through all of these, let alone actually digest and process what they mean. I’ve included 54 links here to the announcements and related blog posts, and there are even more on the What’s New blog! I’m listing all of the ones that seemed interesting and relevant to serverless, and categorized them the best I could. It’s going to take us awhile to make sense of all these, so stay tuned for more in-depth analysis in the weeks to come.

Event-Driven Workflows:

Observability and Monitoring:

Database:

Developer Tools:

Machine Learning:

Security:

General:

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

December 2-6, 2019 – AWS re:Invent 2019

December 3, 2019 – Taco Tuesday with the AWS Heroes

December 4, 2019 – #ServerlessForEveryone Community Party @ AWS re:Invent 2019

December 14, 2019 – ServerlessDays Fukuoka Japan

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Ryan Kroonenburg (@KroonenburgRyan). Ryan is the co-founder of A Cloud Guru and an AWS community hero (you might remember when I featured his brother Sam a few weeks ago). When Ryan was a solution architect five years ago, he taught himself AWS, and realized the community was underserved in AWS courses. Fast forward to today, and Ryan’s work with A Cloud Guru has reached over one million students and he continues to teach the community about AWS, serverless, and the cloud. Thank you for making serverless accessible to all who want to learn, Ryan! 🙌

Final Thoughts 🤔

I don’t know about you, but my brain is fried from trying to process all these new AWS serverless announcements, and we’re still over a week away from the big keynotes at re:Invent. I have so many things I want to write about and share, but with all this information to look at, there just isn’t enough time in the day. And if you’re feeling overwhelmed, don’t sweat it. I have a feeling that we’ll be seeing some excellent posts, videos, and tutorials from the serverless community over the next few months to help us make sense of it all.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Hoping to see you at re:Invent,
Jeremy

Off-by-none: Issue #64

pre:Invent is just getting started… 🤯

Welcome to Issue #64 of Off-by-none. Thanks for being here! 👋

Last week, we rehashed the Fat Lambda debate and met some new AWS heroes. This week, we look at a ton of announcements and feature launches leading up to re:Invent, plus we share lots of awesome content from the serverless community.

🥳 Don’t forget! If you’re planning on going to AWS re:Invent this year, be sure to register for the #ServerlessForEveryone Community Party @ AWS re:Invent 2019. We’re adding names to the waiting list, but we’d still love for you to join us and our awesome guests! We have some amazing sponsors (StackeryThundra, AWS, CloudZero, Lumigo, Protego, Serverless, Inc., Edrans, and New Relic Serverless) that are making this event possible. I hope you’ll be able to join us!

Lots to get to today, so let’s get to it! 🚀

Serverless News & Product Announcements 📣

Cloud Run, a managed Knative service, is GA
This really interesting service has been in beta for awhile, and is finally available in both a fully-managed version, and one for Anthos.

Launching our knowledge base
The team over at the Dashbird has put together a robust knowledge base that is loaded with all kinds of serverless information including DynamoDB modeling, API Gateway configurations, and much more.

Solo.io Releases Production-Ready Gloo Enterprise 1.0 for Modernizing to Cloud Native Applications
Gloo is an interesting solution for implementing the strangler pattern for aggregating and modernize distributed APIs. I don’t know much about the new Enterprise version, but it could be a useful bridge for companies that are straddling on-prem and cloud.

Aqua Security Acquires CloudSploit to Monitor Configs
Misconfiguring your cloud resources seems to be all the rage nowadays. This pickup by Aqua Security should help add some piece of mind, especially for those staring at huge cloud configuration files.

Pulumi infrastructure-as-code tool adds .NET language support
Pulumi continues on its quest to compete with AWS CDK, CloudFormation, and Terraform by adding additional language support for C#, F#, and VB.NET.

Serverless Use Cases 🗺

AWS Lambda: Real-World Use Cases for the DevOps Engineer
If you’re in the DevOps space and are curious how serverless can make your life easier, check out this excellent post from Emrah Samdan.

Stateful Programming Models in Serverless Functions
This is a good summary by Steef-Jan Wiggers that recounts Chris Gillum’s presentation at QCon. He talks about two ways in which you can add state to your Azure Functions to accommodate specific use cases.

Serverless Concepts 🏗

Error Handling in AWS Lambda and API Gateway
Ben Arena outlines four different approaches to handling errors with API Gateway and AWS Lambda. He didn’t mention Lambda API as a possible solution, but we’ll let it slide. 😉

Comparing the multi-table and single-table approaches to designing a DynamoDB data model
Paul Swail has an excellent article that discusses the multi-table versus single-table approach to designing data models in DynamoDB. I’m a big fan of the single table approach, but your mileage may vary, as Paul explains.

Why use Google Cloud Run? Here are 5 Compelling Reasons to Opt for Serverless Containers
Geshan Manandhar makes some good points, but as much as I love the idea of Google Cloud Run, there is still a fair amount of heavy lifting involved to get up and running. Plus, not having to learn a new paradigm might seems tempting, but ultimately it will be required in order to embrace a serverless mindset.

Serverless Patterns
I saw this pop up the other day. Toby Hede has put together some serverless patterns for you to learn from and use.

Serverless Patterns implemented, part 1
And speaking of serverless patterns, Vicenç García has taken some of the patterns from my post and created code samples for implementing them. Very cool.

Designing durable serverless apps with DLQs for Amazon SNS, Amazon SQS, AWS Lambda
AWS released DLQs for SNS topics this week, and with that, a bunch of new design patterns open up for building durable applications. Otavio Ferreira has an awesome post on how to use this new feature (along with existing fallback mechanisms) to build a highly durable serverless application.

EventBridge vs CloudWatch Events, Kinesis and SNS
Very informative post by Aviad Mor about EventBridge and how it compares to Kinesis and SNS. He mentions that EventBridge requires SQS to add resiliency, but that really depends on your downstream target. Unless you expect your target to be down for more that 24 hours, there is a fair amount of durability and resiliency built in. TLDR; don’t over engineer if you don’t need to.

AWS Serverless WebSockets – Introduction Around the Pitfalls
This is a great post by Joonas Laitio that goes through the details of how API Gateway WebSockets work, and what are some of the pitfalls that you need to be wary about.

AWS Lambda and Node.js 12: Support and Benchmark
AWS also released a bunch of new runtimes for Lambda, and Ran Ribenzaft benchmarked Node.js 12. According to this post, it’s a bit slower (which seems strange), but it’s packed with new features, so the tradeoff might be worth it.

Kinesis vs SNS/SQS
John Gilbert shares why he uses Kinesis for first-class events rather than SNS to SQS. While I agree that Kinesis is a workhorse that has lots of great use cases, I think you’ll see more of these event-driven workloads shift to EventBridge as a more flexible alternative.

Serverless Tutorials 🏗

Building a serverless data pipeline using Kinesis > Lambda > S3 Glacier pt. 1
Serkan Özal outlines the first few steps to building a serverless data pipeline using Kinesis.

Using AWS Lambda and SQS to perform mass, distributed and asynchronous updates on DynamoDB Tables
Great post by Rafal Wilinski that discusses possible serverless patterns to reliably perform DynamoDB table updates across a large number of items. I’ve implemented similar patterns (without the orchestrator) and have found that creating batches of 25 records PER SQS message is incredibly efficient. That way you set your Lambda batch size to 10, and you get 250 record updates per Lambda execution for a total of 10 batch writes.

Serverless Security 🔒

7 best practices when using AWS SSM Parameter Store
This useful post by Hoang Le outlines seven common practices when using AWS Parameter Store. #5 recommends fetching your parameters at runtime, which I agree with, but I prefer not to save the data in environment variables. Maybe I’m just paranoid.

Is Cloud Computing Secure? 7 FAQs About Cloud Security Answered (2020)
Trisha Paine from Protego answers seven frequently asked questions about cloud security.

Amazon GuardDuty Supports Exporting Findings to an Amazon S3 Bucket
I’m including this in the security section because it makes more sense here. Exporting your GuardDuty findings into S3 makes aggregating data across multiple regions and accounts super simple.

Serverless Reads 🤓

O’Reilly serverless survey 2019: Concerns, what works, and what to expect
Super interesting data in these survey results. Crazy to think that only 40% of respondents have adopted serverless in their organizations. The experience versus successful outcomes data is also quite fascinating. We still have a long way to go to get people on board, but it is certainly spreading.

How does improved VPC help reducing Lambda cold start
Crespo Wang explains how the improvements to VPC networking for Lambda dramatically reduced cold starts in his organization.

Towards Serverless (FaaS) as the next step in Infrastructure-as-a-Service (IaaS) evolution
Jonathan Eisenzopf has a interesting post that provides some thoughts on the current serverless ecosystem and how computing on the edge (or even on the client using WASM) might be the next evolution of compute.

Containers Or Serverless? The Battle For Your DevOps Mindshare
I’m including this post by John Anthony Radosta because I hold a different point of view. He mentions that “the complexity of inter-calling Lambda functions made it near impossible to debug anything.” If that’s the case, it sounds to me like it may have been a poorly designed system. Granted, knowing how to design serverless systems isn’t easy, but this seems like a case of blaming the hammer instead of the carpenter.

For the audiophile… 🎧

Episode #23: Serverless Application Security with Ory Segal (Part 1)
In this episode, I chat with Ory Segal about the differences between traditional and serverless security, the importance of the CSA’s 12 Most Critical Risks for Serverless Applications, and what the future of serverless security looks like.

And the flurry of AWS announcements and launches begins… 💨

Note: There were a TON of announcements this past week. I only included a handful of them that were really interesting to me, but it’s worth checking out the What’s New blog on AWS to see them all.

Amazon SNS Adds Support for Dead-Letter Queues (DLQ)
This is an amazing update! My biggest concern with sending events to SNS with destinations other than Lambda, was not being able to capture failures and replay messages. Now you can add an SQS queue to handle any failed delivery. Game changer.

AWS Lambda Supports Amazon SQS FIFO (First-In-First-Out) as an Event Source
Here’s an amazing edition to the Lambda event source catalog. SQS support has been available for quite some time, but SQS FIFO queues required a CloudWatch Events Rule hack to trigger your Lambda function every minute. Now you can simply subscribe a Lambda, and it will process your ordered messages serially as soon as they enter the queue.

You now can configure table settings when you restore a table from your Amazon DynamoDB backups
Love this feature! If you wanted to restore data to a new table, you had to restore all of the indexes and capacity settings. Now you can restore data without that extra overhead, which might even give you some flexibility with your data model.

Amazon DynamoDB adaptive capacity now handles imbalanced workloads better by isolating frequently accessed items automatically
This is a super cool feature. DynamoDB will now take frequently accessed items (like popular product records) and automatically distribute them to different partitions to avoid creating hot partitions and requiring more throughput.

AWS Lambda now supports Node.js 12
If you’ve been hoping for import/export statements, async stack traces, and private class variables, the wait is over.

AWS Lambda now supports Python 3.8
This is the newest major release of the Python language, and contains many new features such as assignment expressions, positional-only arguments, and typing improvements.

AWS Lambda now supports Java 11
You can use Java 11 features such as its improved HTTP Client API and new methods for reading and writing strings when authoring your functions.

AWS Step Functions adds Amazon EMR service integration
AWS Step Functions is now integrated with Amazon EMR, making it faster to build and easier to monitor EMR big data processing workflows. Read more here.

AWS AppSync adds Real-Time enhancements with Pure WebSockets support for GraphQL Subscriptions
Real-time updates to connected clients is becoming a standard practice in modern applications, and this new feature from AppSync, which basically handles all of this for you, is absolutely amazing. Read more here.

Amazon Transcribe Now Supports Speech-to-text in 8 Additional Languages
Repeating my previous comment for prosperity: “I’m looking forward to them fully supporting English.”

Support record-level insert, update, and delete on Amazon S3 with Amazon EMR
Super interesting use cases are possible with this, including a whole bunch of data privacy compliance. More about this here.

AWS CodePipeline Enables Passing Variables Between Actions At Execution Time
You can now pass variables from one action to another in your pipeline and dynamically configure your actions with variables that will be evaluated at execution time.

Amazon CloudWatch Launches Embedded Metric Format
This certainly adds a bunch of new capabilities, but it does seem to add quite a bit of heavy-lifting to your logging strategy.

AWS CloudFormation Launches Resource Import
So this is a super cool feature that allows you to import existing resources into CloudFormation stacks. Still waiting for the feature that allows you to export a resource configured in the console to a CloudFormation template. More detail here.

Introducing AWS Data Exchange
AWS Data Exchange is a new service that makes it easy for millions of AWS customers to securely find, subscribe to, and use third-party data in the cloud.

Thoughts from Twitter 🐦

I have some serious questions for you Twitter. What currently prevents the wide adoption of chaos engineering in your organization? ~ Adrian Hornsby
According to Adrian’s poll, it looks like most people feel as though there is “enough chaos in production” already, which obviously isn’t the point. It’s a fascinating discipline that most orgs would definitely benefit from.

Upcoming Serverless Events 🗓

There are a lot of upcoming serverless events, webinars, livestreams, and more. If you have an event you’d like me to mention, please email me.

November 20, 2019 – Serverless for PCI DSS Blueprint (Webinar)

November 22, 2019 – Is Serverless SecureLess? (Webinar)

December 2-6, 2019 – AWS re:Invent 2019

December 14, 2019 – ServerlessDays Fukuoka Japan

Serverless Star of the Week ⭐️

There is a very long list of people that are doing #ServerlessGood and contributing to the Serverless community. These people deserve recognition for their efforts. So each week, I will mention someone whose recent contribution really stood out to me. I love meeting new people, so if you know someone who deserves recognition, please let me know.

This week’s star is Jeff Hollan (@jeffhollan). Jeff is the Principal Program Manager for Microsoft Azure Functions. I know this issue is mostly about AWS and re:Invent, but let’s not forget that Microsoft launched a bunch of cool serverless stuff at Microsoft Ignite a couple of weeks ago. Jeff is leading the charge with Azure Functions, and when he gets free time, you will likely find him chatting up how to leverage cloud technology and serverless at various conferences and events around the world. Thank you, Jeff, for your advocacy and continued push for serverless innovation! 🙌

Final Thoughts 🤔

The number of announcements from AWS, two weeks before re:Invent, is already overwhelming. I know more are coming next week, and the ones at re:Invent are going to be mind-blowing! If you like to geek out on this stuff like I do, hold on to your hats, because the next few weeks are going to be a wild ride.

I hope you enjoyed this issue of Off-by-none. Please send me your feedback and suggestions as they help to make this newsletter better each week. You can reach me via Twitter, LinkedIn, Facebook, or email and let me know your thoughts, criticisms, or (perhaps) even how you’d like to contribute to Off-by-none. If you like this newsletter, and think others would too, please do me the honor of sharing it with friends and coworkers who are interested in serverless.

Take care,
Jeremy