It’s AWS re:Invent time, and once again, developers, architects, business leaders, and everyone in between are faced with the daunting task of selecting from thousands of hours of re:Invent content. As usual, I will be focusing most of my time on serverless, so I’ve combed through the massive session catalog and picked out the ones that look the most interesting to me. If you’re looking to focus on serverless during this re:Invent, perhaps you’ll find my suggestions useful.
My picks are also available on the Cloud Pegboard re:Invent Tool (Thanks, Ken). Select “Jeremy Daly” from the “AWS Hero Picks” dropdown and you’ll see all my selections with the options to add them to your wishlist and export them to your calendar. I have about 60 sessions on my list, which I’ve categorized below. But I realize that no human is likely going to be able to watch them all, so I’ve also made a list of Sessions you can’t miss!
There are sure to be plenty of announcements throughout the three weeks of re:Invent, so be sure to subscribe to the Off-by-none newsletter for weekly recaps.
Recently, Symphonia co-founders Mike Roberts and John Chapin wrote a book called Programming AWS Lambda: Build and Deploy Serverless Applications with Java. I personally abandoned Java long ago, but I knew full well that anything written by Mike and John was sure to be great. So despite the title (and my past war stories of working with Java), I picked up the book and gave it a read. I discovered that it’s not really a book about Java, but a book about building serverless applications with the examples in Java. Sure, there are a few very Java specific things (which every Java developer probably needs to read), but overall, this book offers some great insight into serverless from two experts in the field.
I had the chance to catch up with Mike on a recent episode of Serverless Chats. We discussed the book, how John and Mike got started with serverless (by building Java Lambda functions, of course), and what are some of the best practices people need to think about when building serverless applications. It was a great conversation (which you can watch/listen to here), but it was also jam packed with information, so I thought I’d highlight some of the important takeaways.
Fellow serverless advocate, and AWS Data Hero, Alex DeBrie, recently released The DynamoDB Book, which ventures way beyond the basics of DynamoDB, but still offers an approachable and useful resource for developers of any experience level. I had the opportunity to read the book and then speak with Alex about it on Serverless Chats. We discussed several really important lessons from the book that every DynamoDB practitioner needs to know. Here are twelve of my favorites, in no particular order.
On a recent episode of Serverless Chats, I spoke with Stephen Pinkerton and Darcy Rayner of Datadog to dig into The State of Serverless report, which was released at the end of February 2020. After frequently fielding customer questions about the topic, Datadog looked at its data and customer use cases, and examined how they were using serverless. Datadog’s report is a way to break it all down, but it’s also an opportunity for its customers (and serverless users alike) to see how other people are using serverless in a data-driven way. I discussed methodology, findings, and key takeaways with Stephen and Darcy, and thought it’d be worthwhile to consolidate and share that insight.
AWS re:Invent 2019 is a wrap, but now the real work begins! There are hundreds of session videos now available on YouTube. So when you have a few days (or weeks) of downtime, you can dig in to these amazing talks and learn about whatever AWS topics you fancy.
I was only able to attend a few talks this year, but one that I knew I couldn’t miss in person, was Rick Houlihan’s DAT403: Amazon DynamoDB deep dive: Advanced design patterns. At the last two re:Invents, he gave similar talks that explored how to use single-table designs in DynamoDB… and they blew my mind! 🤯 These videos were so mind-bending, that they inspired me to immerse myself in NoSQL design and write my How to switch from RDBMS to DynamoDB in 20 easy steps post. I was hoping to have a similar experience with this year’s edition, and I WAS NOT DISAPPOINTED.
As expected, it was a 60 minute firehose of #NoSQL knowledge bombs. There was A LOT to take away from this, so after the session, I wrote a Twitter thread that included some really interesting lessons that stuck out to me. The video has been posted, so definitely watch it (maybe like 10 times 🤷♂️), and use it to get started (or continue on) your DynamoDB journey.
I’m a big fan of following the Single Responsibility Principle when creating Lambda functions in my serverless applications. The idea of each function doing “one thing well” allows you to easily separate discrete pieces of business logic into reusable components. In addition, the Lambda concurrency model, along with the ability to add fine-grained IAM permissions per function, gives you a tremendous amount of control over the security, scalability, and cost of each part of your application.
However, there are several drawbacks with this approach that often attract criticism. These include things like increased complexity, higher likelihood of cold starts, separation of log files, and the inability to easily compose functions. I think there is merit to these criticisms, but I have personally found the benefits to far outweigh any of the negatives. A little bit of googling should help you find ways to mitigate many of these concerns, but I want to focus on the one that seems to trip most people up: function composition.
Developing and testing serverless applications locally can be a challenge. Even with tools like SAM and the Serverless Framework, you often end up mocking your cloud resources, or resorting to tricks (like using pseudo-variables) to build ARNs and service endpoint URLs manually. While these workarounds may have the desired result, they also complicate our configuration files with (potentially brittle) user-constructed strings, which duplicates information already available to CloudFormation.
This is a common problem for me and other serverless developers I know. So I decided to come up with a solution.
I’ve been building serverless applications since AWS Lambda went GA in early 2015. I’m not saying that makes me an expert on the subject, but as I’ve watched the ecosystem mature and the community expand, I have formed some opinions around what it means exactly to be “serverless.” I often see tweets or articles that talk about serverless in a way that’s, let’s say, incompatible with my interpretation. This sometimes makes my blood boil, because I believe that “serverless” isn’t a buzzword, and that it actually stands for something important.
I’m sure that many people believe that this is just a semantic argument, but I disagree. When we refer to something as being “serverless”, there should be an agreed upon understanding of not only what that means, but also what it empowers you to do. If we continue to let marketers hijack the term, then it will become a buzzword with absolutely no discernible meaning whatsoever. In this post, we’ll look at how some leaders in the serverless space have defined it, I’ll add some of my thoughts, and then offer my own definition at the end.
Last week I spent six incredibly exhausting days in Las Vegas at the AWS re:Invent conference. More than 50,000 developers, partners, customers, and cloud enthusiasts came together to experience this annual event that continues to grow year after year. This was my first time attending, and while I wasn’t quite sure what to expect, I left with not just the feeling that I got my money’s worth, but that AWS is doing everything in their power to help customers like me succeed.
There have already been some really good wrap-up posts about the event. Take a look at James Beswick’s What I learned from AWS re:Invent 2018, Paul Swail’s What new use cases do the re:Invent 2018 serverless announcements open up?, and All the Serverless announcements at re:Invent 2018 from the Serverless, Inc. blog. There’s a lot of good analysis in these posts, so rather than simply rehash everything, I figured I touch on a few of the announcements that I think really matter. We’ll get to that in a minute, but first I want to point out a few things about Amazon Web Services that I learned this past week.
Update June 5, 2019: The Data API team has released another update that adds improvements to the JSON serialization of the responses. Any unused type fields will be removed, which makes the response size 80+% smaller.
Update June 4, 2019: After playing around with the updated Data API, I found myself writing a few wrappers to handle parameter formation, transaction management, and response formatting. I ended up writing a full-blown client library for it. I call it the “Data API Client“, and it’s available now on GitHub and NPM.
Update May 31, 2019: AWS has released an updated version of the Data API (see here). There have been a number of improvements (especially to the speed, security, and transaction handling). I’ve updated this post to reflect the new changes/improvements.
On Tuesday, November 20, 2018, AWS announced the release of the new Aurora Serverless Data API. This has been a long awaited feature and has been at the top of many a person’s #awswishlist. As you can imagine, there was quite a bit of fanfare over this on Twitter.
Obviously, I too was excited. The prospect of not needing to use VPCs with Lambda functions to access an RDS database is pretty compelling. Think about all those cold start savings. Plus, connection management with serverless and RDBMS has been quite tricky. I even wrote an NPM package to help deal with the
max_connections issue and the inevitable zombies 🧟♂️ roaming around your RDS cluster. So AWS’s RDS via HTTP seems like the perfect solution, right?
Well, not so fast. 😞 (Update May 31, 2019: There have been a ton of improvements, so read the full post.)