More control is a good thing, right? I mean, who doesn’t want the ability to twist every knob and tweak every settings to your heart’s desire so that each minute detail of your infrastructure is artisanally handcrafted? I’ll tell you who: this guy! Why? Because I have neither the time, energy, nor expertise to even begin to understand the impact of most of those configuration changes. Not only that, but I likely don’t even have the information needed to understand which optimizations are required, let alone a way to prioritize them.
All Posts
The side effect of increasing developer responsibility
It wasn’t that long ago that the vast majority of developers didn’t worry much about infrastructure. Sure, there were plenty of us configuring Linux servers and setting up the occasional MySQL database, but that certainly wasn’t the norm. For those that worked in larger organizations, your code was likely checked into perforce or subversion, and then magically ended up in production (some days, weeks, or even months later). For many, this is probably still how they ship code.
Why we can’t stop talking about ChatGPT
Posted in Random Thoughts on February 18, 2023
When ChatGPT was first released, I remember my Twitter timeline being inundated with screenshot after screenshot of AI generated responses. Everything from simple questions to complex programming logic, with most marveling at the technological advancement. The tech was incredibly interesting, for sure, but to me, it quickly became quite tiring. I even contemplated muting the keyword for a bit! It wasn’t because I don’t welcome progress, quite the opposite. I just had this sinking feeling that AI generated text was going to start polluting the Internet. I certainly wasn’t wrong about that, but I think there are other much more concerning angles to this.
Not so serverless Neptune
Several years ago I wrote a post asking people to stop calling everything serverless. I even gave a keynote at Serverless Days Milan the following year pleading the same message. My contention was quite simple: “when everything’s serverless, nothing will be.”
Back then, “serverless” was still relatively new, and the possibilities were seemingly endless. Sure, there were a few people starting to mislabel things, and of course, haters were gonna hate, but for the most part, the argument was less about the nuances of the technology and more about the “nature” of serverless and the serverless-first mindset. But then something changed.
The cloud isn’t the issue, you’re just using it wrong
#Tech Twitter was all abuzz recently after DHH boldly proclaimed and explained why [37signals is] leaving the cloud. A lot of people cheered, some of us jeered, and everyone else just pitched web3 as an alternative solution. DHH’s success has earned him a giant platform and a tremendous amount of influence, and while I often disagree with him, it’s clear that many others do not. I spent quite a bit of time reading through all the retweets, reposts, comments, and hot takes, and I came to a fairly simple conclusion: these people are using the cloud wrong.
Serverless Platform Engineering
I’m a huge fan of Charity Majors, and I always learn something new when she shares her thoughts on anything that has to do with cloud and engineering. So when I came across this Twitter post, I grabbed some 🍿 and got ready to have my mind blown. I was not disappointed.
Getting abstractions wrong with AWS SAM Serverless Connectors
I was intrigued when I first saw the announcement of AWS SAM Serverless Connectors. I don’t use SAM very much (if at all anymore), so it wasn’t the hope of this being some sort of silver bullet for my occasional IAM frustrations that got my attention. Rather, it was another opportunity to learn how AWS is trying to abstract away their mostly self-imposed complexity problems. Unfortunately, I think they missed the mark on this.
Cloud Native versus Native Cloud apps
I’ve been spending a lot of time lately thinking about the next evolution of the cloud, and more importantly, what the developer experience looks like. A few years ago, I think that most of us in the serverless ecosystem thought that the path forward seemed quite clear. Serverless-first was obviously “the way.” Small, discrete, single-purpose functions interconnected through a series of planet-scale, self-upgrading, managed services with built-in redundancy was the holy grail of cloud development.
Of course there were some gotchas in there, and not every use case was a perfect fit, but over time we figured these would be addressed as the technology evolved. For the most part, that has come to pass. Even if AWS hasn’t quite yet solved some of these issues, other cloud providers and startups have certainly tried. But while serverless was slowly preparing to cross the chasm, another already widely accepted technology was gaining traction in the cloud: containers.
Can serverless really save polar bears?
JV Roig wrote a great article recently about polar bears, serverless and sustainability. The premise is quite simple: if we make better choices in the cloud, we can reduce our impact on the environment, and thus, save the polar bears. The sentiment is nice, but it is, of course, much more complicated than that. This begs the question, how much impact can our cloud choices really have?
Is learning serverless really that hard?
I came across Paul Johnston’s Learning Serverless (and why it is hard) post one Saturday morning, and ended up with a sore neck because I was nodding in agreement the entire time I was reading it. Okay, maybe the neck pain had more to do with how I slept the night before, but I’m quite sure the agreeing nods contributed. But when it comes to learning serverless, a little bit of neck pain, IMO, is the least of your problems.
Let’s talk about the cloud skills gap
Lydia Leong recently wrote a thought-provoking piece suggesting that cloud adoption will fail because of the skills gap. This certainly isn’t (or shouldn’t be) news to those of us paying attention. The cloud has become progressively more complex as it has matured. There has been an explosion of cloud services, a rapid expansion of public cloud competitors that are achieving (or exceeding) feature parity for the most common use cases, and a third-party market that now contains more than 1,000 “cloud-native” tools, services, and platforms.
Ensuring Data Integrity with DynamoDB
DynamoDB is an incredibly powerful NoSQL database. It’s schema-less, which gives you lots of flexibility, but it also means that you are responsible for managing the integrity of your data. This includes ensuring the structure of your data, as well as the ability to preserve metadata throughout your data’s lifecycle.
Unfortunately, DynamoDB doesn’t currently store any metadata associated with items. If you want to know when a particular item was written to the table, for example, you have to store that information yourself. While it’s not particularly difficult to add these attributes to an item, maintaining their integrity can come with some challenges.
In this article, we’ll discuss several strategies that can be used to ensure data integrity in your DynamoDB tables.
The Unfulfilled Potential of Serverless
Posted in Serverless on November 14, 2021
Corey Quinn, Cloud Economist (and perpetual thorn in AWS’s side), recently published a post titled The Unfulfilled Promise of Serverless. Twitter reacted as we would expect, with plenty of folks feeling vindicated, others professing their staunch disagreement, and perhaps even a few now questioning their life (and technology) choices. My take is that he’s not wrong, but he’s also not entirely right.
Big Serverless Chats Announcement!
Posted in Serverless on June 28, 2021
When I started the podcast in 2019, I wanted to invite others into the great off-the-cuff, offline conversations I was having with members of the serverless community. The chats I’d share after an event over drinks, in the hallway between conference sessions, or while mingling at a meetup, were thought-provoking and interesting, and offered fresh ideas, perspectives, and sometimes, even solutions that I could take back to my work and projects. After more than 100 episodes of inviting listeners and guests to these conversations, I am now extending a different sort of invitation for one of our serverless community members.
I am extremely excited to announce that Rebecca Marshburn will be joining Serverless Chats as a co-host!
New Year, New Job, Same Serverless Mission
Posted in Serverless on February 18, 2021
I took a new job as the GM of Serverless Cloud at Serverless, Inc. I’m really excited about what I’m working on there, and here’s why…
Some Background
When I first discovered AWS Lambda six years ago, I became fascinated with serverless, or rather, I became fascinated with the potential of serverless. Having spent an inordinate amount of time during the first 15 years of my career racking, configuring, patching, maintaining, and (poorly) scaling servers, the notion of abstracting all of that away seemed too good to be true. I had already been using AWS for five years at that point, so even though the cloud had eliminated the need for physical hardware, all the complexity around building high-scale, highly-available, highly-distributed applications was still there. For me, serverless was the way to solve that problem. So I went all in.
Aurora Serverless v2: The Good, the Better, and the Possibly Amazing
Three years ago at re:Invent 2017, AWS announced the original Amazon Aurora Serverless preview. I spent quite a bit of time with it, and when it went GA 9 months later, I published my thoughts in a post titled Aurora Serverless: The Good, the Bad and the Scalable.
If you read the post, you’ll see that I was excited and optimistic, even though there were a lot of missing features. And after several months of more experiments, I finally moved some production workloads onto it, and had quite a bit of success. Over the last 18 months, we’ve seen some improvements to the product (including support for PostgreSQL and the Data API), but there were still loads of problems with the scale up/down speeds, failover time, and lack of Aurora provisioned cluster features.
That all changed with the introduction of Amazon Aurora Serverless v2. I finally got access to the preview and spent a few hours trying to break it. My first impression? This thing might just be a silver bullet!
I know that’s a bold statement. 😉 But even though I’ve only been using it for a few hours, I’ve also read through the (minimal) docs, reviewed the pricing, and talked to one of the PMs to understand it the best I could. There clearly must be some caveats, but from what I’ve seen, Aurora Serverless v2 is very, very promising. Let’s take a closer look!
Update December 9, 2020: I’ve updated the post with some more information after having watched the “Amazon Aurora Serverless v2: Instant scaling for demanding workloads” presentation by Murali Brahmadesam (Director of Engineering, Aurora Databases and Storage) and Chayan Biswas (Principle Product Manager, Amazon Aurora). The new images are courtesy of their presentation.
Jeremy’s Guide to a Very Serverless re:Invent 2020
Posted in Serverless on November 29, 2020
It’s AWS re:Invent time, and once again, developers, architects, business leaders, and everyone in between are faced with the daunting task of selecting from thousands of hours of re:Invent content. As usual, I will be focusing most of my time on serverless, so I’ve combed through the massive session catalog and picked out the ones that look the most interesting to me. If you’re looking to focus on serverless during this re:Invent, perhaps you’ll find my suggestions useful.
My picks are also available on the Cloud Pegboard re:Invent Tool (Thanks, Ken). Select “Jeremy Daly” from the “AWS Hero Picks” dropdown and you’ll see all my selections with the options to add them to your wishlist and export them to your calendar. I have about 60 sessions on my list, which I’ve categorized below. But I realize that no human is likely going to be able to watch them all, so I’ve also made a list of Sessions you can’t miss!
There are sure to be plenty of announcements throughout the three weeks of re:Invent, so be sure to subscribe to the Off-by-none newsletter for weekly recaps.
LAMBDA – A Serverless Musical
Posted in Serverless on September 20, 2020
Join the Serverless Revolution!
Learning a new paradigm can be really difficult, especially something as revolutionary (and different) as serverless. Thanks to a little inspiration from fellow AWS Serverless Hero, Forrest Brazeal, I created this Hamilton parody to help teach people what serverless is all about and why it’s such an amazing way to build applications. Hopefully it inspires you as well. Enjoy!
🚀 Project Update: July 9, 2020
Data API Client: v1.1 Released
Bug fixes and feature updates including support for native JavaScript dates (thanks @cklam2), support for non-specific database queries, and deprecation of the HTTP keepAlive workaround in favor of the native SDK support. Read More...
Announcing the Serverless Reference Architectures Project
Posted in Serverless on July 2, 2020
Serverless gives us the power to focus on delivering value to our customers without worrying about the maintenance and operations of the underlying compute resources. Cloud providers (like AWS), also give us a huge number of managed services that we can stitch together to create incredibly powerful, and massively scalable serverless microservices.
Almost 2 years ago now, I wrote a post on Serverless Microservice Patterns for AWS that became a popular reference for newbies and serverless veterans alike. The capabilities of serverless have changed dramatically since then, opening up a ton of new patterns and possibilities. Today I’m announcing the Serverless Reference Architectures Project. This project is intended to capture, share, explore, and debate the patterns and practices being used in serverless production applications today.
Takeaways from Programming AWS Lambda by Mike Roberts and John Chapin
Recently, Symphonia co-founders Mike Roberts and John Chapin wrote a book called Programming AWS Lambda: Build and Deploy Serverless Applications with Java. I personally abandoned Java long ago, but I knew full well that anything written by Mike and John was sure to be great. So despite the title (and my past war stories of working with Java), I picked up the book and gave it a read. I discovered that it’s not really a book about Java, but a book about building serverless applications with the examples in Java. Sure, there are a few very Java specific things (which every Java developer probably needs to read), but overall, this book offers some great insight into serverless from two experts in the field.
I had the chance to catch up with Mike on a recent episode of Serverless Chats. We discussed the book, how John and Mike got started with serverless (by building Java Lambda functions, of course), and what are some of the best practices people need to think about when building serverless applications. It was a great conversation (which you can watch/listen to here), but it was also jam packed with information, so I thought I’d highlight some of the important takeaways.
Making the Case for Serverless Use Cases
For quite some time, there was a running joke that “serverless” was just for converting images to thumbnails. That’s still a great use case for serverless, of course, but since AWS released Lambda in 2014, serverless has definitely come a long way. Even still, newcomers to the space often don’t realize just how many use cases there are for serverless. I spoke with Gareth McCumskey, a Solutions Architect at Serverless Inc, on a recent two part episode (part 1 and part 2) of Serverless Chats, and we discussed nine very applicable use cases that I thought I’d share with you here.
12 Important Lessons from The DynamoDB Book
Fellow serverless advocate, and AWS Data Hero, Alex DeBrie, recently released The DynamoDB Book, which ventures way beyond the basics of DynamoDB, but still offers an approachable and useful resource for developers of any experience level. I had the opportunity to read the book and then speak with Alex about it on Serverless Chats. We discussed several really important lessons from the book that every DynamoDB practitioner needs to know. Here are twelve of my favorites, in no particular order.