The Scalable Webhook

Last Updated: July 17, 2020

Simple pattern for handling high-velocity or unpredicatable workloads while mitigating downstream pressure.

Interactive Reference Architecture

Click on the components or numbered steps below to explore how this architecture works.

If you’re building a webhook, the traffic can often be unpredictable. This is fine for Lambda, but if you’re using a “less-scalable” backend like RDS, you might just run into some bottlenecks. There are ways to manage this, but because Lambda supports SQS triggers, we can throttle our workloads by queuing the requests and then using a throttled (low concurrency) Lambda function to work through our queue. Under most circumstances, your throughput should be near real-time. If there is some heavy load for a period of time, you might experience some small delays as the throttled Lambda chews through the messages.

You’ll also want to handle failed messages using a Dead Letter Queues (DLQ). The SQS Poller will adjust its polling frequency based on your Lambda function’s concurrency. You’ll need to configure your redrive policies to appropriately handle failed messages.

Deploy this Pattern

Below are the basic configurations for deploying this pattern using different frameworks and platforms. Additional configuration for your environment will be necessary. The source files and additional examples are available in the GitHub repo.

  • Are you a CDK Guru?
    Would you like to contribute patterns to the community?
    Check out the Github repo!

Want to Contribute?

All patterns and sample code are free to use and available under the MIT License. If you'd like to contribute to these patterns, please submit PRs to the GitHub repo.

» Explore more Serverless Reference Architectures and Patterns

3 thoughts on “The Scalable Webhook”

  1. Have you considered adding event bridge to this arch? It can integrate with apigw and replace the first lambda (less code to maintain)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.