When I start with new Serverless teams, the first thing I do is to start getting them to think differently about what they are building. Most developers who have been around a while have worked on more monolithic code bases where software is built in structured layers. This is all well and good and can translate to pushing code like this to the AWS Lambda service as a starting point. This will likely work nicely for quite a while - AWS Lambda is super-resilient out of the box without doing any additional configuration so the code will probably run fine and won’t cause problems. This is a great way to get going especially if your expected volumes are not large.
This is not “Thinking Serverless”
Thinking about Serverless processing is about being granular in your thinking. Sheen Brisals, an AWS Serverless Hero, has some great articles along this theme which I recommend you go and check out. Start with Think Granular To Go Faster In Serverless! — Part 1: Individuality and continue through the series.
“Thinking Atomic” is where you need to be, don’t look to write code first up to do everything - break down your processing into singular steps which can be completed in a single action. Everything must be singular and atomic - one input, one output. As soon as you start having more than a single output or side effect (database update, notification) you create additional complexity and cause problems when the code is retried or re-run. When you have multiple steps in code in one place and it fails at step 3 - what do you do with all the other steps? Is it safe to re-run those again or will you cause duplication?
The key to high scale, concurrent processing in a Serverless system is to “Think Atomic”.
A rule of thumb I like discuss with teams is limiting the number of outbound API calls to a minimum, any more than 3 outbound API calls and developers need to consider refactoring their solution. in my experience with busy systems I find increased latency the main concern that nobody considers. Every interaction in the cloud will involve API calls which naturally means network latency, which is why I highlight this as a problem to be aware of.
When your thinking is atomic and your code is doing less latency will be less of a problem and your lambda duration will be shorter enabling higher throughput with lower concurrency which means your overall account health will be better. Thinking atomically will also create a quicker pathway to refactoring to no code and using step functions which are powerful Serverless constructs.
So next time you start working on a new service, stop, and rethink the processing before you code and “Think Atomic”!