Author | Tridib Bolar
Translated by | Zhang Weibin
Planning | Ding Xiaoyun
In the field of software architecture and application design, design patterns are one of the basic building blocks. The concept of design patterns was proposed by Christopher Alexander in the late 1970s (The Timeless Way of Building, 1979 and A Pattern Language—Towns, Buildings, Construction, 1977):
Each pattern describes a problem that is constantly emerging in our environment, and then describes the core of the solution to that problem. In this way, we can use those solutions that we already have countless times without having to repeat the same work. —— Alexander et al
Subsequently, this concept was adopted by the software community, resulting in different kinds of design patterns applied to the field of software design.
The object-oriented design pattern is an abstract tool for designing code-level building blocks that follow an OOP approach. The book Design Patterns Elements of Reusable Object-Oriented Software co-authored by Erich Gamma, Richard Helm, Ralph Johnson and John Vlissides (Gangs of Four - GoF) was published by Mechanical Industry Press. Provides developers with a guidebook in the field of object-oriented design. The book was first published in 1994, and since then, design patterns have become an integral part of software design.
Design patterns also apply to organizations. A large organization is indeed like a huge machine, it has a lot of gears, pipes, filters, motors, and so on. In the digital age, we're trying to digitize the human brain, so digitizing enterprise machines isn't a big deal. It is not enough to digitize a component or area of an enterprise. In fact, to control an enterprise, it is necessary to integrate its different components. Enterprise and solution architects are experimenting with usage patterns to solve everyday integration scenarios. The process is truly agile. Every day, thinkers from all corners of the world are solving problems and inventing new models of enterprise integration. Here, I want to mention two masters in the field, Martin Fowler and Gregor Hohpe.
Top management is constantly chasing new technology trends, and new digital product variants are being introduced every day. Business people are looking for ways to get the most out of this digital ocean, so it is necessary to modernize legacy systems, so-called digital transformation. In this area, researchers like Ian Cartwright, Rob Horn, and James Lewis have also proposed patterns in a recent Patterns of Legacy Displacement article based on their years of migration experience.
In this age of rapid change, agility is the key to success. Resiliency, continuous delivery, faster time to market, efficient development, and more are all forces driving the shift to a microservices architecture. But at the same time, not all scenarios are suitable for microservices. To help us understand where this boundary lies, Chris Richardson, author of microservice patterns, proposes a number of microservice patterns for different use cases.
In addition to the ones I mentioned above, there are many more pattern categories. In fact, there is a large body of literature on the patterns of enterprise system architecture and software. This means that architects need to choose wisely how to meet their requirements.
Enter the realm of serverless
So far, we've discussed different types of patterns for different requirements and architectures, but we've overlooked an important scenario, which is serverless systems. Serverless is one of the most important and dynamic approaches within the current technological landscape, especially in the IaaS and cloud computing space.
Serverless platforms can be divided into two broad categories, Function as a Service (FaaS) and Backend as a Service (BaaS). The FaaS model allows customers to build, deploy, run, and manage their applications without having to manage the underlying infrastructure. In contrast, BaaS provides online services that handle specific tasks through the cloud, such as authentication, storage management, notifications, messaging, and so on.
All services for serverless computing fall into the FaaS category (e.g., AWS Lambda, Google Cloud Function, Google Run, Apache OpenWhisk), while other serverless services can be classified as BaaS, such as serverless storage (AWS DynamoDB, AWS S3, Google Cloud Storage), serverless workflows (AWS Step). Function), serverless messaging (AWS SNS, AWS SQS, Google PubSub), and more.
The term serverless is very appealing, but it can be misleading. Is there really a service that can exist without a server? Behind all the serverless components offered by cloud vendors, there is a simple magic: behind these components, there is a server. The cloud provider is responsible for managing the scalability (auto-scaling), callability, concurrency, network, etc. of physical machines and/or virtual servers, while also providing an interface for end users to configure them, including things like custom runtimes, environment variables, versions, security libraries, concurrency, read/write capacity, etc.
If we focus on implementing an architecture in a serverless way, then some basic, high-level problems ensue.
What is the preferred architectural style when designing a system using serverless building blocks?
Will our application take a purely serverless approach or a hybrid approach?
In which use cases should we adopt a serverless approach?
What are the reusable architectural building blocks or patterns when implementing serverless applications?
For the remainder of this article, I'll explain the answers to the four questions above.
Serverless mode
In the technology world, serverless models are relatively new and are in the midst of rapid development. It involves different aspects, including operating mechanism, applicability, usage scenarios, usage patterns, implementation patterns, etc., and each step is constantly changing. Not only that, but as cloud vendors continue to invent new serverless products, the same microservices model can be implemented in a variety of ways, and their prices and performance vary. Around the world, software engineers are thinking from different perspectives and in different ways. So, so far, a universal way to build serverless systems has not been formed.
At the API Days Australia conference, Cassandra Bonner, solutions architect from Amazon Cloud Technologies, gave a presentation on five main usage patterns for Lambda's serverless services. She defines these five patterns from a needs perspective:
Event-driven data processing.
Web app.
Mobile and IoT applications.
Application ecosystem.
Event workflows.
Peter Sbarski, in his book Serverless Architectures on AWS, gives five patterns for solving common design problems in serverless architectures. They are:
Command
Messaging
Priority queue
Fan-out
Pipes and filters
These patterns are not unique to serverless architectures. In fact, they are a subset of distributed system patterns, such as the 65 message patterns summarized by Gregor Hohpe and Bobby Woolf, which represent the broadest set of such patterns.
My goal in writing this article is to implement pipe and filter patterns in a serverless manner in an AWS cloud environment. I'll discuss some of the alternative implementations and their respective strengths and weaknesses. Reusability is a specific aspect of my implementation.
Pipe and filter mode for serverless architecture
In agile programming, and in a microservices-friendly environment, the way of design and coding has changed from the monolithic era. Agile and microservices developers no longer put all the logic into one functional unit, preferring more fine-grained services and tasks that follow the single responsibility principle (SRP). With this, developers can break down complex functionality into a series of tasks that can be managed independently. Each task takes some input from the client, which is then consumed to perform its specific duties, and produces some output that is carried over to the next task. According to this principle, multiple tasks form a chain of tasks. Each task converts the input data into the desired output, which in turn serves as input to the next task. These transformers have traditionally been referred to as filters, and connectors that pass data from one filter to another are called pipes.
A very common use of pipes and filters is this: when a client's request arrives at the server, the request payload must go through a process of filtering and authentication. When a request is processed, new traffic may come in, and before the business logic can be executed, the system must perform some common tasks, such as decryption, authentication, validation, and removal of duplicate messages or events from the request payload.
Another scenario is the process of adding items to a shopping cart in an e-commerce application. In this case, the task chain may contain tasks such as checking the availability of items, calculating prices, adding discounts, updating the total number of shopping carts, and so on. For each of these steps, we can write a filter and then use a pipeline to connect them all.
The easiest way to implement this pattern is to use lambda functions. We know that there are two ways to call AWS services, either synchronously or asynchronously. In a synchronous scenario, lambda runs the function and waits until the lambda that made the call receives a response from the lambda being invoked, whereas in the asynchronous case, there is no need to wait. AWS supports callback methods and future objects to receive responses asynchronously. Here, the role of the pipeline is played by the internal network.
In this direct lambda-to-lambda invocation, whether synchronous or asynchronous, throttling is possible. When requests are coming in faster than the function's scalability, and the function has reached the maximum concurrency level (1000 by default), or the number of lambda instances reaches the configured reserved concurrency limit, all additional requests fail due to a throttling error (status code 429). To handle this situation, we need to add some intermediate storage between the two lambdas, which can temporarily store requests that cannot be processed immediately and implement a retry mechanism for throttling messages, which will take these messages and start processing them as soon as a lambda instance is available.
We can do this by using AWS's Simple Queue Service (SQS), as shown in the following figure. Each lambda filter processes an event and pushes it to the queue. In this design, Lambda can poll multiple events from SQS and process them as a single batch, which can also improve performance and reduce costs.
This way reduces the risk of throttling, but it is not completely avoidable. Here are some configurable parameters that we can use to balance throttling. In addition to this, we can implement a Dead Letter Queue (DLQ) for lambda to handle throttling events/messages and prevent these messages from being lost. There is a great article titled "Lessons learned from using SQS and Lambda in combination in data projects" that readers can use to understand the key parameters to solve the problem.
In the next section, I'll build a generic, reusable solution that uses another AWS component for serverless event processing, Amazon EventBridge, and I'll implement pipeline and filter design patterns.
Implement pipe and filter patterns in serverless architectures
Amazon EventBridge is a serverless event bus that makes it easier to build event-driven applications at scale by leveraging events generated from your applications, integrated software-as-a-service (SaaS) applications, and AWS services.
Before we can understand how it works, we need to understand some of the terminology associated with AWS EventBridge.
The EBus is one of the key components of EventBridge. The EBus receives events/messages from different sources and matches them to a defined set of rules. EventBridge has a default EBus, but users can also create their own EBuses. In this POC, I created an event bus called "pipe".
The rule must be associated with a specific EBus. In this POC, I created three rules for three different filters, as shown in the following figure.
For each rule, event patterns and goals are two very basic configurations.
The event pattern is a condition. It has the same structure as the events it matches. If the incoming event has a matching pattern, the rule is activated and the incoming event is passed to the destination (destination). The target is a resource or endpoint to which EventBridge is able to send events. For a particular pattern, we can set multiple targets.
In our case, I set the lambda name to the detail.target in the pattern, and once the lambda name matches, the target lambda is triggered.
Note: detail.target is a json field. The target is a configurable endpoint/destination for the event.
In an event flow, the different steps that you can perform are as follows:
The source generates an event (it must follow the pattern defined by the event source generator and event bridge rule creator).
To test our implementation, I used the following event:
Based on the specific detail.target value of the test event, there will be a rule match and executed. In our scenario, this would cause the event/message to be routed to the target lambda associated with the rule, i.e. filter1_lambda.
The target lambda completes its task and replaces the event target (detail.target) with the next lambda in the detail.filterlist json list, which is the filter2_lambda.
The target lambda then calls the tool function of the lambda layer next_filter().
The next_filter() function is responsible for building the final event and putting it into the event bridge.
Based on the new target value (i.e. filter2_lambda), another rule can be matched, making a separate filter lambda invoked.
After all tasks are completed, the terminal filter sends a message to the next destination that is not a filter. In this POC, the terminal filter is the filter3_lambda. Instead of calling next_filter function, this lambda calls the DynamoDb API to save the data to a table in DynamoDb.
As we can see, with EventBridge's pattern-matching routing feature, we can implement pipe and filter patterns with a single event bus, and even if one of the successor stages in the chain is still busy processing the previous event, the other stages in the chain are free to start processing the next event, improving overall efficiency.
As shown in the figure above, the event initially goes into the filter1_lambda because the detail.target property of the client event matches the filter-rule1 event pattern with the target filter1_lambda. After execution is complete, filter1_lambda set the event's detail.target to the next lambda, which is filter2_lambda, and sends the modified event back to the event bus. Since the value of detail.target is filter2_lambda, filter-rule2 is triggered, and so on. Through this recursive process, all filters are executed. The last filter can call some other resource instead of calling the next_filter() tool layer.
In the implementation above, one of the important tasks common to each lambda is to modify the event target (detail.target) to the next lambda in the filterlist. To accomplish this task, we used the lambda layer.
The lambda layer is a feature of lambda that helps developers extract generic functionality or libraries from lambda code and put them into a layer. This layer can be used as a tool-style block of code, and the actual lambda code can be executed on top of this layer. Lambda can reuse the layer's common features and/or libraries as needed. The AWS documentation says this:
The Lambda layer is an archive file that contains extra code, such as libraries, dependencies, and even a custom runtime.
For this POC, I wrote a tool layer that exports next_filter functions. The Lambda filter uses this function to infer the name of the next filter from the filterlist. The associated code snippets are given in the appendix at the end of this article.
The entire POC code, as well as the infrastructure code for the AWS Cloud Development Kit (CDK), can be found in the github repository.
Summary
Patterns are one of the most useful and effective tools in the field of software design. In order to solve common design problems in a standard way, we can use suitable design patterns. Patterns are like a design plugin. On the technology side, serverless is a fast-growing area, with all cloud providers regularly rolling out newly hosted serverless services. Therefore, it is difficult to decide on the appropriate technology stack for serverless management services. In this article, I discussed how to use different AWS serverless hosting services to accomplish different implementations of a design pattern in a serverless manner.
Appendix
code snippet for next_filter:
Resources
Lambda SQS extension
（https://aws.amazon.com/cn/premiumsupport/knowledge-center/lambda-sqs-scaling/）
Short and long polls for SQS messages
（https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html#sqs-long-polling）
Throttling
（https://docs.aws.amazon.com/lambda/latest/operatorguide/throttling.html）
Lessons learned from using SQS and Lambda in a data project
（https://data.solita.fi/lessons-learned-from-combining-sqs-and-lambda-in-a-data-project/）
About the Author:
Tridib Bolar works in Kolkata, India, and is an architect of cloud solutions for an IT company. He has been working in the field of programming for over 18 years. He works primarily on the AWS platform while also exploring GCP. In addition to being a proponent of the serverless model of cloud computing, he is also a fan of IoT technology.
https://www.infoq.com/articles/design-patterns-for-serverless-systems/