The popularity of event-driven architectures is on the rise as they offer a preferred approach to building large distributed microservice-based applications. This approach enables the development of scalable, agile, resilient, fault-tolerant, and cost-effective solutions. Unlike the classic three-tier applications that revolve around databases, EDA focuses on the flow of events through the system, resulting in a revolutionary change in application design.

The foundation of event-driven architecture is based on events that serve as signals to inform interested parties of a change in state within a business system. These events can be anything from an online purchase to the shipment of a product, or the delivery of an item to a customer’s doorstep, and they occur frequently across various industries.

Understanding an event

An event refers to an action that initiates a notification or induces a change in the application’s state. For instance:

• A new order has been received (notification) -> A new order has been placed (action)
• Withdrawal of funds has failed (state change) -> Retry the withdrawal (action 1) or end the session (action 2)
• Your order is in transit (state change) -> Notify the customer about the delivery date and time (action)
• Request to reset a password (notification) -> Send a password reset email to the customer (action)

All of these are events; however, not every event is relevant to an event-driven solution. For an event to be added, it must be pertinent to the business. For instance, a user ordering a product is a pertinent event, but their lunch choice is not.

How events flow?

In an event-driven architecture, events are unidirectional and flow from the producer to the consumer. The producer doesn’t expect a response from the consumer, which means that code execution doesn’t need to be blocked while waiting for a response. This inherent asynchronicity eliminates the risk of timeouts.

Events are triggered as a result of actions and don’t have a specific target system. We can’t say that service ‘A’ triggers events to service ‘B’, but we can say that service ‘B’ is interested in the events produced by service ‘A’. Additionally, other services, such as ‘C’ or ‘D’, may also be interested in these events.

How event driven architecture works?

Event-driven architecture relies on the interaction between producers and consumers. The producer initiates events that are transmitted asynchronously to interested consumers via an event channel. This loose coupling between the producer and consumer allows both parties to perform their tasks independently without waiting for each other.

Decoupling improves scalability by separating communication and business logic. The publisher is not affected if a subscriber goes offline or slows down consumption. If a subscriber can’t keep up with the rate of events, the event stream records them for later retrieval. This allows the publisher to send notifications without any throughput limitations and with high resilience to failure.

Publish/subscribe (pub/sub) is a common design pattern used in event-driven architecture that provides a framework for exchanging messages between publishers and subscribers. A message broker is responsible for receiving all events from the publisher and routing them in real-time to subscribers who have expressed interest. The broker also records the events. Consumers can access the event stream at any time to read the most recent message or process the series of messages received since the last time they checked the stream.

A message broker allows the publisher to deliver notifications to a range of consumers across different devices and platforms. This approach eliminates the need for the publisher to know its subscribers, and it is unaffected if the number of interested parties scales up.

Challenges with traditional architecture and how EDA overcomes them

1. Availability
In traditional architectures, web apps depend on the real-time availability of services for data transfer. If a service is momentarily down, it could lead to errors and data not being processed. However, in an event-driven architecture, message brokers are used to storing events in a queue, ensuring that even if the destination service is down, events will be processed when the service comes back online.

2. Polling and webhooks
Polling and webhooks are inefficient ways of communicating states between different components in traditional architectures. Event-driven architectures allow for events to be filtered, routed, and pushed downstream to consuming microservices, resulting in less bandwidth consumption, CPU utilization, and potentially lower cost. This approach also reduces complexity, with smaller functional units and less code.

3. Polling and webhooks
Services that fire events do not need to contain the logic associated with other services’ processes. An event broker will retain all messages until they are read by the target service, enabling asynchronous communication.

4. Fault tolerance
Event-driven architectures are fault-tolerant, with faults isolated to individual services. If a subscribing service fails, it does not impact the service that sent the event. Communications are queued in a broker until the subscriber is repaired or replaced.

5. Reduced technical debt
Event-driven architectures allow individual services to operate and update independently, reducing inter-service dependencies and technical debt between releases.

6. Real time processing
Event-driven architectures make it easier to design near-real-time systems by generating events at the time when state in the application changes. Custom code should be designed to handle the processing of a single event, with scaling of the system as events scale up.

7. Real time processing
Event-driven architectures enable the decomposition of complex workflows into decoupled services that communicate asynchronously with event messages. This approach also makes it possible to assemble services that process data at different rates, with events buffered in a queue until they are processed by the appropriate microservice.

Some common solutions to implement EDA

Message queue: In an EDA, a message queue is often used as a solution for storing, accepting, and delivering messages. Messages can be any type of information that needs to be processed by the system, and the message queue helps to decouple the event producers from the event consumers. This ensures that events are processed efficiently and in a timely manner. 

Pub/Sub: Publish-subscribe, or Pub/Sub, is a messaging pattern that is commonly used to implement EDA. In this pattern, publishers of messages do not program messages to be sent directly to specific subscribers. Instead, messages are sent to a topic, and any subscriber interested in the topic will receive a copy of the message. This approach allows for a flexible and scalable EDA architecture. 

Event stream processing: Event stream processing is a solution for implementing EDA that involves analyzing events in real-time as they occur. This approach allows for the detection and response to events in near real-time. Event stream processing can be used to implement complex event processing (CEP), which is a technique for analyzing patterns in event streams. 

Event sourcing: Event sourcing is a solution for implementing EDA that involves storing all changes to an application state as a sequence of events. These events are then used to rebuild the application state at any point in time. This approach can be used to implement a scalable and fault-tolerant EDA architecture. 

Serverless functions: Serverless functions are a solution for implementing EDA that involves running code in response to events. This approach can be used to build highly responsive and scalable applications. Serverless functions can be used to implement a loosely coupled EDA architecture, where different functions can be triggered by different events. 

AWS – The one-stop leading solution for EDA 

AWS offers top-notch services for implementing EDA. Many AWS services can act as an event source for Lambda and generate events. In Lambda, your code is stored in a code deployment package, and all interactions with the code are done through the Lambda API. The service does not allow direct invocation of functions from outside. 

Lambda functions are not continuously running, unlike traditional servers. Instead, they are triggered by events, which is known as invocation. Serverless applications typically consist of multiple AWS services combined with custom code executed in Lambda functions. Although Lambda can integrate with most AWS services, the most commonly used services in serverless applications are:

There are several established patterns in distributed architectures that you can either build yourself or implement using AWS services. For most customers, investing time in developing these patterns from scratch has little commercial value. Therefore, use the corresponding AWS service when your application requires one of these patterns:

These services are specifically designed to integrate with Lambda, and infrastructure as code (IaC) can be used to create and delete resources in these services. You can use the AWS SDK to work with any of these services without the need to install applications or configure servers. Learning how to use these services through code in your Lambda functions is a crucial step in creating well-designed serverless applications.

Things to ponder while designing an EDA – Cons

• Event duplication: If not properly planned and coordinated, a single event may generate multiple identical messages across various services, resulting in communication gaps.
• Naming convention confusion: With multiple publishers and subscribers involved, it is easy to create naming errors or duplicate names, particularly across teams that do not regularly collaborate.
• Unclear workflow order: Each workflow step is triggered when a service or broker receives a specific event. If the receiving component is not correctly calibrated to respond to the event, it can lead to errors and create cascading workflow problems.
• Error handling and troubleshooting:A typical web-scale application may include numerous message brokers and services, which will continually pass and receive events. EDA requires comprehensive monitoring tools and practices to provide visibility into the event flow.
• Data contract: When building event-driven microservices applications, it is essential to agree on the data contract for the event producers and consumers. This will help to validate the events and automatically generate bindings for the programming language used.
• Event tracing: As all event-driven applications are distributed, it is critical to use tracing to comprehend and observe service dependencies and diagnose any bottlenecks and application issues.