CQRS and Event Sourcing

What is CQRS?

CQRS can be considered an extension to the Command Query Separation principle, which Bertrand Meyer described in Object-Oriented Software Construction. According to the Command Query Separation principle, methods within an object should be either commands or queries. A query returns data and does not modify the state of an object; a command mutates state.

CQRS takes this principle forward by creating two separate objects, one to handle commands and the other to handle queries. It is conceptually a straightforward pattern, that of separation. However, it is what the pattern enables that makes it interesting. Before we dive into what the pattern enables, let’s understand the scenario where CQRS is applicable.

CQRS is an architectural pattern applied at a significantly low level, typically within a bounded context or a sub-domain. CQRS does not apply to the whole system unless your entire system is a single bounded context. In addition, CQRS comes with an implementation cost, which means it should be used for bounded contexts, which are complex collaborative domains. Complex collaborative domains are domains in which multiple actors work on the same data. In such bounded contexts, the commands have complex business rules and many validations, and the queries require complicated calculations or pull large volumes of data. Simple CRUD systems are not a good fit for CQRS since they trade design simplicity for implementation complexity.

Another aspect to consider when using CQRS is the consistency requirements for queries and commands. Typically the commands need to work with up-to-date data and expect strong consistency. This implies that commands should not depend on querying data from other sub-domains, which lowers consistency requirements. Since there is a segregation between writes and reads, queries need to have the ability to tolerate some amount of staleness, and hence reads provide weaker consistency.

Let's now dive into what segregation using CQRS enables.

• Systems typically have different read and write characteristics. Systems might be read-intensive, write-intensive, or both. Segregation of reads and writes provides the opportunity to optimize them independently.
• A single conceptual model that tries to encapsulate both read and write operations, especially in a complex collaborative domain, may do neither well. Segregating the two leads to simpler, maintainable, and flexible domain models. Segregation can happen not only at the domain model level but also at the persistence layer. For example, when using a relational database, writes may use a highly normalized schema, and the reads can have a highly denormalized schema, thereby simplifying the reads.

What is Event Sourcing?

Event Sourcing and CQRS are frequently mentioned together, although neither necessitates the other. To understand Event Sourcing, it is important to define what an event is in the context of event sourcing.

An event is something of business significance that has already occurred in the system. Events are immutable, they cannot be modified once they have been recorded in the system. Events are treated as one way messages. An event has a single source, the publisher of the event and generally multiple consumers.

Event Sourcing is a persistence pattern in which the system’s state is persisted by storing the history of events for an entity. The history of events is then used to construct the entity's current state. The system which stores events in an Event Sourcing system is called an event store. For example, consider an inventory system that maintains the inventory of items for a catalog on a retail website. The system could persist the current state for items in two possible ways.

• It could store the count for a particular item and adjust it whenever purchased or canceled. You can think of the count as an integer value stored in a column within a table with a row for each item.
• It could store all the purchase and cancellation events for an item and then replay the events to calculate the current inventory for the item.

Event sourcing is inspired by domains like accounting, where an audit trail of all transactions needs to be maintained, and the events themselves are immutable. Once a transaction has happened, you cannot change it but only create a corrective or retractive event.

The primary benefit of event sourcing is that it provides an in-build audit mechanism to ensure transactional and audit data consistency. In addition, the representation via immutable events enables the construction of the system's state at any moment in time.

Event sourcing can also help with complex testing scenarios to ascertain that a certain action triggered a specific result. The result would be an event that the tests can verify.

CQRS with Event Sourcing

When used in conjunction with CQRS, you can use the events you persist in your event store and propagate all the updates made on the write side to the read side. The read side can use the information contained in the events to maintain whatever denormalized data you require on the read side to support your queries.

The event store publishes events after it persists them. This avoids the need to use a two-phase commit, which you would need if the write-side were responsible for saving the event to the event store and publishing the event to the read side. Normally, these events enable you to keep data on the read side up to date with some delay due to the transport mechanism.

Another benefit that is called out is the ability to rebuild the read-side from scratch or have a different representation. However, care must be exercised when doing this, especially if there are multiple consumers of events from the event store or downstream systems that depend on a particular representation of the read model.

When to use Event Sourcing

In my experience, Event sourcing is a good pattern to use if the domain requires temporal reasoning. Temporal data is data that has an associated time property to it. For example, Customer Loyalty might specify a time period for which a customer is eligible for benefits based on their spend each year or other business criteria.

Another area where Event sourcing is a good fit if the domain needs the ability to retract events. Order management, for example, might need a retract event for returns, or an inventory management system would need to retract events for overselling or underselling.

Event Store

The event store plays a big role when using event sourcing. The choice of the data model is important when either building an event store or selecting an existing one. Since there would be different kinds of events stored within the event store, it is important to decide whether the data store implements schema-on-write(relational databases) or schema-on-read(schemaless databases) semantics.

When an event store persists an event, it must also publish that event. To preserve the system's consistency, both operations must succeed or fail together. Traditional two-phase-commits are not an option due to lack of support from the data store and the messaging infrastructure, and the added performance overhead. One way to achieve consistency is to leverage change data capture(CDC) and the Outbox pattern. Using CDC, a tool like debezium can be used to read the append log of the database and push the events to a messaging infrastructure like Kafka. Since Kafka provides at least once guarantees, the consumer would need to take care of duplicates. The application would also need to take care of retries, write unprocessed messages after retries to a dead letter queue, and acknowledge processed messages.

For performance and scalability, saving an event should be a simple, fast, and append-only operation on the underlying store. During reads, if an instance has many events, it may affect the time it takes to replay all events to reload the state. One option to consider in this scenario is to use a snapshot mechanism. In addition to the full stream of events in the event store, you can store a snapshot of the instance state at some recent point in time. To reload the instance state, you first load the most recent snapshot and then replay all subsequent events.