Defining an event may seem straightforward, and when I wrote my first Kafka article I thought of all events as business events. I’ve now realized that there are various types, with Change Data Capture (CDC) events emerging as one of the most prevalent. These are some of the event definitions I’ve found related to Kafka:
FCT Events (Fact-based Events)
FCT, or Fact-based events, involve sending messages in the form of business events, transmitting only the altered data.
| Event | Id | Name | Address | PhoneNumber |
|---|---|---|---|---|
| UserAdded | 1 | Luke | 1 Street | 111-222 |
| AddressUpdated | 1 | 2 Street | ||
| PhoneNumberUpdated | 1 | 111-222 |
With this type of events, all events are needed to build the current state. That means that the events cannot be deleted without creating a snapshot.
If the primary objective is data distribution, consumers face the challenge of writing substantial logic to interpret FCT events while building the current state, resulting in duplicated code. However, for triggering specific actions, FCT events prove highly efficient, designed for a singular purpose.
CDC Events (Change Data Capture)
CDC, or Change Data Capture, events typically originate from a database connector capturing events such as insert, update, or delete. Messages consistently follow the same format, encompassing all data:
| Id | Name | Address | PhoneNumber |
|---|---|---|---|
| 1 | Luke | 1 Street | 111-222 |
| 1 | Luke | 2 Street | 111-222 |
| 1 | Luke | 2 Street | 222-333 |
For consumers, determining what has changed becomes challenging with CDC events. However, if data distribution is the main goal, the reason for the change becomes less crucial. Consumers can uniformly process each message, storing it in the database. Topics can be purged, retaining data for a specific period or only the latest messages.
There is of course also possible to send FCT events with all data is present in every message, as with the CDC events. The difference would then be that the CDC events are captured from the database, while the FCT events would be triggered based on a business event.
CMD (Command)
A Cmd is a message that is sent as a command, requiring an action. A command is something that can happen but might not, compared to events that we know has happened.
Given that a command is more of a request, the language used differs slightly from that of an event. Rather than using “UserAdded,” it is described as “AddUser”:
| Command | Id | Name | Address | PhoneNumber |
|---|---|---|---|---|
| AddUser | 1 | Luke | 1 Street | 111-222 |
| UpdateAddress | 1 | 2 Street | ||
| UpdatePhoneNumber | 1 | 222-333 |
Commands prompt actions, and their execution may result in the generation of events. While a command can be rejected, an event denotes a confirmed occurrence. Commands are designed for only one command handler.
How to choose
CDC events offer the advantage of providing data to downstream teams without overly involving the upstream team. They are easily processed with KSQL and Kafka Streams, allowing topic purging to save disk space. This approach is well-suited for legacy systems or high-pressure producing teams.
FCT events require extra effort to combine events into the current state, making them preferable for event sourcing within a specific microservice. Due to difficulties in purging, they may occupy additional disk space (snapshots can be used).
Producing messages based on business events is often preferable over CDC, especially when sending and consuming the entire data aggregate together. In such cases, a FCT event containing all data or the complete aggregate would be the preferred approach.
Commands should be used as requests that may be denied, and potentially leading to the generation of corresponding events. Commands are designed for single consumer consumption, although they can be produced by multiple sources. Hence, if your primary focus is on utilizing commands, Kafka may not be the most straightforward choice. Personally, my experience with commands and command handlers has been within the context of a CQRS.
Links
Read more here: