Overview

The purpose of this post is to introduce Lambdora - a side project I’ve been working on that turns change data capture (CDC) events into a clear, queryable timeline of how data evolves over time.

Lambdora takes raw CDC streams and transforms them into structured change histories, showing what changed, when it changed, and what a database entity’s state looked like at any point in time. This makes it easier to debug production issues, analyze change patterns, audit data behavior, or simply prove that a specific change actually occurred.

In this post, I’ll explain what Lambdora is and why I built it, walk through its core components and architecture, explore the use cases it’s designed to address, and highlight its key features with a few demos.

Background Context

Lambdora started as a small side project over the Christmas holidays. The original goal was simple - I wanted to experiment with a few AI tools and models and see how far they could realistically take me when building something from scratch.

What surprised me most was how quickly the idea came together. With AI assisting throughout design discussions, implementation details, refactoring, and even edge-case reasoning, the project moved from a rough concept to a working prototype in just a few days. I originally shared the idea behind Lambdora in a LinkedIn post, which you can check out for some early context on how the project started.

As the holidays ended, Lambdora had already grown past a throwaway prototype. The core ideas felt solid, the use cases were real, and the tool was genuinely useful when pointed at actual CDC streams. That’s when I decided to keep building - refining the model, adding features, and treating Lambdora more like a potential production tool rather than a one-off experiment.

The goal of this post is to explore Lambdora in more detail, walk through a few demos, and gather early feedback before making it available for wider use.

What Is Lambdora

Lambdora is a service that integrates into your existing architecture through a change data capture (CDC) pipeline, for example using Debezium and Kafka. It consumes CDC events emitted by your systems and persistently records all data changes - creates, updates, and deletes - in a structured and queryable form.

Based on these events, Lambdora builds a historical timeline for each data entity. This timeline can be queried across specific time ranges, filtered by operation types, or narrowed down to changes affecting particular fields. The result is a clear and reliable view of how your data evolves over time and what exactly changed at each step.

Core Concepts

• Source - the system from which change events originated, e.g. a MySQL database in your application stack

• Collection - a logical grouping for change events within a source, e.g. a table in your MySQL database

• Entity - a single data entity within a collection, e.g. a row in your MySQL DB table

• Change Event - an immutable record indicating that a change occurred on an entity at a specific point in time, e.g. an update event on an entity

• Change Event Field - an individual field change within a change event, capturing how a specific field was modified

• Operation - the action performed in a given change event; Lambdora supports five operation types:

  • CREATE - indicates that an entity was created, e.g. INSERT statements in your MySQL DB table; this is either the first event for an entity or must occur after a delete event

  • UPDATE - indicates that one or more fields of an existing entity were modified, e.g. UPDATE statements in your MySQL DB table

  • DELETE - indicates that an entity was deleted, e.g. DELETE statements in your MySQL DB table

  • BACKFILL - indicates that an existing entity from the source system was created in Lambdora during the initial backfill when Lambdora was first bootstrapped for the given source; it is similar to a create event, but occurs only as part of a backfill process

  • SNAPSHOT - a Lambdora-specific operation representing a compaction result. Multiple historical events are collapsed into a single snapshot event that captures the accumulated changes. Snapshots can be created automatically based on retention rules (such as maximum event age or count per entity) or triggered manually via the Lambdora UI

• Change History - a chronological sequence of change events that represents the lifecycle of an entity over time

• Entity State - the complete reconstructed state of an entity at a specific point in time, derived by applying its change history up to that moment

• Entity Statistics - several metrics describing an entity’s lifecycle, including timestamps of the first and most recent change events and a breakdown of event counts by operation type. Statistics are available in two forms:

  • Lifetime Statistics - metrics that reflect the full lifecycle of an entity as observed by Lambdora, regardless of compaction; these counters include all events ever received, even if some of them have since been compacted

  • Queryable Statistcs - metrics calculated only from the change events that currently exist in Lambdora; events removed through compaction are excluded from these counts

Architecture & Components

Lambdora is designed to integrate cleanly with existing CDC-based architectures, layering observability and historical insight on top of your data changes without interfering with production systems.

Source Databases

Lambdora works with one or more source systems, such as MySQL or PostgreSQL, where data changes originate. These systems remain entirely unaware of Lambdora.

Debezium Kafka Connectors

Debezium connectors monitor the source databases and capture row-level changes (inserts, updates, deletes). These changes are emitted as CDC events and published to Kafka topics in a reliable, ordered manner.

Kafka CDC Topics

Kafka acts as the transport layer for CDC events. Topics provide buffering, ordering guarantees, and replayability, allowing Lambdora to process events asynchronously and at its own pace.

Lambdora CDC Processor

The CDC processor consumes events from Kafka, normalizes them into Lambdora’s internal change event model, and applies validation and enrichment logic (such as operation type classification and field-level diffs).

Lambdora Core

The core service persists change events, reconstructs entity state, maintains statistics, and applies retention rules. It exposes APIs for querying timelines, inspecting diffs, and time-traveling to historical states.

Lambdora UI

The UI sits on top of the core API and provides an interactive way for users to explore entity history, inspect changes over time, view statistics, and trigger manual compaction when needed.

Key Features

Lambdora focuses on making CDC data easy to understand, explore, and reason about. Rather than exposing raw event streams, it provides higher-level primitives for inspecting how data evolves over time.

Field-Level Change History

Lambdora captures changes at the field level, showing exactly what changed, from what value, and to what value for every event. This makes it easy to understand the intent and impact of each update, not just that a row changed.

Time-Travel Queries

You can query an entity’s state at any point in time. Lambdora reconstructs the full entity state by replaying relevant change events (and snapshots when available), enabling historical inspection and point-in-time debugging.

Queryable Change Timelines

Change events can be queried by:

• Time range

• Operation type (create, update, delete, backfill, snapshot)

• Affected fields

This allows you to narrow down large histories to only the changes that matter for a specific investigation.

Automated and Manual Compaction

To keep storage and query performance under control, Lambdora supports event compaction:

• Automated compaction based on event age or events count

• Manual compaction triggered via the UI

Compaction replaces multiple historical events with a snapshot that preserves the effective state while reducing history size.

Snapshot and Backfill Awareness

Lambdora distinguishes between:

• Backfill events, created during initial bootstrap

• Snapshot events, created during compaction

This makes it clear how and why an entity’s history was shaped over time, rather than treating all events as identical.

Entity-Level Statistics

For each entity, Lambdora exposes:

• First and last change event timestamps

• Total number of events and a breakdown by operation type

Statistics are available both as:

• Queryable statistics (based on retained events)

• Lifetime statistics (reflecting the full history, even after compaction)

Interactive UI for Exploration

The Lambdora UI provides a visual way to:

• Inspect current and historical entity state

• Browse change timelines

• View statistics

• Copy entity state as JSON

• Trigger compaction actions

This makes CDC data accessible not only to backend engineers but also to anyone analyzing data behavior for a specific database.

Demos

To make Lambdora’s concepts more concrete, this section walks through a series of short demos based on two instances of a simple orders service backed by relational databases - one using MySQL and the other USING PostgreSQL. Change events are captured via Debezium, processed by Lambdora, and explored through the UI.

Each demo highlights a specific capability and builds progressively on the previous one.

Exploring the Current Entity State

The first demo shows how Lambdora allows you to query the current state of an entity.

Browsing the Change Timeline of an Entity

Next, the demo switches to the change history view. This view presents a chronological timeline of all change events for an entity, allowing you to filter by time range, operation type, or specific fields. It also supports time-traveling through the entity’s history, making it possible to inspect how the entity evolved and reconstruct its state at any point in time.

Compaction in Action

The next demo focuses on compaction of change events, showing how Lambdora reduces long histories into efficient snapshots without losing the ability to query state accurately.

Exploring Entity Statistics

The next demo explores entity statistics and highlights the difference between queryable and lifetime metrics. Because compaction was performed in the previous demo, the number of queryable events is lower than the lifetime counts, which include events that have since been compacted.

Next Steps

Lambdora is still in its early stages, but I’m actively working toward making it easier for others to try it out in real environments. A key focus right now is preparing production-ready Docker images along with deployment templates that simplify running Lambdora locally or in managed infrastructure.

In parallel, I plan to share progress and updates regularly as the project evolves. The goal is to gather early feedback from engineers and practitioners who work with CDC and data-intensive systems, and to use that feedback for shaping Lambdora’s direction as it matures.

If this post has sparked your interest in Lambdora or you’d like to explore it further, feel free to reach out to me on LinkedIn - feedback and discussion are very welcome.