Migrating Between EventSourcingDB Instances

This is not only a technical process, but also a conceptual one: migrating event streams means reestablishing the system's truth in a new place. It requires careful preparation, awareness of the system's invariants, and a clear understanding of what is – and isn't – allowed.

Migration Workflow#

EventSourcingDB supports migrations via its built-in backup and restore functionality. A backup captures all relevant data from an instance and stores it in newline-delimited JSON format. This includes:

• All events, in the order they were written
• All registered event schemas

Snapshots are implicitly included because they are modeled as regular events. There is no separate format or mechanism for exporting them. If your system uses snapshots, they will be backed up and restored automatically.

Licenses are not included in the backup. These must be configured separately on the target instance, either via a license file or inline string. The same applies to runtime configuration such as API tokens, TLS certificates, and port settings – these are not part of the database itself.

A typical migration consists of the following steps:

1. Put the source instance into read-only mode to prevent new writes during backup.
2. Create a backup using the HTTP API.
3. Changed in 1.3Optionally process the backup file to filter or transform events, which requires rebuilding its chain, as GDPR and Data Minimization describes.
4. Start a new instance on an empty data directory.
5. Restore the backup into the new instance.
6. Resume normal operation on the target system.

This approach ensures a clean and deterministic transfer of data between instances, with no race conditions or data loss.

Design Constraints and Limitations#

There are several important limitations to be aware of:

• The target instance must be completely empty. It is not possible to restore into an instance that already contains data.
• Events are immutable and strictly ordered. Individual events cannot be removed, rewritten, or selectively restored. You must restore a consistent sequence that starts from ID zero: a restore checks for every event that its ID follows on the one before it, that its content matches its hash, that it names the hash of the one before it, and that it is not older than that one.
• Backups and restores operate at the database level, not per subject or stream. You cannot extract a single stream or restore only part of a hierarchy.
• Changed in 1.3There is no mechanism for merging data from multiple backups or for appending new events to an existing instance via restore. The restore process assumes full control over the target system.

Because of these constraints, migration is most appropriate for clearly scoped, well-prepared scenarios – such as moving between environments or replacing infrastructure.

GDPR and Data Minimization#

Removed in 1.3Since EventSourcingDB does not delete individual events, the only way to remove or redact data after the fact, as the General Data Protection Regulation (GDPR) may require, is to export a backup, filter or anonymize it, and restore it into a new instance. That is a last resort, and harder than it looks: a restore checks the backup as a chain, as listed above, and refuses it at the first event that does not fit. An event whose data you changed no longer matches its hash, and an event you removed leaves a gap in the IDs.

So after changing or removing events, you have to rebuild the chain from the first event you changed or removed on: number the events again from 0 without gaps, compute the hash of every event again, and set the predecessor hash of every event to the hash of the one before it. Work on a full backup, and keep the time of every event as it is. EventSourcingDB provides no tooling for this; since a backup is a simple, line-based format, a script can do it. Rebuilding the chain changes the IDs and hashes of the events from the first change on, so read models, the positions of observers, and anything else that stored them have to be rebuilt as well, and the result of a previous audit no longer applies.

Added in 1.1Changed in 1.3How to compute the hash of an event is described in Verifying Event Signatures.

It is far easier to design for deletion from the start. Keep personal data out of events, store only opaque identifiers in them, and keep the personal data in an external system whose records you can delete. Or encrypt the personal data in events with a key per person, and destroy the key when the person asks for erasure, which is called crypto-shredding: the data stays in the store, but nobody can read it any more.

Whatever approach you choose, the key point is that compliance must be designed into your system. Event-sourced architectures are not inherently incompatible with GDPR, but they require proactive modeling and clear separation of concerns to support compliance safely.

Operational Considerations#

Before starting a migration, always ensure that the source system is in a stable state. Writing must be disabled, either via the read-only mode or by stopping the application that produces events. This guarantees that the backup reflects a consistent and complete snapshot of the system at a specific point in time.

The backup file can be large – especially in systems with millions or billions of events – but it is written in a streaming-friendly format that can be processed incrementally. You do not need to load the entire file into memory, and most filtering or transformation tasks can be done line by line.

Restoring into the target instance requires it to be empty, but no special mode or startup flag: the restore puts the database into lockdown by itself, in which no other reads or writes are allowed, and lifts it once the restore is complete. The instance then serves requests as usual, without a restart.

This design ensures that the restored state is trustworthy, deterministic, and uncorrupted by concurrent activity.

Planning for the Future#

Even if you do not currently plan to migrate your instance, it is a good idea to design your events and domain model with migration in mind. Avoid storing large amounts of sensitive data directly in events unless necessary. Use versioned event types, well-defined schemas, and clear subject hierarchies to make it easier to analyze, transform, and filter your data later.

EventSourcingDB provides a solid and transparent foundation for durable event storage. But it is your responsibility to define what should be stored, how long it should be retained, and how it should be interpreted. Migration is not just a technical tool – it is a strategic option that supports maintainability, compliance, and long-term evolution.