Concepts / Analyzing / Designing Analytical Projections
Analyzing

Designing Analytical Projections

This guide explains how to turn an event stream into datasets for analysis, statistics, and machine learning. It covers the difference between operational and analytical projections, the qualities a good analytical projection needs, and the special case of time series.

Raw events hold a wealth of information, but they are rarely in the shape an analysis needs. A question such as how many loans did each title have per week cannot be answered by looking at a single event – it needs many events, grouped, counted, and put in order. Analytical projections do exactly this: they transform the event stream into structured, queryable datasets that make patterns, trends, and relationships visible.

They are the bridge between the immutable history in the event store and the dashboards, statistics, and models that build on it.

Operational and Analytical Projections#

Technically, an analytical projection works like any other read model: it reads events in order and derives a structure from them. The difference lies in its purpose.

Operational projections serve the running application. They answer a specific question in real time, often for a user interface – for example, which books are on loan right now. They need low latency and hold only what the current state requires.

Analytical projections serve understanding. They are built to:

  • Reveal trends and patterns over time
  • Calculate derived metrics such as rates, durations, and averages
  • Produce clean, consistent datasets for statistics and models
  • Support exploration of questions that are still taking shape

Because their requirements differ, it is usually best to keep both kinds separate. An analytical projection may take minutes to rebuild and hold years of history; an operational one must stay fast and small. Mixing them makes both worse.

Qualities of a Good Analytical Projection#

A good analytical projection is more than the result of a query. It should be:

  • Accurate – it reflects the underlying events faithfully, without silently dropping or double-counting any of them.
  • Complete – it contains all attributes a meaningful analysis needs, not just the ones the first question asked for.
  • Point-in-time correct – every value reflects only what was known at its moment in history – see Point-in-Time Correctness.
  • Rebuildable – it can be regenerated at any time from the original events, with the same result.
  • Documented – it states which events it is based on and which version of its logic produced it, so that results can be reproduced and audited.

These qualities make the dataset reliable, whether someone explores it interactively or a model is trained on it for years.

Example: Loans per Title per Week#

In a library, a projection could track loans per title per week. It is based on book-borrowed, loan-extended, and book-returned events, and derives for every title and week:

  • The number of active loans
  • The average loan duration
  • The overdue rate

The same projection serves two audiences. Library staff use it to monitor circulation and plan acquisitions. Models use it as historical data for demand forecasts and the detection of unusual patterns.

Time-Series Projections#

Many of the most valuable analyses rely on time series – sequences of values over time. They are the basis for forecasting demand, detecting anomalies such as sudden spikes or drops, and revealing seasonal or long-term patterns.

Event Sourcing is a natural source for time series, because the event store preserves the exact order in which things happened. A time-series projection turns this order into a timeline. To be useful, it should follow a few principles:

  • Consistent intervals. Fixed buckets – daily, weekly, monthly – make results comparable and easy to use in models and charts.
  • Complete coverage. A week without loans is a data point with the value zero, not a missing row. Gaps distort trends and confuse forecasting methods.
  • Point-in-time correctness. Each data point includes only what was known at the end of its interval.
  • A deliberate calendar. Timestamps are usually stored in UTC. Whether a day starts at midnight UTC or at midnight local time changes daily and weekly totals, so decide explicitly which calendar the business means.

Time series can also be enriched with derived measures such as moving averages or rolling sums, which smooth out short-term fluctuations and make trends easier to see.

Try it with EventSourcingDB

EventQL, the query language of EventSourcingDB, can group events by date parts such as YEAR, MONTH, DAY, and WEEKDAY, which is often enough to explore a time series before building a projection for it – see EventQL.

Rebuilding and Evolving#

An analytical projection is never the source of truth; the events are. This gives you a freedom that a traditional data warehouse lacks: a projection can be dropped and rebuilt at any time, and a new one can be built over the entire history, not just from the day it was introduced.

A projection that starts as a simple weekly count can later grow into a richer dataset with additional metrics – the events it needs have been there all along. See Optimizing Event Replays for how to keep rebuilds fast.

When the logic of a projection changes, version it. Record which version produced which dataset, so that an analysis or a trained model can always be traced back to the exact logic behind it. In regulated environments, this is often not optional.

From Projections to Models#

Analytical projections are where domain modeling meets data science. They precompute the signals that analyses and models need again and again – seasonal patterns, overdue frequencies, borrowing diversity. Having them ready:

  • Reduces preprocessing for everyone who works with the data
  • Allows more frequent retraining of models
  • Improves explainability, because every value can be traced back to the events that produced it

The next step, turning these datasets into the variables a model learns from, is described in Features from Events.

Designing for Analysis#

Design analytical projections around questions, not tables. Start from what someone wants to understand, keep the projection separate from the operational ones, make it rebuildable and documented, and treat time with care. The event history will support far more projections than you build today – and each new one can look back over everything that has already happened.