Closing the Loop

A prediction is not the end of the story. The value of analytics and AI emerges only when predictions lead to decisions and actions – and when the outcomes of those actions are fed back into the system. Event Sourcing is a natural foundation for this cycle, because every prediction, every action, and every outcome can be captured as an immutable event.

Predictions and Actions as Events#

A prediction should lead to a decision, and the decision to an action. In a library, for example:

• A model predicts a high risk of a late return, and the reader receives an early reminder.
• A forecast predicts high demand for a title, and the acquisitions team orders more copies.
• An anomaly alert fires, and someone starts an investigation before the issue grows.

Recording each of these steps as an event – late-return-predicted, reminder-sent, copies-ordered, investigation-started – creates a traceable link from the prediction to the response it triggered. A prediction event should also state which model version made it, and on which inputs, so that it can be explained and evaluated later.

Measuring the Impact#

With predictions, actions, and outcomes in the same history, their effect can be measured precisely:

• How often were the predictions correct? Every late-return-predicted event can be compared with the book-returned event that followed.
• Did the actions achieve their goal? Late returns of reminded readers can be compared with those of readers who were not reminded – see Causal Inference from Event Streams for doing this properly.
• How did the business change? Key figures before and after AI-supported decisions can be compared on the same, complete data.

Every recorded outcome is also new training data for the next version of the model. Past situations can be recreated exactly as they were, new models can be tested on them, and improvements can be verified without losing explainability.

Humans in the Loop#

Even good models misread signals, miss edge cases, or produce confident but wrong results. People bring what models lack: knowledge of exceptions, ethical judgment, and the ability to handle situations that have never occurred before. In a human-in-the-loop system, this judgment is part of the process by design, not an afterthought.

Such a loop typically has four steps:

1. Prediction – the model makes a recommendation.
2. Review – a person confirms, adjusts, or rejects it.
3. Action – the decision is carried out.
4. Recording – both the model's output and the person's decision are stored as events.

In a library, a recommendation system might suggest titles to readers, with a librarian reviewing suggestions for rare items first. Events such as recommendation-generated, recommendation-approved, recommendation-modified, and recommendation-delivered show where human input changed the result – and whether those changes made a difference.

Over time, this reveals where the model is reliable enough to act alone and where review remains valuable. Define clear criteria for when a person steps in, so that attention goes where it adds the most value, and feed the corrections back into training.

Detecting Drift#

Models degrade over time, because the world they describe changes. This is called drift, and it takes several forms:

• Data drift – the inputs change, for example when new genres become popular.
• Concept drift – the relationship between inputs and outcome changes, for example when borrowing behavior shifts after a policy change.
• Label drift – the outcome itself changes its meaning, for example when longer loan periods redefine what counts as late.

Because the event history records inputs, predictions, and outcomes in order, drift can be detected by comparing distributions over time and by monitoring accuracy against actual outcomes. Replaying the history makes it possible to test whether a new model would have performed better than the old one under the same past conditions. Combine statistical signals with the judgment of domain experts, so that normal fluctuations do not trigger unnecessary retraining.

Real Time and Batch#

A closed loop can run at different speeds. Real-time processing reacts to events as they happen – a late-return prediction the moment a book is borrowed, so that a reminder can still make a difference. Batch processing works on accumulated data at intervals – a monthly analysis of borrowing trends to plan acquisitions and staffing.

Real time offers immediate impact but demands low-latency infrastructure and sees less context at once. Batch offers depth and efficiency but arrives later. Most systems benefit from a combination: models score new events in real time and are retrained in batch on the full history; quick first estimates are refined later with features that take longer to compute.

Both modes can work on the same event history, which keeps them consistent. Decide per use case how quickly a result is actually needed, and use real-time processing where it truly adds value.

A Continuous Cycle#

Closing the loop is not a project with an end, but a cycle:

1. Predictions lead to actions.
2. Actions produce new events.
3. New events improve the next predictions.

With every turn, the system becomes better informed and more closely aligned with reality. Event Sourcing ensures that nothing along the way is lost: every prediction, decision, and outcome remains recorded, reproducible, and available to learn from.