Dynamic consistency boundaries

On this page

  1. The problem streams cannot solve
  2. Tags and boundaries
  3. Union and intersection boundaries
  4. The append condition
    1. When appends collide
  5. Reading the boundary
  6. Models
  7. Identifiers carry their boundary
  8. How an append is written
  9. Snapshots
  10. What it costs
  11. Related

A consistency boundary is the set of events a decision must be consistent with. In the streamed model that set is a stream, chosen when you design the aggregate. Under dynamic consistency boundaries (DCB) it is a query over tags, chosen per decision and evaluated at the moment of the write.

Everything else is the same. Events are the same events, folding is the same fold, Result and ReadMode and the failure classifications are the same. Only what defines “consistent with what” changes.

The problem streams cannot solve

A school where a student may take at most ten courses, and a course has a fixed capacity. Subscribing has to check both:

  • the course exists and is not full — a fact about the course
  • the student is registered and not already on it — a fact about both
  • the student is on fewer than ten courses — a fact about the student

Pick any stream:

Stream What it cannot see
One per course The student’s other subscriptions
One per student How full the course is
One for the school Nothing — but now every subscription in the school is serialised

There is no right answer, because the decision’s boundary is not the same shape as any stream. It is this course and this student, and a different subscription has a different one.

Tags and boundaries

An event carries tags naming the things it concerns:

Add(new StudentSubscribedEvent(studentId, courseId),
    new Tag("course", courseId), new Tag("student", studentId));

A TagQuery is a boundary. The commonest shape is the events carrying any of its tags:

var boundary = TagQuery.AnyOf(new Tag("course", "maths"), new Tag("student", "alice"));

That one event above falls inside every boundary naming course:maths and every boundary naming student:alice. Seen through the course it fills a seat; seen through the student it uses up one of their ten. One event, two meanings, no duplication.

Union and intersection boundaries

AnyOf is a union: the events carrying any of its tags. AllOf is an intersection: only the events carrying all of them.

// Everything about the course, plus everything about the student.
var union = TagQuery.AnyOf(new Tag("course", "maths"), new Tag("student", "alice"));

// Only the events concerning both.
var intersection = TagQuery.AllOf(new Tag("course", "maths"), new Tag("student", "alice"));
Boundary Selects Use for
AnyOf(a, b) events carrying a or b a rule spanning both things — the course’s capacity and the student’s ten
AllOf(a, b) events carrying a and b a fact about the pair — is alice already on maths?

The subscription rule above needs the union: a boundary of only the events concerning both cannot see how full the course is. But “is alice already on maths?” is answered by a single event, and reading it through the union means folding every seat in the course and every course alice has ever taken to find it. The intersection reads that one event, and keeps doing so as the school grows.

Narrowing the fold also removes work from the model. A model over the union has to sort out what it folded — is this subscription about my course, or another of this student’s? — so its Apply is full of when subscribed.CourseId == CourseId guards. Under an intersection the boundary has already done that, and the guards go away.

Two things to be careful of:

  • An intersection narrows what you read, not what you may condition on. A decision that folded AnyOf(course, student) must condition its append on AnyOf(course, student). Conditioning on the narrower AllOf would accept an append resting on a capacity that has since changed. Condition on the boundary you folded, or a wider one.
  • An intersection is only as good as the tagging. CourseDefinedEvent is appended under the course alone, so it is not inside AllOf(course:maths, student:alice) however much it concerns the course that alice is on. If a model needs it, its boundary is a union.

A boundary that mixes the two — an or of ands — is not in this release. Neither is a per-boundary filter on event type: EventTypeFilter on the model does that job.

The append condition

Positions are global to the log, not per stream. A decision reads where its boundary stands, decides, and then appends on condition that the boundary has not moved:

var position = (await dcb.GetLatestPosition(boundary)).Value;

// ... fold the boundary, decide ...

var result = await dcb.SaveEvents(events, new AppendCondition(boundary, position));

If anything matching the boundary was appended in between, the decision rested on stale facts and the append is refused with memoria/concurrency-conflict — the same failure a stream conflict produces, so an existing retry policy works unchanged. The failure carries latestPosition, so a retry needs no extra read.

Two subscriptions contend only when their boundaries overlap. dave/maths and erin/greek share neither a course nor a student, so both commit; frank/greek and gina/greek share a course, so one of them loses.

Pass no condition to append unconditionally. That is correct only when the decision depended on nothing it read — registering a student, defining a course. An unconditional append still makes conditioned appends over the same tags fail; it simply has nothing of its own to be invalidated.

When appends collide

A refusal is the store working, not failing: the boundary moved, so the decision has to be made again against the new facts. Read the boundary again, fold it again, decide again and append again. When many writers share one boundary, wait a little between attempts, for a random time under a ceiling that grows with each refusal in a row. Retrying at once sends every loser straight back into the same race:

for (var refusals = 0; ; refusals++)
{
    var position = (await dcb.GetLatestPosition(boundary)).Value;

    var events = Decide(await dcb.GetEvents(boundary)); // fold and decide afresh every time

    var result = await dcb.SaveEvents(events, new AppendCondition(boundary, position));
    if (result.Failure?.Type != StoreFailures.ConcurrencyConflictType || refusals == 9)
    {
        return result;
    }

    // Full jitter: uniform under a ceiling of 1, 2, 4 ... 20 ms.
    var ceiling = Math.Min(20, Math.Pow(2, refusals));
    await Task.Delay(TimeSpan.FromMilliseconds(Random.Shared.NextDouble() * ceiling));
}

With sixteen writers on one boundary, that backoff took the accepted appends per second from 366 to 660 on one machine, and from 104 to 112 against a SQL Server container. The store leaves retrying to you because only the caller can decide again; the benchmarks have the full measurements.

Reading the boundary

Read the position before the fold, and never after:

var position = await dcb.GetLatestPosition(boundary);
var events = await dcb.GetEvents(boundary, model.EventTypeFilter);

The order is not cosmetic. The position is a claim about what the decision saw, and an event landing between the two reads breaks that claim in one of two directions:

Order An event lands in between Outcome
Position, then fold The fold sees more than the position admits The append is refused; retry
Fold, then position The position admits more than the fold saw The append is accepted on a decision that never read it

The second is a lost update, signed off by the very check meant to prevent it. Reading the position first is what makes the failure land on the safe side.

The position cannot simply come from the fold instead. The fold stops at the last event the model’s EventTypeFilter accepted, which can be behind the boundary’s head with nothing else running at all — so conditioning on it would refuse every append that happened to follow an event the model ignores.

There is no one-call form of this cycle, and that is deliberate: the position, the fold and the append belong to the decision, and a helper that hid them would hide the ordering above with them. UpdateAggregate is not it — like its streamed counterpart it only refreshes a snapshot and appends nothing.

Note also that GetInMemoryAggregate constructs the model itself, so a decision model that must know what it is about before it folds — one spanning two entities, say — reads with GetEvents and folds by hand instead.

Models

DcbAggregateRoot and DcbProjection reuse the fold from EventSourcedModel unchanged — version tracking, EventTypeFilter, Apply. They differ from AggregateRoot and Projection in exactly two ways: they belong to no stream, and they record how far they folded as a long LatestPosition across the whole log rather than an int sequence within one stream.

A read model differs from a write model in one thing only: it never produces events. Everything else is the same and is offered on both — identity, LatestPosition, the boundary in Tags, all four read modes, GetInMemory…, Get…, Update…, and the same Aggregate Folded diagnostics. Only Add, UncommittedEvents and SaveAggregate’s append belong to the write model.

IEventSourcedModel        Version, EventTypeFilter, Apply, IsEventHandled
 ├ IStreamedModel        + StreamId, LatestEventSequence   (int, within one stream)
 │   ├ IAggregateRoot    + AggregateId, UncommittedEvents
 │   └ IProjection       + ProjectionId
 └ IDcbModel             + LatestPosition                  (long, across the whole log)
     ├ IDcbAggregateRoot + AggregateId, Tags, UncommittedEvents
     └ IDcbProjection    + ProjectionId

Identifiers carry their boundary

IDcbAggregateId and IDcbProjectionId expose a TagQuery Boundary alongside Id. It is the DCB answer to IAggregateId.EventPropertyFilter — how this model’s events are selected — except that tags select on their own, so it is the whole boundary rather than a narrowing inside a stream:

public class SubscriptionDecisionId(string courseId, string studentId)
    : IDcbAggregateId<SubscriptionDecision>
{
    public string Id { get; } = $"{courseId}-{studentId}";

    public TagQuery Boundary { get; } =
        TagQuery.AnyOf(new Tag("course", courseId), new Tag("student", studentId));
}

// Everything aggregate-scoped then needs only the identifier
var decision = await dcb.GetAggregate(new SubscriptionDecisionId("maths", "alice"));

Binding them makes it impossible for a model and the events it may read to disagree. It does not fix a boundary at design time the way a stream does — the identifier is constructed per decision, so its boundary varies with it.

The boundary must be stable for a given Id. A snapshot is keyed by the boundary that produced it, so an identifier whose boundary changes — because you edited it and redeployed — misses its own snapshots and rebuilds them. Wasteful, never wrong.

The event-level methods (GetEvents, GetLatestPosition, SaveEvents) still take a TagQuery directly. They are not aggregate-scoped, so there is no identifier to carry one.

Once the model has been loaded, its own Tags are set from that boundary. A model spanning more than one entity can read them in Apply to know which ones it is about, and Add without explicit tags appends under the boundary the model was folded from.

How an append is written

On SQL Server and PostgreSQL an append is one round trip: a call to the store’s append routine, a stored procedure on SQL Server and a function on PostgreSQL, generated from your context’s model. See Install the store schema for how it gets into the database. The routine:

  1. Reads the boundary’s position and refuses at once if it has moved, before taking any lock.
  2. Locks the tag heads of every tag the append writes or is conditioned on, always in the same order, so two appends over overlapping tags wait for each other rather than deadlocking.
  3. Reads the boundary again with the locks held, and refuses if it moved while the append waited.
  4. Writes the events, their tags and the snapshot, if there is one, and commits.

Appends whose boundaries share no tag share no lock, so they run side by side. Each step reads afresh, so the check is correct with SQL Server’s READ_COMMITTED_SNAPSHOT either on or off.

On SQLite the store appends through Entity Framework Core instead: optimistically, guarding each tag head with a concurrency token, in a few statements. In process a round trip is almost free, so a routine would buy nothing. A context that implements IDcbDbContext without deriving from DcbDbContext takes that path on every engine.

Changes you have staged on the same context — an entity of your own added before the append — are saved in the append’s transaction on every path: they commit if the append is accepted and are left pending if it is refused.

Snapshots

A snapshot is keyed by the model and the boundary that produced it. The same aggregate id folded over a wider boundary is a different state, so reading it back under a different query misses rather than returning the wrong fold. All four read modes behave as they do for streams.

What it costs

  • One round trip per append on SQL Server and PostgreSQL, against the streamed store’s two, and in the benchmarks a DCB append is cheaper than a streamed one on both. The price is a routine in the database, installed alongside the tables. See Streams or DCB? for the numbers.
  • A row per distinct tag, forever. DcbTagHeads holds one row per tag ever written under or conditioned on — one per student, per course, per order. Small, and the price of contention that follows the boundary instead of the whole store.
  • Tags are case-sensitive, and the store pins a case-sensitive collation to keep the database agreeing with .NET. See Entity Framework Core (DCB).
  • Relational only. There is no Cosmos DB provider, for a reason that is not effort — see Providers.