Skip to navigation
Stream

Processing batches and checkpoints

View as Markdown

A durable consumer saves effects so that redelivery after a crash does not apply them twice.

Batch boundaries

A batch contains matching events from one atomic journal transition. sourceEventCount is the original count; matchedEventCount is the delivered count. Original ordinals are retained. A filter selecting only ordinal "2" from three events has not lost the other two in delivery: they did not match.

Zero matches produce no batch frame. A checkpoint can still advance through that position.

Before ACK

  1. Validate version, active session and the next frameSequence for every application frame.
  2. Validate counts, increasing ordinals and supported payloads. Filtered positions and ordinals can have gaps.
  3. Stage idempotent entity/revision/correction updates in journal order.
  4. On a checkpoint, save all staged effects through its position, dedup state and its complete token in one durable local transaction.
  5. After successful commit, ACK with the active connection ID and subscription generation.

A checkpoint may cover several batches: save the complete page, not only its last batch. A checkpoint with no batches still needs saving and ACK. Keep staging bounded by count/bytes/time. When capacity is insufficient, disconnect and resume from committed state rather than ACKing unsaved work.

Server transport acceptance does not establish your local commit. Your ACK asserts durable acceptance but does not control usage accounting. A physically sent event can be redelivered after reconnect.

Replay completion

caught_up means the subscription’s fixed cut was reached. It does not establish transaction finality. The same reader continues with live batches and checkpoints after the cut.

deliveryMode describes delivery, not event age. Replay is not a new trading signal; live delivery can also contain late observations. Use durable consumer pseudocode and recovery.