Skip to navigation

Durable consumer

View as Markdown

The following is pseudocode, not a runnable SDK. It defines the persistence boundary: effects and checkpoint commit before ACK. Supply your own durable database and supported event reducer.

Store stream/history, consumer/subscription IDs, filter, generation/hash and frozen mappings. Also store dedup records, domain revisions/state, source quality and the last committed checkpoint position/token.

Processing loop

on every application frame:
verify the active session and contiguous frameSequence
on new hello:
create a local session epoch and fence old callbacks
validate protocol, scope and history; sequence starts at 1
authenticate; subscribe with saved token or explicit first live_cut
on subscribed:
validate binding and cut
start bounded staging for this session and generation
retain sourceQuality snapshot with its cut
on batch:
validate counts, ordinals and supported payloads
stage events in journal order, not observedAt order
on checkpoint:
validate active session, generation and covered batches
BEGIN local durable transaction
for every staged event through this checkpoint:
identical saved eventId: do not repeat its effect
conflicting saved eventId: abort processing
otherwise: apply entity/revision/correction and save dedup
save quality state and subscription binding
at the cut: reconcile replay quality with the cut snapshot
save checkpoint position and its entire opaque token
COMMIT
clear accepted staging
if session is still active: ACK its connection and generation
on caught_up:
mark the fixed cut reached; do not change transaction finality
on transport failure:
discard uncommitted staging; retain committed state
reconnect with finite backoff and the saved durable token
on reset_required:
stop automatic continuation; obtain explicit gap/start decision

A checkpoint with no batches still advances durable state and requires ACK. A checkpoint covering several batches commits all of them together. Validate sequence for control frames as well as batches.

The quality snapshot describes the cut, not an earlier replay position. At the cut reconcile scope state using journal positions and lastPosition; an older replay notice must not overwrite newer quality state.

Deterministic recovery checks

These are concrete acceptance oracles for your implementation. Positions and entity labels are synthetic, not wire credentials.

ScenarioMessages and boundaryExpected saved state
Crash before commitSaved C=3; receive batch at 4 and checkpoint 4; abort local transactionC remains 3, no new effect; resume redelivers 4 and applies it once
Commit before ACK / lost ACKCommit batch/checkpoint 4, then disconnect without accepted ACKC=4 and one effect survive; resume starts after 4 despite server ACK lag
Duplicate/conflictApply identical event again; then same ID with different contentsDuplicate adds no effect; conflict aborts acceptance
Zero matchesScan through 5 with no matching batch; receive checkpoint 5Save C=5; effect count unchanged
Old connectionActivate new session while old callbacks remainStale callbacks neither modify active state nor send its ACK
CorrectionApply revision 2 for a known entity/occurrenceState changes without adding a new execution or volume
Expired/unavailable cursorReceive reset with floor/historyStop; no implicit jump to live
RebindGeneration 1 cut at 1; new filter generation 2 scans to cut 2Apply new-filter suffix after 1; reject old-generation ACK

Implement corrections against actual domain identities, not merely an increasing effect counter. A missing target does not authorize fabricating an execution.

External actions

A database transaction does not make an external trading call exactly-once. Replayed data must not automatically repeat an action. External side effects require their own idempotency and reconciliation. Whalehunt does not execute trading commands in this product.

See recovery, corrections and frames.