Cheetah / Cheetah documentation

Checkpoints and failures

A ViewSnapshot saves two kinds of information together: reducer state and one opaque end cursor per canonical source stream. The state says what the reducer has concluded; each cursor says which retained messages were included in that conclusion.

One cycle reads one page from every stream

For every suffix in reducer.source_streams, process() builds {canonical_user_id}/{suffix}, loads that stream's saved cursor, and asks history for at most batch_size messages after it. Streams are visited in reducer order.

There is no global timestamp merge across streams. If a result depends on exact ordering between two sources, put the relevant events in one ordered stream or establish product-owned ordering before reduction. Multiple sources work well for order-independent collections, latest-per-key maps, and commutative summaries.

After all source pages have been visited, a non-empty cycle saves one replacement snapshot. State and cursors are one store value, so a successful save cannot expose new state with old cursors or new cursors with old state through a conforming store.

That checkpoint proves only what it contains. It does not prove that another message has not arrived, that the processor has no lag, or that old history needed by a later rebuild still exists.

The default skip policy drops one bad reduction

With ViewErrorPolicy.skip, the engine must allow progress past a permanently malformed event without retaining a reducer's partial mutation.

For a page with messages, it keeps one rollback copy of the starting state. If a reducer call fails, the engine restores that page boundary and replays the page using an isolated candidate state for each event. A failed call contributes no state change. Successful calls before or after it are retained.

The failed event still counts in process()'s return value and advances the page cursor when the snapshot is saved. An all-skipped page therefore advances rather than poisoning every future run, but it sends no on_change callback.

This policy has three design consequences:

• view state must be deep-copyable;
• reducers must treat retained event dictionaries as read-only; and
• choosing skip means accepting permanent omission of a failed event from that view.

Log and monitor skipped events. The view store is not a dead-letter queue, and a skipped message cannot be restored by merely calling process() again after its cursor is committed.

The fail policy preserves the last commit

With ViewErrorPolicy.fail, the first reducer exception aborts the cycle and propagates to the caller. The engine performs no final save, so the committed state and cursors remain unchanged. The same history page can be read again after the reducer or event handling is corrected.

Maintained stores detach loaded snapshots from their committed values. In-place mutation before a fail-fast exception therefore cannot leak into the saved in-memory snapshot. A custom IViewStore must preserve the same save/load isolation contract.

Fail-fast processing still cannot undo reducer side effects elsewhere—which is why reducer I/O and external effects are forbidden.

Processing is at least once around publication

If a process stops after reducer calls but before the replacement snapshot is successfully saved, the committed cursor has not advanced. The next processor reads those messages again. Reducers must tolerate that replay according to their stated identity horizon.

A successful store save is the publication point. The subsequent synchronous on_change callback is outside that atomic write. Callback failure is logged, the snapshot remains committed, and Cheetah does not retry the callback as a durable delivery.

Consumed, applied, changed, and caught up are different

Avoid collapsing these observations:

None of these alone proves ongoing scheduler health. Combine them with processor health and a backlog or lag signal when freshness matters.

Next: Store, rebuild, and operate views.