Incremental evaluation

Argon has two incremental systems with one deterministic artifact boundary:

They are deliberately separate. Source compilation is demand-driven and definition-granular. Runtime evaluation is transaction-stream-driven and tuple-granular. Salsa identities never enter an artifact; runtime tuples never invalidate compiler queries.

Incremental compilation

A long-lived compiler session retains source-file and workspace input identities. Changing a file updates its existing input. Declaration identity is semantic — file, name, declaration kind, visibility, and deterministic same-class occurrence — while its byte range is revision-local.

Parsing, declaration extraction, resolution, checking, macro expansion, elaboration, and dependency analysis are tracked at the narrowest sound boundary. An unchanged declaration subtree retains its downstream results even when earlier text moves its byte position. Package discovery still checks authoritative manifests and filesystem shape; it reuses a workspace input only when the resulting files, root, and preludes are unchanged.

Each build produces one .oxbin covering the entry package’s dependency closure: every reachable source file is elaborated and its events are merged into that single artifact. Artifact assembly is whole-package — an incremental build reuses parse, resolution, and check results and then reassembles the artifact from them — and the result must be byte-identical to a clean build of the same inputs.

Status: target architecture. Multi-package composition — deterministic per-package contributions combined by a workspace orchestrator, with unchanged encoded sections reused rather than re-encoded — is not shipped. Section-level reuse is what would make assembly itself incremental; today assembly is a whole-artifact cost paid on every build, and it is what dominates an edited one.

Incremental runtime evaluation

Status: target architecture. The sections below describe the intended DBSP runtime. The current runtime maintains the derived read-model with a delta-seeded semi-naive insert path (monotone programs) and DRedc two-counter retraction support, falling back to the batch stratified fixpoint elsewhere; the circuit/trace/frontier machinery described here is not yet shipped. The Incremental compilation section above is present-tense except where it carries its own status note.

The event log is the source of truth. A mutation advances the maintained model by the delta its transaction implies; the rule program is not re-run over the complete event history. This is the DBSP model described in the full technical paper and the SIGMOD overview.

The delta algebra, the Z-set representation, and the per-operator incrementalization (join arrangements, distinct zero-crossing, stratified negation, recursion) are runtime internals. The design record owns them: Incremental System Architecture §3.1–3.2.

Scope, frontier, and atomicity

Each (tenant, fork, module commitment) owns an independent evaluation scope with a monotonically published transaction frontier. Its retained state — traces, arrangements, dictionaries, and the commitments they are valid under — is enumerated in the design record (Incremental System Architecture §3.3).

A successful mutation appends one atomic event batch, advances the circuit with the corresponding signed delta, applies its derived delta, then publishes the new frontier. A read pinned to a published frontier observes base and derived state from exactly that frontier. No thread observes a frontier before both the event batch and maintained read model are ready.

If maintenance fails after the event batch is durable, the new frontier remains unpublished. The circuit epoch is poisoned and cannot accept another delta. Recovery restores the last valid arrangement checkpoint and replays the event suffix, or rebuilds from the event log through the batch evaluator. Partially advanced traces are never served.

The backend generation counter remains an atomic, monotonic coherence signal: Postgres increments it in the event transaction; the in-memory backend changes it under the same write lock as its indexes. Generation is not the evaluation algorithm. It detects stale cache epochs and coordinates recovery while the DBSP circuit performs tuple-level maintenance.

Bitemporal reads

The continuously maintained circuit serves the current published transaction frontier. A transaction-time historical read uses a checkpoint at or before its requested point and replays only the required event prefix, or falls back to a batch reconstruction. It does not rewind or mutate the live current-frontier circuit. Valid-time filtering remains part of the concrete read point and the rule-validity contract described in AsOf semantics.

Fallback and correctness

The batch evaluator remains the cold fallback and independent oracle. An unsupported circuit operator, incompatible arrangement section, poisoned epoch, or disabled incremental path rebuilds explicitly; no rule is silently omitted. Derived traces and arrangements are replaceable caches over the event log.

For every transaction prefix:

integrate(incremental output deltas) = batch evaluation at that prefix

The equality covers tuple weights, undefined companions, temporal extents, dictionary identities, and provenance projections. Compiler incrementality has the analogous obligation: incremental diagnostics and artifact bytes equal a fresh compilation.