Missing-subject audit vs "Architecting the Agent Multiverse" (slide deck)¶
Source: Architecting_the_Agent_Multiverse.pdf (15 slides, NotebookLM deck of the
Chronos Protocol / CausalFlow research). This maps every subject in the deck to
the current library (v0.4.0) and specifies the missing features with enough
context to implement each. Companions: HANDOFF.md, ANALYSIS.md.
Slide-by-slide coverage¶
| # | Slide subject | Status in v0.4.0 |
|---|---|---|
| 1 | Multiverse DAG of forked executions | ⚠️ partial — see Gap 1 |
| 2 | Entropy of autonomy (alignment decay over 70h) | ✅ drift() entropy/health curve — Gap 6 |
| 3 | Diagnostic trap (14% LLM-judge; hand vs brain) | ✅ the library's premise |
| 4 | Execution-as-Code ("model may change, history must not") | ✅ cassette + seeds |
| 5 | Workflow/Activity dichotomy, event ledger, replay mode | ✅ record/replay; ⚠️ resume — Gap 5 |
| 6 | OS-level substrates (DeltaBox/CRIU/Wasm table) | ✅ documented out-of-scope |
| 7 | Execution Merkle DAG, CAS, O(1) branching, node = parent/action/output/fs hashes | ⚠️ linear chain only — Gap 1; action/output hash split — Gap 7 |
| 8 | Network determinism: VCR, virtual time, seeded entropy (uuid/rng) | ⚠️ rng only — Gap 2 |
| 9 | SCM (C_t → A_t → O_t → Y) + attribution formula | ✅ |
| 10 | do() calculus tracks (null vs forced intervention) | ✅ |
| 11 | Stochastic confound + Point-of-Commitment | ✅ |
| 12 | Shapley (AND/OR gates) + antithetic pairing | ✅ ground-truth tested |
| 13 | Step-level faithfulness: causality vs coverage quadrant | ❌ — Gap 3 |
| 14 | Repair loop (isolate → inject → validate, minimality) | ✅ repair v2 |
| 15 | Multiverse Console: timeline graph, branch compare, REPL in frozen state, audit trail | ❌ — Gaps 1, 4, 5 |
The gaps, with implementation context¶
Gap 1 — The Multiverse itself: first-class forking & persisted branches¶
Deck: slides 1, 7, 15 — "forking an agent at Step 45 creates a new node with a
shared parent hash… instantly instantiates a parallel universe"; the console
compares Failed vs Fixed timelines side by side.
Today: forking exists implicitly (every ablation rollout is a transient
fork via ReplayPlan), but nothing is user-facing or persisted: sessions are
linear; a counterfactual run cannot be saved, named, listed, or diffed.
Build (v0.5 candidate — highest value of these):
- fork(trajectory, at_step, *, do=None, seed) -> Trajectory in a new
multiverse.py: run ReplayPlan(held=<prefix, optionally forced>), record the
live continuation with a branch-aware recorder, return a new Trajectory
whose meta carries parent_session, fork_step, intervention.
- Store: add parent_session / fork_step columns (nullable, migration like
resamplable); CheckpointStore.branches(session_id) lists children.
Storage is already O(1)-ish: forked steps dedupe through the CAS blobs.
- diff(traj_a, traj_b) → first divergent step + per-step action diff — the
"State Diff" the research UI shows. CLI: agent-replay fork / branches / diff.
Gap 2 — Virtual time & entropy (clock/uuid determinism)¶
Deck: slide 8 — "hook system calls (wasi:clocks/get_time) to inject logged
timestamps; seed RNGs deterministically".
Today: only ctx.rng is deterministic. An agent calling datetime.now(),
time.time() or uuid.uuid4() silently breaks replay fidelity (held steps are
fine, but resampled/live paths diverge from wall-clock and fresh uuids, and any
recorded output embedding a timestamp changes its idempotency key across runs).
Build (small, high leverage):
- ctx.now() and ctx.uuid() on both contexts: during recording they capture
real values as steps (kind=memory, name=__now__/__uuid__,
resamplable=False); during replay they serve the recorded values — no API
change to the engine, they ride the existing cassette.
- instrument.RECIPES["time"] patching time.time/datetime.datetime.now and
uuid.uuid4 onto those ctx ops for unmodified agents (opt-in, since global).
Gap 3 — Step-level faithfulness (causality vs coverage)¶
Deck: slide 13 — correctness ⊥ faithfulness quadrant (correct-but-unfaithful
= "dangerous post-hoc rationalization"; wrong-but-faithful = "ideal debugging
target"); FaithCoT-style masking.
Today: the verifier scores outcome only; nothing measures whether the
recorded reasoning steps actually drive the answer.
Build (v0.5/v0.6, research-grade feature): faithfulness.py —
- Causality score per reasoning step: mask/remove step i (existing remove
intervention), hold everything else, N rollouts → shift in outcome
distribution; AUC over progressive masking = trajectory faithfulness.
- Classify runs into the 2×2 (correct×faithful) using outcome score ×
faithfulness score; flag correct-unfaithful runs in the report ("passed, but
the reasoning did not cause the answer") — these are the silent risks.
- Reuses AblationEngine wholesale; mostly a new scorer + report section.
Gap 4 — Multiverse Console (serve UI) with branch graph & frozen-state inspection¶
Deck: slide 15 — timeline graph with counterfactual branches, reverse-step
navigation, "Interactive REPL inside frozen states".
Today: static HTML report only.
Build (already roadmapped as v0.5 agent-replay serve; the deck adds specifics):
stdlib http.server browsing sessions → per-step state view (inputs/outputs =
the app-level "frozen state"), branch graph once Gap 1 lands, and a
"step back/forward" inspector. A true REPL into process memory is OS-substrate
territory (out of scope) — the app-level equivalent is dumping the step's
recorded inputs/outputs and context digest.
Gap 5 — Durable resume (event-sourcing recovery)¶
Deck: slide 5 — replay = "fast-forward through days of successful operation
in milliseconds" to resume a crashed run, not only to analyse it.
Today: replay serves attribution; there is no "continue this run live from
step k" convenience.
Build: thin wrapper over Gap 1's fork: resume(trajectory) =
fork(trajectory, at_step=len(trajectory), seed=...) — hold the entire recorded
prefix, run live from the end. Cheap once fork exists.
Gap 6 — Drift/entropy observability (nice-to-have)¶
Deck: slide 2 — alignment decay curve, context drift, silent failure. Today: nothing tracks per-step health over long runs. Build (optional): if the verifier can score intermediate state, record a per-step score series and chart it in the report — the "entropy curve" for a run. Low priority; useful mainly for long-horizon demos.
Gap 7 — Minor: split action vs output hash in the Merkle node¶
Deck: slide 7 — node = parent + action hash + output hash (+ fs root hash).
Today: step_hash = link(parent, op_key, output_hash) — the parts exist but
only the combined hash is stored. Persisting action_hash/output_hash
separately enables "same decision, different observation" queries and cheaper
diffing (Gap 1). Trivial store migration.
Priority recommendation¶
- Gap 2 (virtual time/uuid) — small, closes a real determinism hole.
- Gap 1 (fork/branches/diff) — the deck's headline; unlocks Gaps 4 & 5.
- Gap 5 (resume) — nearly free after Gap 1.
- Gap 4 (serve UI w/ branch graph) — already v0.5 roadmap.
- Gap 3 (faithfulness) — differentiating research feature.
- Gaps 6–7 — opportunistic.
Status (v0.6.0) — deck fully covered¶
- ✅ Gap 1 —
multiverse.fork/afork/diff,CheckpointStore.branches, CLI. - ✅ Gap 2 —
ctx.now/ctx.uuid,instrument.enable_virtual_time. - ✅ Gap 3 —
faithfulness()+ quadrant classification, CLI. - ✅ Gap 4 —
serve.pyMultiverse Console,agent-replay serve. - ✅ Gap 5 —
multiverse.resume. - ✅ Gap 6 —
drift()per-step entropy-of-autonomy curve + optional intermediate-statestate_scoreralignment-health overlay, silent-decay detection,DriftResult.to_htmlSVG chart, CLIdrift.stats.binary_entropy. - ✅ Gap 7 —
Step.action_hash/output_hash.
Every slide subject in the deck now maps to a shipped capability. Gap 6 resolved
its "needs an intermediate-state scorer" caveat by (a) providing the
state_scorer hook for callers who have one and (b) always charting the
verifier-only entropy-of-autonomy curve when they don't.