# EigenScript Trace & Replay EigenScript can record its own execution to a tape and play it back. The tape captures every nondeterministic input a run consumed, so a replayed run produces byte-identical output: the same random sequence, the same `monotonic_ns` timestamps, the same HTTP responses. Two environment variables control it: | Variable | Effect | |----------|--------| | `EIGS_TRACE=` | Record: open `` for writing and log line, assignment, and nondet events | | `EIGS_REPLAY=` | Replay: open a previously recorded tape and serve its nondet values to builtins in order | ``` $ EIGS_TRACE=run.tape eigenscript sim.eigs > first.out $ EIGS_REPLAY=run.tape eigenscript sim.eigs > second.out $ diff first.out second.out # identical ``` Both are off by default. The disabled cost at each hook site is one predicted-not-taken load + branch. ## Tape Format The tape is plain text, one record per line, three record kinds: | Record | Meaning | |--------|---------| | `L ` | Source-line event (from `OP_LINE`). Adjacent duplicate lines with no `A`/`N` between them are deduped — the compiler emits per-statement LINEs and bare repeats are noise. | | `A =` | Assignment delta: a top-level binding changed. | | `N =` | Nondeterministic builtin return — the replay-determinism substrate. | ### Value serialization `N` records are written with full fidelity so they can be parsed back into real values on replay: - Numbers, `null`, and booleans are written verbatim. - Strings are double-quoted; `\"`, `\\`, `\n`, `\r` are escaped, other control/non-printable bytes become `\xNN`. - Lists and dicts are emitted recursively: `[1, 2, 3]`, `{"key": value}`. - Buffers get a leading `b` — `b[1,2,3]` — to disambiguate from lists. - Each record has a 64 KiB byte budget. On overflow the record ends with a `…` marker so partial records remain visually parseable (truncated records are not replayable; the builtin falls back to its live source). ## Recorded Builtins Every builtin whose return value is nondeterministic from the script's perspective lands on the tape as an `N` record: - **Random:** `random`, `random_int`, `random_normal`, `random_hex` - **Time:** `monotonic_ns`, `monotonic_ms` - **Environment / files:** `env_get`, `read_text`, `read_bytes`, `read_bytes_buf` - **HTTP extension:** `http_post` (success and all error paths), `http_request_body`, `http_session_id`, `http_request_headers` The hook is the `TRACE_NONDET_RET` macro in `src/trace.h`, used at every nondet return site — adding a new nondet builtin means wrapping its return in the same macro. ## Non-Replayable Builtins (issue #148) Some nondet builtins are *not* wrapped, and **fail loudly** when called under `EIGS_REPLAY`. They sit on the wrong side of the replay boundary because the tape's recorded return value does not pin down the host-side causal structure the call depends on — re-running the underlying source under replay would re-execute real side effects that the original tape neither captured nor re-creates: - **Subprocess streaming I/O:** `proc_spawn`, `proc_write`, `proc_read_line`, `proc_read`, `proc_close`, `proc_wait`. Replaying a recorded fd is meaningless — the child process from the recorded run does not exist; forking a fresh one would change the world a second time. - **Bulk-output exec:** `exec_capture` — same reason. The tape carries the captured stdout, but the child fork would still happen, and its real side effects (writes outside the captured pipe, file changes, network calls) would re-run. - **Concurrent channel receive:** `recv`, `try_recv`, `recv_timeout`. Channel ordering depends on the live scheduler — replay against a tape with a different interleaving would deadlock or silently diverge. These builtins raise a catchable runtime error under `EIGS_REPLAY`, with the message format `": not replayable under EIGS_REPLAY (subprocess/concurrency boundary; see docs/TRACE.md)"`. Programs that need to be replay-safe must guard these call sites or avoid them entirely. ## Replay Semantics With `EIGS_REPLAY` set, each nondet builtin call takes the next `N` record from the tape instead of invoking its underlying source. The contract: - **Strict ordering.** Records are consumed in tape order. The recorded *sequence* of nondet calls is the contract. - **Lenient names.** If the builtin name doesn't match the record's name, a warning is logged to stderr but the recorded value is used anyway — names are for human-readable debugging. Set `EIGS_REPLAY_STRICT=1` to make a mismatch fatal instead: the process reports the divergence and exits with status 3. Use it in harnesses where tape/program drift should fail loudly rather than produce a subtly wrong replay. - **Graceful exhaustion.** When the tape runs out, replay switches off and remaining calls hit the real source. - **Unparseable records** fall back to the builtin's live source. All value shapes round-trip: numbers, null, booleans, strings, lists, dicts, and buffers (including nested containers). Regression coverage: `tests/test_replay.sh` — each case mutates the underlying source (e.g. rewrites the file `read_bytes` read) between record and replay to prove the value comes from the tape. ## Temporal Interrogatives and `state_at` `prev of x`, the `at ` qualifier, and `state_at of line` query a per-name assignment history (line-stamped, append-only) that is fed by the same assignment hooks. This history is **independent of `EIGS_TRACE`** — it is language surface, always on, no tape required. The tape exists for cross-run reproducibility; the history exists for in-run time travel. - History tracks top-level (global) bindings — the same assignments that produce `A` records when tracing is on. - Recording is compile-gated: the compiler enables it when the program contains `prev of`, any `at ` qualifier, or a reference to `state_at` (and `EIGS_TRACE` enables it unconditionally). Programs with no temporal queries pay nothing per assign — profiling showed the previous always-on recording cost roughly a third of a dispatch-heavy workload's runtime. Since a program cannot observe history without containing a query, the gate is invisible — with one edge: code compiled mid-run (`eval`, REPL) that introduces the *first* temporal query starts recording at that point, so assigns executed earlier are not visible to it. Aliasing `state_at` through a dict or eval-built string also hides it from the compiler's scan. - When the compiled program contains a `where`/`why`/`how ... at` query, each history entry also stamps an observer snapshot (entropy, dH) at assign time, so the observer-derived interrogatives answer historically with exactly what a live query at that moment would have returned. The capture is compile-gated: no such query in the program, no per-assign cost. - `state_at of line` walks every tracked name's history backward and returns a dict of each binding's value at or before `line`. - Backward queries (`at`, `state_at`) are pruned by a periodic line-floor index: each 64-entry segment of a name's history caches its minimum line stamp, so segments that cannot contain a hit are skipped in one compare. Loop-heavy histories — thousands of assigns stamped with the same few lines, the debugger-scrub worst case — resolve in O(history/64) instead of O(history). The index adds one `int` per 64 history entries and an O(1) min-update per assign. - Per-assign cost of the history: one cache line + a pointer compare. Language-level syntax and examples: [SYNTAX.md](SYNTAX.md), [GRAMMAR.md](GRAMMAR.md). ## Debugger Step-Back The graphical debugger (`examples/debugger.eigs`) offers F8/F11 history navigation while paused. That layer does **not** read the trace tape: the tape tracks host-VM globals, and the meta-circular interpreter has its own env dict — so the debug hook captures its own `(line, env-snapshot)` pairs per statement, FIFO-capped at 10 000 steps.