From 6e9ff337af21208b8ce53e61ea04beb509e1e715 Mon Sep 17 00:00:00 2001 From: InauguralPhysicist Date: Sat, 18 Jul 2026 01:34:59 -0500 Subject: [PATCH] fix(trace): tape-record file_exists / ls / mkdir / getcwd / exe_path (#585) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit These five fs builtins predated the trace tape and were neither recorded nor replay-blocked, so under EIGS_REPLAY they ran LIVE: a program branching on `file_exists of p` replayed against the current filesystem, not the recorded one — a tape recorded on one machine silently diverged on another. Same closed-world hole #471 closed for `args` and #148 covered for proc_*. The inconsistency was visible next to the already-taped is_dir (#576) and read_line (#558) in the same BUILTINS.md table. Each is now wrapped with the TAKE/RECORD pair (their live paths do real work — fopen, readdir bulk-list construction, a syscall — before the return exists, so TRACE_NONDET_RET alone would run the live work under replay). Under EIGS_REPLAY the early TAKE short-circuits before any fs access and serves the recorded value. mkdir is the design call: a WRITE whose return (a success bit) is filesystem-dependent. It is Recorded rather than #148-non-replayable because that bit IS pinnable by the tape (unlike a subprocess fd), and under replay the TAKE short-circuits before mkdir(2) — so the recorded bit is served and the directory is NOT created a second time. Replay does not re-run the write side effect, the same rule as the subprocess/audio boundary. Oracle: test_replay.sh records with the fs in one state, mutates it, and replays; the recorded answer must win (a live probe would now disagree). Verified JIT on and EIGS_JIT_OFF. mkdir additionally asserts the directory is not re-created on replay. exe_path uses the same getcwd pattern but is invariant between record and replay (same interpreter), so it is documented rather than mutation-tested. docs/TRACE.md lists the five under Recorded Builtins with mkdir's write-side-effect note; docs/BUILTINS.md marks each trace-recorded, closing the side-by-side inconsistency the issue flagged. Consumer gates green: liferaft/test/replay.sh (unaffected, N=1 argv only), tidelog/test/replay.sh (calls file_exists; byte-for-byte identical). Suite 3090/3090 release, 3094/3094 ASan+UBSan (detect_leaks=1, tally 0); test_replay.sh 23/23 incl. strict-mode. --- CHANGELOG.md | 13 ++++++++++ docs/BUILTINS.md | 10 ++++---- docs/TRACE.md | 9 ++++++- src/builtins.c | 49 +++++++++++++++++++++++++---------- tests/test_replay.sh | 61 ++++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 123 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d13e8c0..124c3c8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,19 @@ All notable changes to EigenScript are documented here. being recomputed by render and click separately. ### Fixed +- **`file_exists` / `ls` / `mkdir` / `getcwd` / `exe_path` are now + trace-recorded (#585).** These fs builtins predated the tape and ran live + under `EIGS_REPLAY`: a program branching on `file_exists of p` replayed + against the current filesystem, not the recorded one, so a tape recorded on + one machine silently diverged on another (the closed-world hole #471 closed + for `args`). Each is now wrapped with the `TAKE`/`RECORD` pair, so replay + serves the recorded answer and never touches the live fs. `mkdir` is + Recorded rather than #148-non-replayable — its success bit is pinnable by + the tape — and under replay the `TAKE` short-circuits before `mkdir(2)`, so + the recorded bit is served and the directory is not created a second time + (replay does not re-run the write side effect). Replay tests in + `test_replay.sh` mutate the fs between record and replay to prove the value + comes from the tape; verified JIT on and off. - **`prev of` now binds unary-or-tighter like every other `of` (#634).** `prev of x + 1` parsed as `prev of (x + 1)` — the operand parser used `parse_expression` where every other `of` uses `parse_unary` — and diff --git a/docs/BUILTINS.md b/docs/BUILTINS.md index b49912a..207b7b1 100644 --- a/docs/BUILTINS.md +++ b/docs/BUILTINS.md @@ -268,7 +268,7 @@ Boolean keywords that check the most recently observed value: | Name | Signature | Description | |------|-----------|-------------| | `load_file` | `load_file of "path.eigs"` | Load and execute EigenScript file. A missing/unreadable path raises a catchable `io` error (matching `import`); a parse/compile failure in the file raises `parse`. | -| `file_exists` | `file_exists of "path"` | 1 if file exists, 0 otherwise | +| `file_exists` | `file_exists of "path"` | 1 if file exists, 0 otherwise. Trace-recorded, so replay is deterministic (#585) | | `is_dir` | `is_dir of "path"` | 1 if the path names a directory, 0 for a plain file / missing path (#576 — replaces the `file_exists of "path/."` probe). Trace-recorded, so replay is deterministic | | `read_text` | `read_text of "path"` | Read file contents as string ("" on failure, 10 MB cap) | | `read_line` | `read_line of null` | Blocking line read from **stdin**: next line without its trailing newline (`\r\n` stripped as one unit), `null` at EOF; an empty line is `""`. Works on pipes — the stream-safe primitive `read_text of "/dev/stdin"` can't be (fseek fails on unseekable fds, #558). Trace-recorded, so replay is deterministic | @@ -285,10 +285,10 @@ Boolean keywords that check the most recently observed value: | `env_get` | `env_get of "VAR_NAME"` | Get environment variable (empty string if unset) | | `random_hex` | `random_hex of n` | Generate n random hex characters from /dev/urandom | | `try_parse` | `try_parse of code_string` | 1 if string is valid EigenScript syntax, 0 otherwise | -| `mkdir` | `mkdir of "path"` | Create directory (and parents). 1 on success, 0 on failure | -| `ls` | `ls of "path"` | List directory contents as list of strings | -| `getcwd` | `getcwd of null` | Current working directory as string | -| `exe_path` | `exe_path of null` | Absolute path of the running interpreter binary. Lets a script re-invoke the same interpreter (e.g. `exec_capture of [exe_path of null, file]`) without assuming `eigenscript` is on PATH | +| `mkdir` | `mkdir of "path"` | Create directory (and parents). 1 on success, 0 on failure. Trace-recorded: replay serves the recorded bit and does not re-create the directory (#585) | +| `ls` | `ls of "path"` | List directory contents as list of strings. Trace-recorded, so replay is deterministic (#585) | +| `getcwd` | `getcwd of null` | Current working directory as string. Trace-recorded, so replay is deterministic (#585) | +| `exe_path` | `exe_path of null` | Absolute path of the running interpreter binary. Lets a script re-invoke the same interpreter (e.g. `exec_capture of [exe_path of null, file]`) without assuming `eigenscript` is on PATH. Trace-recorded, so replay is deterministic (#585) | | `chdir` | `chdir of "path"` | Change working directory. 1 on success, 0 on failure | | `mktemp` | `mktemp of null` | Create temporary file, return its path | | `rm` | `rm of "path"` | Remove a file. 1 on success, 0 on failure | diff --git a/docs/TRACE.md b/docs/TRACE.md index 2f58cc3..ed136fa 100644 --- a/docs/TRACE.md +++ b/docs/TRACE.md @@ -57,7 +57,14 @@ perspective lands on the tape as an `N` record: - **Random:** `random`, `random_int`, `random_normal`, `random_hex` - **Time:** `monotonic_ns`, `monotonic_ms` - **Environment / files:** `env_get`, `read_text`, `read_bytes`, - `read_bytes_buf`, `read_line` (stdin, #558), `is_dir` (#576). + `read_bytes_buf`, `read_line` (stdin, #558), `is_dir` (#576), + `file_exists`, `ls`, `getcwd`, `exe_path`, `mkdir` (#585). + `mkdir` is a *write* whose return (a success bit) is filesystem-dependent: + it is Recorded rather than #148-non-replayable because that bit **is** + pinnable by the tape (unlike a subprocess fd). Under `EIGS_REPLAY` the + `TAKE` short-circuits before the `mkdir(2)` calls, so the recorded bit is + served and the directory is **not** created a second time — replay does not + re-run the side effect, the same rule as the subprocess/audio boundary. `read_bytes_buf`'s over-cap **raise** (#601) also rides the tape: the observed file size is recorded as a `VAL_NUM` `N` record (unambiguous — success records a `VAL_BUFFER`, open-failure records null) and the diff --git a/src/builtins.c b/src/builtins.c index d970351..7a3963e 100644 --- a/src/builtins.c +++ b/src/builtins.c @@ -2119,8 +2119,15 @@ Value* builtin_path_ext(Value *arg) { /* mkdir of "path" → 1 on success, 0 on failure. Creates parents. */ #if !EIGENSCRIPT_FREESTANDING +/* #585: mkdir's return (a success bit) is filesystem-dependent, so it is a + * taped nondeterminism source. Under EIGS_REPLAY the TAKE short-circuits + * before any mkdir(2), so the recorded bit is served and the filesystem is + * NOT mutated a second time — the same "don't re-run side effects on replay" + * rule as the proc-star / audio-capture boundary. Its return IS pinnable by + * the tape (unlike a proc fd), so it is Recorded, not #148-non-replayable. */ Value* builtin_mkdir(Value *arg) { - if (!arg || arg->type != VAL_STR) return make_num(0); + if (!arg || arg->type != VAL_STR) TRACE_NONDET_RET("mkdir", make_num(0)); + TRACE_NONDET_TAKE("mkdir"); /* Simple recursive mkdir */ char *path = xstrdup(arg->data.str); int len = strlen(path); @@ -2134,7 +2141,8 @@ Value* builtin_mkdir(Value *arg) { } free(path); struct stat st; - return make_num(stat(arg->data.str, &st) == 0 && S_ISDIR(st.st_mode) ? 1 : 0); + TRACE_NONDET_RECORD("mkdir", + make_num(stat(arg->data.str, &st) == 0 && S_ISDIR(st.st_mode) ? 1 : 0)); } #endif /* !EIGENSCRIPT_FREESTANDING */ @@ -2142,17 +2150,21 @@ Value* builtin_mkdir(Value *arg) { * Matches `ls -1` default behavior: hidden entries (starting with '.') are excluded. */ #if !EIGENSCRIPT_FREESTANDING Value* builtin_ls(Value *arg) { - if (!arg || arg->type != VAL_STR) return make_list(0); + if (!arg || arg->type != VAL_STR) TRACE_NONDET_RET("ls", make_list(0)); + /* #585: builds its return (a list) via readdir, so under EIGS_REPLAY the + * TAKE short-circuits before opendir — the recorded listing is served and + * the live directory is never read. */ + TRACE_NONDET_TAKE("ls"); Value *list = make_list(0); DIR *d = opendir(arg->data.str); - if (!d) return list; + if (!d) TRACE_NONDET_RECORD("ls", list); struct dirent *entry; while ((entry = readdir(d))) { if (entry->d_name[0] == '.') continue; list_append_owned(list, make_str(entry->d_name)); } closedir(d); - return list; + TRACE_NONDET_RECORD("ls", list); } #endif /* !EIGENSCRIPT_FREESTANDING */ @@ -2160,9 +2172,12 @@ Value* builtin_ls(Value *arg) { #if !EIGENSCRIPT_FREESTANDING Value* builtin_getcwd(Value *arg) { (void)arg; + /* #585: the cwd is process environment, nondeterministic across + * invocations and machines — taped so replay serves the recorded path. */ + TRACE_NONDET_TAKE("getcwd"); char buf[4096]; - if (getcwd(buf, sizeof(buf))) return make_str(buf); - return make_str(""); + if (getcwd(buf, sizeof(buf))) TRACE_NONDET_RECORD("getcwd", make_str(buf)); + TRACE_NONDET_RECORD("getcwd", make_str("")); } #endif /* !EIGENSCRIPT_FREESTANDING */ @@ -2174,14 +2189,18 @@ Value* builtin_getcwd(Value *arg) { #if !EIGENSCRIPT_FREESTANDING Value* builtin_exe_path(Value *arg) { (void)arg; + /* #585: the interpreter path is machine-dependent — taped so replay + * serves the recorded path without touching /proc/self/exe. */ + TRACE_NONDET_TAKE("exe_path"); char buf[4096]; ssize_t n = readlink("/proc/self/exe", buf, sizeof(buf) - 1); if (n > 0 && n < (ssize_t)sizeof(buf)) { buf[n] = '\0'; - return make_str(buf); + TRACE_NONDET_RECORD("exe_path", make_str(buf)); } - if (g_argv && g_argc > 0 && g_argv[0]) return make_str(g_argv[0]); - return make_str("eigenscript"); + if (g_argv && g_argc > 0 && g_argv[0]) + TRACE_NONDET_RECORD("exe_path", make_str(g_argv[0])); + TRACE_NONDET_RECORD("exe_path", make_str("eigenscript")); } #endif /* !EIGENSCRIPT_FREESTANDING */ @@ -2844,10 +2863,14 @@ Value* builtin_load_file(Value *arg) { #if !EIGENSCRIPT_FREESTANDING Value* builtin_file_exists(Value *arg) { - if (!arg || arg->type != VAL_STR) return make_num(0); + if (!arg || arg->type != VAL_STR) TRACE_NONDET_RET("file_exists", make_num(0)); + /* #585: fs-dependent read — taped so replay serves the recorded answer. + * TAKE short-circuits before the fopen probe under EIGS_REPLAY. */ + TRACE_NONDET_TAKE("file_exists"); FILE *f = fopen(arg->data.str, "r"); - if (f) { fclose(f); return make_num(1); } - return make_num(0); + int ex = (f != NULL); + if (f) fclose(f); + TRACE_NONDET_RECORD("file_exists", make_num(ex)); } #endif /* !EIGENSCRIPT_FREESTANDING */ diff --git a/tests/test_replay.sh b/tests/test_replay.sh index 33d5032..68224bb 100755 --- a/tests/test_replay.sh +++ b/tests/test_replay.sh @@ -329,6 +329,67 @@ else fail "is_dir replay" "rec='$REC_ID' rep='$REP_ID'" fi +# ---- #585: file_exists / ls / mkdir / getcwd are taped nondeterminism ---- +# Each records with the filesystem in one state, mutates it, then replays; +# the recorded answer must win (a live probe would now disagree). mkdir also +# asserts the write side effect does NOT re-run on replay. + +# file_exists: record present, delete, replay must still say 1. +mkdir -p "$TMPDIR/fe585" +touch "$TMPDIR/fe585/probe" +cat > "$TMPDIR/p_fe.eigs" <&1) +rm -f "$TMPDIR/fe585/probe" +FE_REP=$(EIGS_REPLAY="$TMPDIR/fe.tape" "$EIGS" "$TMPDIR/p_fe.eigs" 2>&1) +if [ "$FE_REC" = "1" ] && [ "$FE_REP" = "1" ]; then + ok "file_exists replay: recorded 1 wins after the file is deleted (#585)" +else + fail "file_exists replay (#585)" "rec='$FE_REC' rep='$FE_REP'" +fi + +# ls: record two entries, delete them, replay must serve the recorded count. +mkdir -p "$TMPDIR/ls585" +touch "$TMPDIR/ls585/a" "$TMPDIR/ls585/b" +cat > "$TMPDIR/p_ls.eigs" <&1) +rm -f "$TMPDIR/ls585/a" "$TMPDIR/ls585/b" +LS_REP=$(EIGS_REPLAY="$TMPDIR/ls.tape" "$EIGS" "$TMPDIR/p_ls.eigs" 2>&1) +if [ "$LS_REC" = "2" ] && [ "$LS_REP" = "2" ]; then + ok "ls replay: recorded listing wins after the entries are deleted (#585)" +else + fail "ls replay (#585)" "rec='$LS_REC' rep='$LS_REP'" +fi + +# mkdir: record success, remove the dir, replay must serve 1 AND not re-create it. +cat > "$TMPDIR/p_mk.eigs" <&1) +rm -rf "$TMPDIR/mk585" +MK_REP=$(EIGS_REPLAY="$TMPDIR/mk.tape" "$EIGS" "$TMPDIR/p_mk.eigs" 2>&1) +if [ "$MK_REC" = "1" ] && [ "$MK_REP" = "1" ] && [ ! -d "$TMPDIR/mk585/sub" ]; then + ok "mkdir replay: serves recorded bit, does NOT re-create the directory (#585)" +else + fail "mkdir replay (#585)" "rec='$MK_REC' rep='$MK_REP' recreated=$([ -d "$TMPDIR/mk585/sub" ] && echo yes || echo no)" +fi + +# getcwd: record in one directory, replay from another, recorded path must win. +cat > "$TMPDIR/p_cwd.eigs" <<'EOF' +print of (getcwd of null) +EOF +mkdir -p "$TMPDIR/cwd585" +CWD_REC=$(cd "$TMPDIR/cwd585" && EIGS_TRACE="$TMPDIR/cwd.tape" "$EIGS" "$TMPDIR/p_cwd.eigs" 2>&1) +CWD_REP=$(cd / && EIGS_REPLAY="$TMPDIR/cwd.tape" "$EIGS" "$TMPDIR/p_cwd.eigs" 2>&1) +if [ -n "$CWD_REC" ] && [ "$CWD_REC" = "$CWD_REP" ]; then + ok "getcwd replay: recorded cwd wins when replayed from a different directory (#585)" +else + fail "getcwd replay (#585)" "rec='$CWD_REC' rep='$CWD_REP'" +fi + # ---- #579: audio capture is a taped nondeterminism source ---- # Gated: needs a gfx build AND a working capture device (the dummy SDL # driver provides a silent one; the CI dev image has no libSDL2, so this