tests: the CALC ladder — a ratchet for the CALC.PAS work

CALC-COMPILE.md §1 asks for a NUMBER for "how far into CALC.PAS does the
compiler get", measured before the work that moves it.  Stage 1 lands it.

tests/fixtures/calc_ladder.tsv holds one row, in expected.tsv's shape:
CALC.PAS  ERR  1  7791 (FXMax : Char = 'G', a typed constant).  The row may
only move forward, and the normalised-pos trap is written beside it.

tests/calc_ladder.sh stages a COPY of CALC.PAS and CALC.INC into
tmp/calc_ladder/ — a copy because the Stage 2 include resolves relative to the
including file — runs the built compiler on it, and asserts the exact verdict.
run_all.sh runs it as "CALC ladder".  It is not a fixture in tests/fixtures,
which would enter the compile/COM/exec matrices and need a second CALC.PAS kept
in sync.

The build is not redone in the ladder: run_compile_tests.sh grows a
--build-only branch, so there is ONE recipe that freshness-checks compiletest.
Two staleness tests that could drift are one test wearing two hats, which is
how this harness once reported the previous compiler's verdicts with total
confidence.

Proved red before it was made green: with the row set to 7792 the ladder
reported "expected: ERR 1 7792 / actual: ERR 1 7791" and exited 1.

Also on resuming: the working tree held a leftover nonvacuity mutation of
shell/Compiler.mod (the frame-rule locFree revert) from a nonvacuity.sh run
killed before its restore trap.  It was reverted before anything was measured;
the compile matrix stays green with it in place, so it would have been
invisible — the mutation is byte-for-byte identical for INTEGER locals.

run_all.sh: OVERALL ALL PASS (compile matrix 56/56, CALC ladder 1/1).
tests/nonvacuity.sh: 78 ok, 0 failed.

CALC-COMPILE.md

@@ -52,19 +52,32 @@ A 1,763-line program (182 + 1,581 lines) that stops on the 111th line gives
 no sense of progress unless "how far did we get" is a number in a test.
 Make it one.
 
-**Proposal — `tests/fixtures/calc_ladder.tsv` (or a `run_compile_tests.sh`
-section):** run the compiler on a *copy* of `CALC.PAS` (plus, once it exists,
-`CALC.INC`) and assert the exact verdict:
+**Implemented (Stage 1, 2026-10-19) — `tests/fixtures/calc_ladder.tsv` plus
+`tests/calc_ladder.sh`,** run by `run_all.sh` as `CALC ladder`. Each row names a
+program and gives the exact verdict in the same shape as
+`tests/fixtures/expected.tsv`:
 
 ```
-calc_ladder   ERR   <errno>   <pos>
+CALC.PAS   ERR   <errno>   <pos>
 ```
 
-Each stage updates the row; the row may only move forward. This is the same
-shape as `tests/fixtures/expected.tsv`, so it reuses the existing harness
-instead of inventing a second one.
+The script stages a *copy* of the named program — and of the companions that
+sit beside it, so `CALC.INC` is already there for Stage 2 — into
+`tmp/calc_ladder/`, runs the freshly built compiler on it, and compares. It
+does not build the compiler itself: it calls
+`tests/run_compile_tests.sh --build-only`, which is the one recipe that
+freshness-checks `compiletest`, so the ladder cannot become the second
+staleness test that drifts from the first.
 
-Two traps to write down next to it:
+The copy is deliberate: the include (Stage 2) resolves relative to the
+including file, so `CALC.INC` has to be beside `CALC.PAS` in the staged tree,
+and the row then measures the compiler rather than an edit-in-place of
+`Resources/TP3`. It is not a fixture in `tests/fixtures/*.pas` (which would
+enter the compile/COM/exec matrices and need a second copy of `CALC.PAS` kept
+in sync).
+
+Each stage updates the row; the row may only move forward. Two traps are
+written next to the file itself:
 
 - **`pos` is not a file offset.** `LoadFile` collapses CRLF *and* lone LF to a
   single CR, so `pos` indexes the *normalised* buffer. `CALC.PAS` is 11,241
@@ -336,17 +349,32 @@ ladder order (each stage removes the blocker the previous one exposed).
 
 ### Session state — where to resume
 
-Last touched 2026-10-17. **No compiler change has been made yet**; this line of
-work is still entirely at the planning stage, and everything below is the
-*pre*-CALC baseline that Stage 1 must not break.
-
-- **HEAD** `ce7e8cd` (this document). Nothing pushed; `origin/master` is behind.
+Last touched 2026-10-19. **No compiler change has been made yet** — Stage 1 is
+the measuring device, not a compiler feature, so the front end is still exactly
+the *pre*-CALC baseline that every later stage must build on.
+
+- **HEAD** `ce7e8cd` (this document) plus the Stage 1 commit. Nothing pushed;
+  `origin/master` is behind.
+- **Stage 1 (the ratchet) is landed:** `tests/fixtures/calc_ladder.tsv` +
+  `tests/calc_ladder.sh`, wired into `run_all.sh` as `CALC ladder`. The row is
+  `CALC.PAS  ERR  1  7791`, and it was **proved red first** (row set to 7792 →
+  `FAIL expected ERR 1 7792 / actual ERR 1 7791`, rc 1) before the real value
+  was written back. The compile matrix is still **56/56**.
+- **One surprise on resuming:** the working tree was found holding the
+  nonvacuity script's *frame-rule* mutation of `shell/Compiler.mod` (the
+  `locFree := 0FFFEH` / pre-decrement-base revert), i.e. a mutation from a
+  `tests/nonvacuity.sh` run that the trap did not restore — presumably killed
+  before the restore. It was reverted with `git checkout -- shell/Compiler.mod`
+  before anything was measured. Worth knowing: the mutation's whole shape is
+  byte-for-byte the same for INTEGER locals, so the compile matrix would have
+  been green and only `t40_readln` (and later CALC) would have noticed.
 - **Verified baseline, run on this commit:** `./tests/run_all.sh` →
-  **OVERALL: ALL PASS** (~74 s), and the compile matrix is **56/56**. Those are
+  **OVERALL: ALL PASS**, compile matrix **56/56**, CALC ladder **1/1**. Those are
   the numbers Stage 1's ratchet must leave intact.
 - **Not re-run this session:** `tests/nonvacuity.sh` (78 ok / 0 failed at the
   last run). It is not in the default `run_all.sh` and must be run **manually**
-  after touching any check.
+  after touching any check — including, now, `run_compile_tests.sh`'s
+  `--build-only` branch.
 - **Still present and deliberately untracked:** `Resources/TP3/CALC.COM` and
   `Resources/TP3/HELLO.COM`, the two reference binaries used for §3.1. They are
   derived from the proprietary `TURBO.COM`; do not stage them. The recipe and
@@ -379,10 +407,11 @@ work is still entirely at the planning stage, and everything below is the
 
 ### Next session
 
-**Stage 1 — the ratchet (§1).** Small. Lands the measuring device before the
-work it measures. Green for the current blockers, and it must be *proved red*
-first (assert a deliberately wrong position, watch it fail, then set the real
-one).
+**Stage 1 — the ratchet (§1). DONE.** `tests/calc_ladder.tsv` +
+`tests/calc_ladder.sh`, wired into `run_all.sh`; proved red against 7792 before
+it was set to 7791. It also resolved Q4 in favour of staging copies in
+`tmp/calc_ladder/` rather than teaching the compile matrix about companion
+files.
 
 **Stage 2 — `{$I file}` include.** Small, self-contained, and a prerequisite:
 `CALC.PAS:150` *is* the program.
@@ -566,10 +595,12 @@ as the include, because the include is what consumes the slack.
 - **Q3.** `Pages = array[0..200] of string[80]` — confirm the element size is 81
   and that `New (Buffer)` gets 16,281 contiguous bytes from the heap the way
   TP3's `New` does.
-- **Q4.** Does `run_compile_tests.sh`/`CompileTest.mod` support a fixture with a
-  companion `.inc` (Stage 2's fixture, and `calc_ladder` needs a copy of
-  `CALC.INC`)? If not, decide whether the ladder lives in the harness or in a
-  small dedicated script.
+- **Q4 — RESOLVED (Stage 1).** `run_compile_tests.sh`/`CompileTest.mod` still
+  have no fixture-with-companion-file notion, and they do not need one: the
+  ladder is a small dedicated script (`tests/calc_ladder.sh`) that stages
+  `CALC.PAS` and `CALC.INC` as copies in `tmp/calc_ladder/`. That is also what
+  makes the Stage 2 include resolve, since it opens the `.inc` relative to the
+  including file.
 - **Q5.** Are range/index checks on by default for `TURBO.COM`? `CALC` sets
   neither `{$R}` nor `{$V}`; if the default is *on*, `TU_RANGE_CHECK` and
   `TU_INDEX_CHECK` are on the critical path for every index in CALC, not an

SUMMARY.md

@@ -2036,8 +2036,10 @@ reals → string builtins → files → CRT → recursion → layout), plus the
 census of what `CALC.PAS` actually needs and the reference measurements from
 the original `TURBO.COM`. Two tracks run in parallel: the ladder (front-end,
 verifiable on small fixtures) and the layout (§3, the only thing that stops
-the compiler *emitting* a 16,560-byte program). Items 1-4 below are the
-string/robustness work and are independent of both.
+the compiler *emitting* a 16,560-byte program). Stage 1, the ratchet, is landed
+(`tests/calc_ladder.sh` + `tests/fixtures/calc_ladder.tsv`, run by
+`run_all.sh`; CALC.PAS currently stops at `ERR 1` at pos 7791). Items 1-4 below
+are the string/robustness work and are independent of both.
 
 1. The rest of the string language. `v-TP3-STRVAR`/`v-TP3-STRREAD-ASSIGN` cover
    string variables, `s := 'literal'`, `writeln (s)`, `readln (s)`/`read (s)`

shell/tests/calc_ladder.sh

@@ -0,0 +1,80 @@
+#!/bin/bash
+# The CALC ladder — a NUMBER for "how far into CALC.PAS does the compiler get".
+#
+# CALC-COMPILE.md §1: a 1,763-line program that stops on one line is not
+# progress unless the stopping point is asserted somewhere.  Each row of
+# tests/fixtures/calc_ladder.tsv is that assertion — the exact verdict, and
+# once it compiles the exact code/data sizes — and every stage of that plan
+# moves the row forward, never backward.
+#
+# The compiler runs on a COPY, staged in tmp/calc_ladder/, for two reasons:
+#   - the include (CALC-COMPILE.md Stage 2) is resolved relative to the
+#     including file, so CALC.INC must sit beside CALC.PAS in the staged tree;
+#   - the row then measures THIS compiler, not an edit-in-place of Resources.
+#
+# The build is deliberately not redone here.  tests/run_compile_tests.sh
+# --build-only is the one recipe that builds and freshness-checks compiletest,
+# so there is no second staleness test to drift from the first — which is how
+# this project once reported the previous compiler's verdicts with total
+# confidence.
+set -u
+D=/home/eric/Projets/Projets-Modula2/MyWork/TP3-comp/shell
+cd "$D" || exit 9
+
+# One build recipe: run_compile_tests.sh checks compiletest against every input
+# and rebuilds when it is older.  Its build line is kept visible.
+tests/run_compile_tests.sh --build-only || exit 1
+[ -x compiletest ] || { echo "no compiletest after the build"; exit 1; }
+
+TSV=tests/fixtures/calc_ladder.tsv
+[ -f "$TSV" ] || { echo "no ladder file $TSV"; exit 1; }
+
+STAGE=../tmp/calc_ladder
+mkdir -p "$STAGE"
+rm -f "$STAGE"/*
+
+# Stage a program and everything that has to sit beside it.  Companions are
+# copied whether or not the compiler can read them yet: once {$I} lands, the
+# include is what turns CALC.PAS into the whole program, and a ladder that
+# reported "cannot open CALC.INC" for a file that was never staged would be
+# measuring the staging, not the compiler.
+stage () {
+    prog=$1
+    cp "../Resources/TP3/$prog" "$STAGE/$prog" || exit 1
+    case $prog in
+        CALC.PAS) cp ../Resources/TP3/CALC.INC "$STAGE/CALC.INC" || exit 1 ;;
+    esac
+}
+
+# Normalise compiletest's one-line verdict the same way run_compile_tests.sh
+# does: drop the path, and turn "OK code=N data=M" / "ERROR n at pos p" into
+# the row spelling "OK N M" / "ERR n p".
+verdict_for () {
+    printf '%s\n' "$1" | ./compiletest \
+        | grep -F "$1" | head -1 | tr -d '\r' \
+        | sed -E 's/^[^ ]+[[:space:]]+//' | tr -s ' ' \
+        | sed -E 's/^OK code=([0-9]+) data=([0-9]+)$/OK \1 \2/;
+                  s/^ERROR ([0-9]+) at pos ([0-9]+)$/ERR \1 \2/'
+}
+
+pass=0
+fail=0
+while IFS=$'\t' read -r prog v a b; do
+    case $prog in '' | '#'*) continue ;; esac
+    stage "$prog"
+    src="$STAGE/$prog"
+    actual=$(verdict_for "$src")
+    want=$(printf '%s %s %s' "$v" "$a" "$b" | tr -s ' ')
+    if [ "$actual" = "$want" ]; then
+        printf '  %-14s PASS  %s\n' "$prog" "$actual"
+        pass=$((pass + 1))
+    else
+        printf '  %-14s FAIL\n      expected: %s\n      actual:   %s\n' \
+               "$prog" "$want" "$actual"
+        fail=$((fail + 1))
+    fi
+done < "$TSV"
+
+echo "----------------------------------------------------------------"
+echo "CALC ladder: $pass passed, $fail failed (of $((pass + fail)))"
+[ "$fail" -eq 0 ] || exit 1

shell/tests/fixtures/calc_ladder.tsv

@@ -0,0 +1,32 @@
+# The CALC ladder — the ratchet for the work in CALC-COMPILE.md.
+#
+# One row per program, in the same shape run_compile_tests.sh asserts against
+# tests/fixtures/expected.tsv:
+#
+#   <program>	OK	<code>	<data>
+#   <program>	ERR	<tp3-error-no>	<normalised-pos>
+#
+# tests/calc_ladder.sh stages a COPY of the named program — and of the
+# companions that must sit beside it — into tmp/calc_ladder/ and runs the
+# freshly built compiler on it.  A copy, not the Resources file itself: the
+# include (Stage 2 of CALC-COMPILE.md) resolves relative to the including file,
+# so CALC.INC has to be beside CALC.PAS in the staged tree.
+#
+# The rung may only move FORWARD.  Each stage that removes a blocker replaces
+# the row in the SAME commit as the fix and says why in the message.  Never
+# edit a row to make a failing suite pass; that is the one thing this file
+# exists to prevent.
+#
+# pos is NOT a file offset.  TextBuf.LoadFile collapses CRLF and a lone LF to a
+# single CR, so pos indexes the NORMALISED buffer: CALC.PAS is 11,241 bytes on
+# disk and 11,059 normalised, so the reported 7791 is file byte 7901.  Every
+# position written here is a normalised one.  Once {$I CALC.INC} lands the
+# buffer is CALC.PAS with the directive replaced by CALC.INC, so all downstream
+# positions shift — which is exactly why this must be a measured number, not a
+# remembered one.
+#
+# Current rung (2026-10-19): FXMax : Char = 'G' — a typed constant, so
+# DefConst demands '=' through ExpectDelim and reports ENoSemi.  The rungs
+# behind it, in CALC-COMPILE.md §1: 8036 (enumeration, ERR 41), 8060 (set of
+# Attributes, ERR 102), 8237 (record, ERR 102).
+CALC.PAS	ERR	1	7791

shell/tests/run_all.sh

@@ -169,6 +169,12 @@ run "layout helper"   python3 tests/check_comimage.py
 run "mod=11 table"     python3 tests/probe/modrm11.py
 run "runtime image"   python3 tests/check_runtime.py
 run "compile matrix"  tests/run_compile_tests.sh
+# The compile matrix proves the compiler agrees with tests/fixtures.  This one
+# proves it gets a stated distance into the REAL program the CALC work targets
+# (CALC-COMPILE.md §1): one row of tests/fixtures/calc_ladder.tsv, moved
+# forward by each stage.  It reuses the matrix's build, so it is not a second
+# harness with a second stale-compiler bug.
+run "CALC ladder"     tests/calc_ladder.sh
 run "COM linker"      tests/run_com_tests.sh
 # The byte checks above can only ask "are these bytes well formed".  This one
 # asks "does the program do the right thing": every fixture with a .out is

shell/tests/run_compile_tests.sh

@@ -78,6 +78,16 @@ if [ $p2 -ne 0 ]; then
 fi
 echo "compiletest built (p1_rc=$p1, phase 1 rc=1 is the expected rollup)"
 
+# --build-only stops right after the build.  The CALC ladder
+# (tests/calc_ladder.sh) runs this same compiler on a staged copy of CALC.PAS,
+# and it must not be the second recipe that decides whether compiletest is
+# current: that is exactly how the stale-harness bug above happened, and two
+# staleness tests that drift are one test wearing two hats.  So the ladder
+# asks for the build here and does its own running afterwards.
+if [ "${1:-}" = "--build-only" ]; then
+    exit 0
+fi
+
 FX=${1:-tests/fixtures}
 
 # ---- assert against tests/fixtures/expected.tsv ---------------------------