Browse Source

Execute the nine fixtures that only compiled: four more bugs

Nine fixtures compiled and were never run. All nine contained no write or
writeln at all -- they assigned to a variable and fell off the end -- so an
empty .out could only have asserted "did not crash". t12_repeat could have
been compiling repeat..until as a single-pass loop for the whole life of the
project with nothing able to say so. t11_for's body was `i := i`.

All nine now print, each .out derived by hand from the Pascal rather than
blessed from the machine, and run_com_exec.py is 30/30. Four bugs appeared,
each shipping with a green compile matrix, a green .COM layout check, a green
golden and a green emitter audit:

28. `a * b` emitted ADD AX,CX. ParseAdd numbered `+` as 1 and ParseMul
    numbered `*` as 1, and BinOpEmit cannot see which level called it, so every
    multiplication dispatched to the addition: 7 * 6 printed 13. Only the
    constant-folding arm was right, and the single fixture that ever multiplied
    was multiplying two constants. OpAdd/OpSub/OpMul are now named constants.
29. `>` and `>=` had their SETcc opcodes swapped -- 9Dh (SETGE) for `>`, 9Fh
    (SETG) for `>=`. Only the equality boundary could see it; 6>5, 4>5, 5<5
    and -1>-2 were all already correct. Each CASE arm now carries its mnemonic
    beside the hex, because `op` is a number and an opcode alone does not say
    which comparison it answers.
30. REPEAT..UNTIL used JNZ where the body should be re-entered only when the
    condition is FALSE. That is `while`, so the body ran once and stopped;
    i:=0; repeat i:=i+1 until i>5 printed 1 instead of 6. TPSRC8 ~244-300 has
    IF/WHILE/REPEAT all calling excond then a fixed JZ, so one branch shape
    serves all three once the boolean is materialised negated.
31. Sibling procedures shared one parameter namespace: a finished procedure's
    symbols stayed at a level Search's `level <= lexnest` test accepted, so a
    second `a : integer` was error 41 and an unqualified `a` in procedure two
    silently read procedure one's argument. HideLocals relabels a procedure's
    own symbols to 0FFFFH on close -- relabelled, not popped, because
    symtab[old].resvar holds an index into the table being hidden.

expected.tsv is re-baselined upward for the nine, with the reason written into
the file: they grew because they now print. This is not the re-baseline that
turns a red suite green -- each row moved up, none down, and the behavioural
side of every one of them is the new .out.

Non-vacuity: five new cases in nonvacuity.sh mutate each bug back and require
run_com_exec.py to go red on the exact fixture that pins the behaviour (39 ok /
0 failed). Two harness lessons are recorded there because they bit during this
work: a python mutation helper that dies takes `if mutate_foo; then` with it,
so the case is SKIPPED and a skipped case is indistinguishable from a passing
one in the total (the first run of the SETcc case proved exactly that, via an
apostrophe in an assert message); and `assert s != before` is not enough when
there are two edits or two identical call sites, because one edit landing or
the wrong HideLocals going would still leave a changed file and a green run.

Also recorded, not fixed: a call caps at 16 arguments and reports error 99
"compiler overflow"; `,` and `;` are each required in the wrong place for
parameter vs variable lists; a parameter may not shadow a global; and EmJcc /
EmSetcc emit 386-only opcodes (0F 8x, 0F 9x) on an 8086 target, which no test
can see because qemu-i386 defaults to a post-386 CPU. That last one is now
next step 1, with `-cpu 8086` as the one-line thing worth trying first.

run_all.sh 12/12 ALL PASS, nonvacuity 39/39, execution 30/30, compile matrix
33/33, .COM linker 30/30, runtime entries 36/36.
Eric Streit 1 week ago
parent
commit
30f9fae

+ 2 - 2
README.md

@@ -18,11 +18,11 @@ second copy of a status table is a claim about the world that expires silently
 and is never noticed expiring, so there is one now, and this file says where it
 and is never noticed expiring, so there is one now, and this file says where it
 is.
 is.
 
 
-In short, current as of `v-TP3-BP-CONTRACT`:
+In short, current as of `v-TP3-DEAD-FIXTURES`:
 
 
 - The shell and the WordStar-style editor are done.
 - The shell and the WordStar-style editor are done.
 - The compiler front end and 8086 code generator are done for the language
 - The compiler front end and 8086 code generator are done for the language
-  subset the fixtures cover, and the emitted `.COM` images **run**: 21 fixtures
+  subset the fixtures cover, and the emitted `.COM` images **run**: 30 fixtures
   boot in `qemu-system-i386` and print exactly their expected bytes, and 35
   boot in `qemu-system-i386` and print exactly their expected bytes, and 35
   further cases call the runtime's own entries directly.
   further cases call the runtime's own entries directly.
 - **The shell's `R` key is still a stub.** There is no in-process 8086
 - **The shell's `R` key is still a stub.** There is no in-process 8086

+ 142 - 28
SUMMARY.md

@@ -21,6 +21,7 @@ manual, not guessed.
 | Linker: real DOS `.COM` writer + independent byte checker | `v-TP3-COM-IMAGE` | done, executed |
 | Linker: real DOS `.COM` writer + independent byte checker | `v-TP3-COM-IMAGE` | done, executed |
 | Measured encodings: ModR/M table, runtime audit, golden disassembly, `[BP+off]` | `v-TP3-MEASURED-EMITTERS` | done, executed |
 | Measured encodings: ModR/M table, runtime audit, golden disassembly, `[BP+off]` | `v-TP3-MEASURED-EMITTERS` | done, executed |
 | Execution: a boot sector, qemu, and a claim about behaviour | `v-TP3-EXECUTION` | done, **21/21 fixtures run, exact output** |
 | Execution: a boot sector, qemu, and a claim about behaviour | `v-TP3-EXECUTION` | done, **21/21 fixtures run, exact output** |
+| Execute the nine fixtures that only compiled | `v-TP3-DEAD-FIXTURES` | done, **30/30 run; four operator/scoping bugs found** |
 | Runtime entries under qemu, and the register contract stated | `v-TP3-BP-CONTRACT` | done, **36/36 entry checks; `wrchar`/`wrbool` no longer destroy BP** |
 | Runtime entries under qemu, and the register contract stated | `v-TP3-BP-CONTRACT` | done, **36/36 entry checks; `wrchar`/`wrbool` no longer destroy BP** |
 | `CmdRun` (the `R` key), in-process 8086 interpreter | — | **not started** |
 | `CmdRun` (the `R` key), in-process 8086 interpreter | — | **not started** |
 
 
@@ -71,7 +72,7 @@ verdict plus a source excerpt with a caret at `errPos`. No pty, instant.
 ```
 ```
 cd shell && tests/run_compile_tests.sh            # all fixtures
 cd shell && tests/run_compile_tests.sh            # all fixtures
 cd shell && tests/run_compile_tests.sh /some/dir  # another fixture set
 cd shell && tests/run_compile_tests.sh /some/dir  # another fixture set
-printf '@dump\ntests/fixtures/t19_int1.pas\n' | ./compiletest   # hex-dump the image
+printf '@dump \ntests/fixtures/t19_int1.pas\n' | ./compiletest  # hex-dump (trailing space! see TP3-COMPILER.md)
 ```
 ```
 
 
 **The matrix asserts; it does not just count.** `tests/fixtures/expected.tsv`
 **The matrix asserts; it does not just count.** `tests/fixtures/expected.tsv`
@@ -209,12 +210,12 @@ by building it and requiring the check to stay green.
 ### Execution under qemu — `tests/run_com_exec.py`
 ### Execution under qemu — `tests/run_com_exec.py`
 
 
 The sixth check is the one that cannot be written as a byte comparison, so it
 The sixth check is the one that cannot be written as a byte comparison, so it
-is also the one that finds the most: 21 fixtures are compiled to `.COM`, put on
+is also the one that finds the most: 30 fixtures are compiled to `.COM`, put on
 a floppy, booted, and their serial output compared to a committed `.out` file
 a floppy, booted, and their serial output compared to a committed `.out` file
 **exactly** — CRLF included — plus the exit code passed to `INT 21h AH=4Ch`.
 **exactly** — CRLF included — plus the exit code passed to `INT 21h AH=4Ch`.
 
 
 ```
 ```
-execution: 21 passed, 0 failed (of 21)
+execution: 30 passed, 0 failed (of 30)
 ```
 ```
 
 
 The two fixtures it found nothing in are the interesting ones: `t31_procparam`
 The two fixtures it found nothing in are the interesting ones: `t31_procparam`
@@ -278,7 +279,8 @@ checker.
 **It is now closed, and the milestone is `v-TP3-EXECUTION`.** 21 fixtures
 **It is now closed, and the milestone is `v-TP3-EXECUTION`.** 21 fixtures
 compile to `.COM` images, are booted on a floppy by a 512-byte hand-assembled
 compile to `.COM` images, are booted on a floppy by a 512-byte hand-assembled
 boot sector, and are compared **byte for byte** against the exact output the
 boot sector, and are compared **byte for byte** against the exact output the
-fixture demands — `tests/run_com_exec.py`, wired into `run_all.sh`, 21/21.
+fixture demands — `tests/run_com_exec.py`, wired into `run_all.sh`, 21/21 at
+that tag and 30/30 now.
 
 
 Everything below is about establishing what *can* be believed, because the
 Everything below is about establishing what *can* be believed, because the
 first attempt at this used an emulator that was wrong, and a wrong oracle is
 first attempt at this used an emulator that was wrong, and a wrong oracle is
@@ -750,22 +752,58 @@ Details that are deliberate, not incidental:
 ## Honest limitations
 ## Honest limitations
 
 
 - **`CmdRun` — the `R` key — is still a stub.** The compiler's *output* now
 - **`CmdRun` — the `R` key — is still a stub.** The compiler's *output* now
-  executes (21 fixtures, exact output, exit codes), but the shell cannot run a
+  executes (30 fixtures, exact output, exit codes), but the shell cannot run a
   `.COM` in place. The linker writes a real `.COM` and the boot sector runs one
   `.COM` in place. The linker writes a real `.COM` and the boot sector runs one
   under qemu; nothing in the host program yet interprets 8086 code. So the
   under qemu; nothing in the host program yet interprets 8086 code. So the
   user's route to seeing output is "compile, then run under qemu", not "press
   user's route to seeing output is "compile, then run under qemu", not "press
   `R`". TP3's `R` runs in the same 64 KB with no DOS loader, which is why this
   `R`". TP3's `R` runs in the same 64 KB with no DOS loader, which is why this
   is an interpreter and not a `system()` call.
   is an interpreter and not a `system()` call.
-- **21 of 33 fixtures execute.** 3 do not compile (`t14` and `t25` by design
-  as `ENoLib`, `uierror` deliberately), leaving **9 that compile and are never
-  run**: `t08` const · `t09` if/then/else · `t10` while · `t11` for/to ·
-  `t12` repeat/until · `t13` procedure + value param · `t15` label + goto ·
-  `t27` five locals · `t28` the 70-parameter declaration. Those are not
-  incidental omissions — they are the *control-flow* fixtures, and `t27`'s
-  five locals are precisely the `[BP+off]` paths this project got wrong twice.
-  They have byte-level checks and no behavioural check at all. Writing nine
-  `.out` files is cheap and is the highest-value step after `rt_exec.py`; the
-  byte counts in `expected.tsv` will then have a behavioural counterpart.
+- **Every fixture that compiles is now also run.** 30 of 33 execute; the 3 that
+  do not are `t14` and `t25` (`ENoLib`, by design) and `uierror` (a deliberate
+  syntax error). The gap this replaces was nine fixtures that compiled and were
+  *never executed* — `t08` const, `t09` if/then/else, `t10` while, `t11` for/to,
+  `t12` repeat/until, `t13` procedure + value param, `t15` label + goto,
+  `t27` five locals, `t28` the 70-parameter declaration — and all nine
+  **contained no `write`/`writeln` at all**. They assigned to a variable and
+  fell off the end, so the only thing an empty `.out` could have asserted was
+  "did not crash". That is why `t12`'s loop ran exactly once for the whole life
+  of the project with nothing to see it. The nine now print, each `.out` is
+  derived by hand from the Pascal rather than from the machine, and
+  `nonvacuity.sh` mutates each of the four bugs back in and requires the
+  execution check to go red. Finding them cost four fixes; see the milestone
+  section.
+- **A call takes at most 16 arguments, and says "compiler overflow" if you
+  exceed it.** `args` is `ARRAY [0..15] OF ERes` in the three call parsers, and
+  the guard raises `ECompOvf` = 99. So `far (1, 2, ... , 70)` is rejected with
+  error 99, whose text in TP3 means the compiler's own table overflowed — an
+  error about the compiler, for a program that merely has a long argument
+  list. This is why `t28_farparam` can *declare* 70 parameters (which is what
+  puts `p63` at `[BP+128]` and exercises the disp16 encoding) but can only
+  *pass* 16, and why its body reads `p63`/`p70` into a variable whose value is
+  deliberately absent from the `.out`: those two slots hold stack garbage, and a
+  fixture that printed them would be testing the harness, not the compiler.
+- **Parameters are separated by `,` and only by `,`.** `procedure two (a : integer ; b : integer)`
+  is error 1 at the semicolon; `procedure two (a : integer, b : integer)`
+  compiles. The reverse holds for variable declarations — `var a, b : integer`
+  is error 1 at the comma and needs two `var` lines. Inconsistent, and neither
+  spelling is wrong Pascal, so a program that compiles under one compiler may
+  not under another.
+- **A parameter may not shadow a global.** `DefProc` calls `DupTest` on every
+  parameter name, and `DupTest` reports error 41 for any name `Search` finds —
+  including one declared at an outer level. `procedure bump (x : integer)` with
+  a global `x` is rejected. Pascal allows the shadow and the inner one wins.
+  `HideLocals` fixed siblings colliding with each other; this is the
+  parent-vs-child direction of the same question and is untouched.
+- **The branch and compare lowering is 386 code on an 8086 target.** `EmJcc`
+  emits `0F 8x rel16` and `EmSetcc` emits `0F 9x` (SETcc). Neither exists on
+  an 8086. TP3 uses `JZ rel8` over a 3-byte `EJMP`, with `excond` materialising
+  the *negated* boolean (`TPSRC8` ~244-300). Every relational operator and every
+  conditional branch in the compiler is affected, so this is not one call site
+  but the whole idiom. **No test can see it**: qemu-i386 defaults to a
+  post-386 CPU, so every image in `run_com_exec.py` runs correctly on hardware
+  that did not exist when TP3 shipped. Catching it needs `-cpu 8086` (untried) or
+  a rewrite to the `excond` idiom plus a short/near branch policy — a milestone
+  with a wide golden blast radius, deliberately not folded into this one.
 - **The runtime's entries are now each called directly, and two of its
 - **The runtime's entries are now each called directly, and two of its
   invariants are stated rather than implied.** 436 bytes, 14 entries, 100
   invariants are stated rather than implied.** 436 bytes, 14 entries, 100
   emitter helpers decoded against their own names across both modules, the
   emitter helpers decoded against their own names across both modules, the
@@ -773,15 +811,16 @@ Details that are deliberate, not incidental:
   boundaries — and 35 cases that call the entries one at a time under qemu
   boundaries — and 35 cases that call the entries one at a time under qemu
   (`rt_exec.py`, 36/36). What is *not* covered is what a direct call cannot
   (`rt_exec.py`, 36/36). What is *not* covered is what a direct call cannot
   see: `wrtinl` is checked, but only from the harness's side of the calling
   see: `wrtinl` is checked, but only from the harness's side of the calling
-  contract, and the nine fixtures that are compiled but never executed
-  (above) are still reached only through the 21 that are.
+  contract. (The nine fixtures that compiled but were never executed used to sit
+  here as an extra gap; they are now run, so every fixture that reaches the
+  runtime at all also reaches it *through generated code*.)
 - **The pushback slot's address is a moving target.** It sits at
 - **The pushback slot's address is a moving target.** It sits at
   `rtSz + LoadBias + dataAt + D_PUSH`, so every runtime growth moves it, and
   `rtSz + LoadBias + dataAt + D_PUSH`, so every runtime growth moves it, and
   every address derived from it must be recomputed. It is computed, not
   every address derived from it must be recomputed. It is computed, not
   hard-coded — but the two `RT_SZ` constants that *were* hard-coded and had
   hard-coded — but the two `RT_SZ` constants that *were* hard-coded and had
   drifted (see the execution section) are the precedent for why this one gets
   drifted (see the execution section) are the precedent for why this one gets
   stated every time.
   stated every time.
-- **21 fixtures is a small sample of Pascal.** They cover `var`, `const`,
+- **30 fixtures is a small sample of Pascal.** They cover `var`, `const`,
   `if`, `while`, `for`, `repeat`, `case` over scalars, procedures with value
   `if`, `while`, `for`, `repeat`, `case` over scalars, procedures with value
   parameters, `goto`/`label`, string literals and `readln`. They do **not**
   parameters, `goto`/`label`, string literals and `readln`. They do **not**
   cover nested procedures, recursion, `var` parameters, `with`, records, sets,
   cover nested procedures, recursion, `var` parameters, `with`, records, sets,
@@ -1058,9 +1097,69 @@ Eleven more, and the two worst in the project are here.
     (ADDRESS) vs `LdBxVx` (CONTENTS) vs `StVxBx`; `MovAlBl` (`8A C3`, register)
     (ADDRESS) vs `LdBxVx` (CONTENTS) vs `StVxBx`; `MovAlBl` (`8A C3`, register)
     vs `LdAlBx` (`8A 07`, memory); `MovAh0` → `{0xB4}` vs `MovAl0` → `{0xB0}`.
     vs `LdAlBx` (`8A 07`, memory); `MovAh0` → `{0xB4}` vs `MovAl0` → `{0xB0}`.
 
 
+### Then nine fixtures were made to print, and four more appeared
+
+Twenty-seven bugs, and every one of them had been found by looking at bytes,
+decoding them, or reading them. The remaining fixtures — nine of them, the
+control-flow ones — had never been executed, because each one **assigned to a
+variable and fell off the end**: no `write`, no `writeln`, nothing to observe.
+An empty `.out` would have asserted only "did not crash", which is a property of
+the runtime, not of the operator under test. `t12_repeat` could have been
+compiling `repeat…until` as a single-pass loop for the entire life of the
+project and no check would have said.
+
+Rewriting the nine to print, with each `.out` **derived by hand from the Pascal
+and not blessed from the machine**, found four more. All four shipped with a
+green compile matrix, a green `.COM` layout check, a green golden and a green
+emitter audit.
+
+28. **`a * b` emitted `ADD AX,CX`.** `ParseAdd` numbered `+` as `1` and
+    `ParseMul` numbered `*` as `1` as well, and both pass the bare number to
+    `BinOpEmit`, which cannot see which precedence level called it. So every
+    multiplication dispatched to the addition: `a * 2` became `a + 2`, and
+    `7 * 6` printed `13`. The constant-folding arm of `BinOpEmit` was correct,
+    which is the only reason anything looked right — `t08_const`'s `n * n` has
+    two constants, and the one fixture that ever multiplied was multiplying two
+    constants. Now `OpAdd`/`OpSub`/`OpMul` are named constants, so two levels
+    cannot collide on a bare `1`, and `t08` was given a variable multiply.
+
+29. **`>` and `>=` had their SETcc opcodes swapped.** Op 4 (`>`) emitted `9Dh`
+    = SETGE and op 5 (`>=`) emitted `9Fh` = SETG, so `a > b` meant `a >= b` and
+    `a >= b` meant `a > b`. Only the equality boundary could see it: `6>5`,
+    `4>5`, `5<5`, `4<=5` and `-1>-2` were all already correct, and `5 > 5` and
+    `5 >= 5` were both wrong. One letter apart in the mnemonic, and the CASE arm
+    gave no hint which comparison it answered. Every arm now carries its
+    mnemonic beside the hex.
+
+30. **`REPEAT…UNTIL` looped back while the condition was TRUE** — `JNZ` where
+    the body should be re-entered only when the condition is FALSE. That is
+    `while`, so the body ran once, the condition was tested, and it stopped.
+    `i := 0; repeat i := i + 1 until i > 5; writeln (i)` printed **1**; the
+    hand-derived answer is 6. TP3's own sequence (`TPSRC8` ~244-300) calls
+    `excond`, which materialises the *negated* boolean, then a fixed `brnchop`
+    of `JZ`, so a single branch shape serves IF, WHILE and REPEAT alike.
+
+31. **Sibling procedures shared one parameter namespace.** A finished
+    procedure's symbols were left at a `level` that `Search`'s `level <= lexnest`
+    test still accepted, and every procedure body compiles at the same depth.
+    So a second `a : integer` was a duplicate (error 41) and an unqualified `a`
+    inside procedure two silently read procedure one's argument — passing 3
+    into `one` and then computing `x := a + 1` in `two` printed the wrong
+    number with no diagnostic at all. `HideLocals` relabels a procedure's own
+    symbols to `0FFFFH` when it closes, which fails the visibility test at every
+    depth a later procedure can be at. Relabelled rather than popped, because
+    `symtab[old].resvar` holds an index and a function's result variable is one
+    of the entries being hidden.
+
+The theme is worth stating because it is the same theme as bug 27: **a name that
+does not distinguish two things makes the next mistake invisible.** `*` and `+`
+were both `1`; `>` and `>=` were two hex bytes; two procedures' `a` was one
+symbol. In each case the fix is to make the distinction part of the name or the
+surrounding text, not to fix the value and leave the ambiguity in place.
+
 ## The bug family, stated once
 ## The bug family, stated once
 
 
-Nine of the twenty-seven are the *same* bug in different clothes: **loading the
+Nine of the thirty-one are the *same* bug in different clothes: **loading the
 address where the value was wanted, or picking the register one byte or one
 address where the value was wanted, or picking the register one byte or one
 letter away from the right one.** `EmPushVarAddr` had the right bytes for the
 letter away from the right one.** `EmPushVarAddr` had the right bytes for the
 wrong register. `LdAlBx` and `MovAlBl` are one letter apart. `MovAh0` and
 wrong register. `LdAlBx` and `MovAlBl` are one letter apart. `MovAh0` and
@@ -1137,28 +1236,43 @@ independently-scanned inventory at all.
 
 
 ## Next steps
 ## Next steps
 
 
-1. **`CmdRun`** as an in-process 8086 interpreter — the `R` menu key, and a
+1. **8086 branch and compare lowering.** `EmJcc` emits `0F 8x rel16` and
+   `EmSetcc` emits `0F 9x`; neither instruction exists on an 8086. Every
+   relational operator and every conditional branch depends on them. TP3's
+   idiom is `excond` materialising the negated boolean plus `JZ rel8` over a
+   3-byte `EJMP` (`TPSRC8` ~244-300), so this needs a boolean-materialisation
+   strategy and a short/near branch policy, not two opcode substitutions — and
+   it will move nearly every byte in the golden. **First, try
+   `qemu-system-i386 -cpu 8086`** on the existing 30 images: if that turns this
+   class red, it is a one-line addition to `rt_exec.py` and it makes every
+   later fix provable instead of argued.
+2. **`CmdRun`** as an in-process 8086 interpreter — the `R` menu key, and a
    fallback executor for environments with no DOS. Validate it against qemu on
    fallback executor for environments with no DOS. Validate it against qemu on
    the *same images*, so the two oracles check each other. Cross-validation is
    the *same images*, so the two oracles check each other. Cross-validation is
-   the point: an interpreter that agrees with qemu on 21 fixtures is far more
+   the point: an interpreter that agrees with qemu on 30 fixtures is far more
    evidence than either alone.
    evidence than either alone.
-2. **String *variables*** — `s : string`, `s := 'hi'`, `writeln(s)`. The
+3. **String *variables*** — `s : string`, `s := 'hi'`, `writeln(s)`. The
    encoding blocker is gone (`EmBpDisp`); what is left is a length word, an
    encoding blocker is gone (`EmBpDisp`); what is left is a length word, an
    assignment path, and a `WrStr` entry (TPSRC4 `xwrtstr`). `IoCall` currently
    assignment path, and a `WrStr` entry (TPSRC4 `xwrtstr`). `IoCall` currently
    refuses with `ENoLib`.
    refuses with `ENoLib`.
-3. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from
+4. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from
    RESUME-TP3.md §3.11), range/index checks (`TU_RANGE_CHECK`,
    RESUME-TP3.md §3.11), range/index checks (`TU_RANGE_CHECK`,
    `TU_INDEX_CHECK`), typed constants (RESUME-TP3.md §3.14), `array` at its
    `TU_INDEX_CHECK`), typed constants (RESUME-TP3.md §3.14), `array` at its
    point of use (`t14`), `case` with subrange labels.
    point of use (`t14`), `case` with subrange labels.
-4. **`readln` of a `BYTE`** calls `rdint`, which stores 2 bytes and overflows
+5. **`readln` of a `BYTE`** calls `rdint`, which stores 2 bytes and overflows
    into the next variable. TP3 has a separate `xrdbyte`; a `TU_RdByte` entry is
    into the next variable. TP3 has a separate `xrdbyte`; a `TU_RdByte` entry is
    the fix. No fixture exists yet, which is why it has not been done — write
    the fix. No fixture exists yet, which is why it has not been done — write
    the fixture first, so the bug is red before the fix.
    the fixture first, so the bug is red before the fix.
-5. Make the 4 KiB code window an enforced limit rather than a documented one:
+6. Make the 4 KiB code window an enforced limit rather than a documented one:
    report an error when `pc` reaches `dc`, instead of writing over the data.
    report an error when `pc` reaches `dc`, instead of writing over the data.
-6. Comma-separated names: `var i, c : integer;`.
-7. Harden the program-header parameter loop against non-advancing input
+7. Both spellings of a multi-name declaration. `var i, c : integer;` is
+   error 1 at the comma and needs two `var` lines; `procedure f (a : integer;
+   b : integer)` is error 1 at the semicolon and needs a comma. Neither is
+   wrong Pascal, so a program that compiles under one compiler may not under
+   another. A parameter may also not shadow a global (`DupTest` rejects any
+   name `Search` finds at any level), which Pascal allows.
+8. Harden the program-header parameter loop against non-advancing input
    (`program p(1;)`) with a `BOOLEAN` flag — **not** `EXIT`, which ICEs gm2.
    (`program p(1;)`) with a `BOOLEAN` flag — **not** `EXIT`, which ICEs gm2.
-8. FreeDOS (`freedos.qcow2`, FD14-LiveCD) is still untried. Not needed for any
+9. FreeDOS (`freedos.qcow2`, FD14-LiveCD) is still untried. Not needed for any
    claim above, but it is the only way to get a *real* DOS as a third opinion
    claim above, but it is the only way to get a *real* DOS as a third opinion
    on the `INT 21h` shim.
    on the `INT 21h` shim.

+ 14 - 4
TP3-COMPILER.md

@@ -137,7 +137,7 @@ byte is a perfectly well-formed instruction that does something else, so no
 amount of inspecting the compiler's intent will catch it.  This harness closes
 amount of inspecting the compiler's intent will catch it.  This harness closes
 that gap the only way it can be closed -- by running the thing.
 that gap the only way it can be closed -- by running the thing.
 
 
-For each of 21 fixtures it compiles a `.COM`, writes a 1.44 MB floppy with the
+For each of 30 fixtures it compiles a `.COM`, writes a 1.44 MB floppy with the
 image in it, boots `qemu-system-i386 -fda disk.img -serial out.txt`, and
 image in it, boots `qemu-system-i386 -fda disk.img -serial out.txt`, and
 compares `out.txt` to the committed `tests/fixtures/tNN.out` **exactly** --
 compares `out.txt` to the committed `tests/fixtures/tNN.out` **exactly** --
 CRLF included -- plus the exit code the program hands to `INT 21h AH=4Ch`.
 CRLF included -- plus the exit code the program hands to `INT 21h AH=4Ch`.
@@ -172,7 +172,9 @@ partial read or a wrong sector count would have been visible, this was not.
 window at `C000h`; and the copy is executed code, so it is the last thing
 window at `C000h`; and the copy is executed code, so it is the last thing
 that touches the image and no BIOS call follows it.
 that touches the image and no BIOS call follows it.
 
 
-**21 of 33 fixtures execute.**  3 do not compile, and 9 compile without a
+**30 of 33 fixtures execute.**  3 do not compile.  (Until this milestone 9 more
+compiled without a `.out` and were never run at all; they have all been
+rewritten to print and are now covered.)
 `.out` file -- and those 9 are the control-flow fixtures (`t08` const, `t09`
 `.out` file -- and those 9 are the control-flow fixtures (`t08` const, `t09`
 if, `t10` while, `t11` for, `t12` repeat, `t13` procedure, `t15` label, `t27`
 if, `t10` while, `t11` for, `t12` repeat, `t13` procedure, `t15` label, `t27`
 five locals, `t28` 70 parameters).  They have byte-level checks and no
 five locals, `t28` 70 parameters).  They have byte-level checks and no
@@ -268,15 +270,23 @@ to be in the name.**
 
 
 ### Hex-dumping the emitted image
 ### Hex-dumping the emitted image
 
 
-A line reading `@dump` on stdin switches on a hex dump of the emitted 8086
+A line reading `@dump ` on stdin switches on a hex dump of the emitted 8086
 code (via the new `Compiler.CodeByteAt`), which is how the `rel16` off-by-2
 code (via the new `Compiler.CodeByteAt`), which is how the `rel16` off-by-2
 below was found -- sizes alone could never have shown it:
 below was found -- sizes alone could never have shown it:
 
 
 ```
 ```
 cd shell
 cd shell
-printf '@dump\ntests/fixtures/t19_int1.pas\n' | ./compiletest
+printf '@dump \ntests/fixtures/t19_int1.pas\n' | ./compiletest
 ```
 ```
 
 
+**The trailing space is required and the error message does not say so.**
+`CompileTest.IsCmd` requires the line to be exactly six characters — `@`, four
+letters, and a space — because the fifth slot is the delimiter it tests for.
+`@dump` is five, so it is not recognised as a command and falls through to the
+path branch, which reports `CANNOT OPEN` for the literal string `@dump`. That
+is the same shape as a compile error, so the mode looks like a missing file.
+`@image` is six characters with no delimiter and works without one.
+
 ```
 ```
 0000:  01 00 02 00 10 00 00 00 00 00 10 00 00 00 00 00
 0000:  01 00 02 00 10 00 00 00 00 00 10 00 00 00 00 00
 0010:  E8 F5 FF 8B EC B8 01 00 50 E8 04 00 83 C4 02 E8
 0010:  E8 F5 FF 8B EC B8 01 00 50 E8 04 00 83 C4 02 E8

+ 70 - 16
shell/Compiler.mod

@@ -114,6 +114,17 @@ CONST
    TkSet  = 36 ;  TkPacked  = 37 ; TkForward = 38 ; TkExternal = 39 ;
    TkSet  = 36 ;  TkPacked  = 37 ; TkForward = 38 ; TkExternal = 39 ;
    TkAbsolute = 40 ; TkOverlay = 41 ; TkString = 42 ;
    TkAbsolute = 40 ; TkOverlay = 41 ; TkString = 42 ;
 
 
+   (* Codes for the SYMBOL operators, as BinOpEmit numbers them.  The word
+      operators carry their own Tk* token and need no code here.
+      These are named, not bare numbers, because every precedence level's
+      parser picks its own code out of the same set and BinOpEmit cannot see
+      which level called it.  ParseAdd chose 1 for '+' and ParseMul chose 1
+      for '*', so every multiplication dispatched to EmAddAxCx and a * b
+      compiled to a + b.  Only the constant-folding path was right, which is
+      why n * n with n a CONST was correct and a * b with a a variable was
+      not. *)
+   OpAdd = 1 ;  OpSub = 2 ;  OpMul = 3 ;
+
    (* TP3 error numbers *)
    (* TP3 error numbers *)
    ENoSemi    = 1 ;  EPointExp  = 10 ;  ESimpType  = 30 ;
    ENoSemi    = 1 ;  EPointExp  = 10 ;  ESimpType  = 30 ;
    EUnknown   = 41 ; EConstRange = 45 ; EMemOvf    = 98 ;
    EUnknown   = 41 ; EConstRange = 45 ; EMemOvf    = 98 ;
@@ -974,6 +985,27 @@ BEGIN
    END
    END
 END DupTest ;
 END DupTest ;
 
 
+PROCEDURE HideLocals (from : CARDINAL) ;
+(* Make every symbol from index `from' up invisible to everything outside the
+   procedure that declared it.
+   Search accepts a symbol when its level is <= lexnest, and every procedure
+   body is compiled at the same depth (lexnest 1), so a finished procedure's
+   parameters stayed visible to the NEXT procedure: a second `a : integer'
+   was a duplicate (err 41) and an unqualified `a' inside procedure two read
+   procedure one's argument.  Level 0FFFFH fails `level <= lexnest' at every
+   depth a later procedure can be at, and by the time this runs the body that
+   could still legitimately see them is finished.
+   The symbols are relabelled, not popped: symtab[old].resvar holds an INDEX,
+   and a function's result variable is one of the entries being hidden. *)
+VAR p : CARDINAL ;
+BEGIN
+   p := from ;
+   WHILE p < symTop DO
+      symtab [p].level := 0FFFFH ;
+      INC (p)
+   END
+END HideLocals ;
+
 (* ---------------------------------------------------------------- *)
 (* ---------------------------------------------------------------- *)
 (*  lexer                                                           *)
 (*  lexer                                                           *)
 (* ---------------------------------------------------------------- *)
 (* ---------------------------------------------------------------- *)
@@ -1611,9 +1643,9 @@ BEGIN
    IF (left.kind = 0) AND (right.kind = 0) THEN
    IF (left.kind = 0) AND (right.kind = 0) THEN
       okc := FALSE ;
       okc := FALSE ;
       CASE op OF
       CASE op OF
-         1 : f := ConstAdd (left.imm, right.imm) ; okc := TRUE ;
-      |  2 : f := ConstSub (left.imm, right.imm) ; okc := TRUE ;
-      |  3 : f := ConstMul (left.imm, right.imm) ; okc := TRUE ;
+         OpAdd : f := ConstAdd (left.imm, right.imm) ; okc := TRUE ;
+      |  OpSub : f := ConstSub (left.imm, right.imm) ; okc := TRUE ;
+      |  OpMul : f := ConstMul (left.imm, right.imm) ; okc := TRUE ;
       |  TkDiv : okc := (right.imm # 0) AND (right.imm > 0)
       |  TkDiv : okc := (right.imm # 0) AND (right.imm > 0)
                         AND (left.imm >= 0) ;
                         AND (left.imm >= 0) ;
                  IF okc THEN f := left.imm DIV right.imm END ;
                  IF okc THEN f := left.imm DIV right.imm END ;
@@ -1637,9 +1669,9 @@ BEGIN
    LoadAtom (right) ; EmPopCx () ;
    LoadAtom (right) ; EmPopCx () ;
    EmXchgAxCx () ;
    EmXchgAxCx () ;
    CASE op OF
    CASE op OF
-      1   : EmAddAxCx ;
-   |  2   : EmSubAxCx ;
-   |  3   : EmMulAxCx ;
+      OpAdd : EmAddAxCx ;
+   |  OpSub : EmSubAxCx ;
+   |  OpMul : EmMulAxCx ;
    |  TkDiv : EmIDivAxCx ;
    |  TkDiv : EmIDivAxCx ;
    |  TkMod : EmIDivAxCx ; EmXchgAxDx ;
    |  TkMod : EmIDivAxCx ; EmXchgAxDx ;
    ELSE
    ELSE
@@ -1716,13 +1748,21 @@ BEGIN
          LoadAtom (right) ; EmPopCx () ;
          LoadAtom (right) ; EmPopCx () ;
          EmXchgAxCx () ;
          EmXchgAxCx () ;
          EmCmpAxCx () ;
          EmCmpAxCx () ;
+         (* The mnemonic is written next to every opcode on purpose.  `op' is
+            a number, so the arm for ">" and the arm for ">=" differed only
+            by two hex digits that are each one letter from the other
+            meaning -- SETG (9FH) and SETGE (9DH).  They were swapped, which
+            made a > b mean a >= b and a >= b mean a > b.  Only the equality
+            boundary could see it: 6>5, 5<5, -1>-2 and every other case I
+            tried were already right.  An opcode on its own does not say
+            which comparison it is the answer to. *)
          CASE op OF
          CASE op OF
-            1 : EmSetcc (94H) ;
-         |  2 : EmSetcc (95H) ;
-         |  3 : EmSetcc (9CH) ;
-         |  4 : EmSetcc (9DH) ;
-         |  5 : EmSetcc (9FH) ;
-         |  6 : EmSetcc (9EH)
+            1 : EmSetcc (94H) ;          (* =  SETE  *)
+         |  2 : EmSetcc (95H) ;          (* <> SETNE *)
+         |  3 : EmSetcc (9CH) ;          (* <  SETL  *)
+         |  4 : EmSetcc (9FH) ;          (* >  SETG  *)
+         |  5 : EmSetcc (9DH) ;          (* >= SETGE *)
+         |  6 : EmSetcc (9EH)            (* <= SETLE *)
          END ;
          END ;
          r.kind := 2 ;
          r.kind := 2 ;
          r.cls := TBool
          r.cls := TBool
@@ -1739,9 +1779,9 @@ BEGIN
       op := 0 ;
       op := 0 ;
       Skip () ;
       Skip () ;
       IF CurCh () = '+' THEN
       IF CurCh () = '+' THEN
-         op := 1 ; DropCh (GetCh ())
+         op := OpAdd ; DropCh (GetCh ())
       ELSIF CurCh () = '-' THEN
       ELSIF CurCh () = '-' THEN
-         op := 2 ; DropCh (GetCh ())
+         op := OpSub ; DropCh (GetCh ())
       ELSIF KwAhead ("OR") THEN
       ELSIF KwAhead ("OR") THEN
          GetWord () ;
          GetWord () ;
          op := TkOr
          op := TkOr
@@ -1763,7 +1803,7 @@ BEGIN
       op := 0 ;
       op := 0 ;
       Skip () ;
       Skip () ;
       IF CurCh () = '*' THEN
       IF CurCh () = '*' THEN
-         op := 1 ; DropCh (GetCh ())
+         op := OpMul ; DropCh (GetCh ())   (* OpAdd here meant a*b -> a+b *)
       ELSIF CurCh () = '/' THEN
       ELSIF CurCh () = '/' THEN
          op := 2 ; DropCh (GetCh ()) ; Err (ENoLib)
          op := 2 ; DropCh (GetCh ()) ; Err (ENoLib)
       ELSIF KwAhead ("DIV") THEN
       ELSIF KwAhead ("DIV") THEN
@@ -2305,7 +2345,16 @@ BEGIN
       ParseExpr (t) ;
       ParseExpr (t) ;
       LoadAtom (t) ;
       LoadAtom (t) ;
       EmCmpAxi (0) ;
       EmCmpAxi (0) ;
-      zj := EmJcc (85H, L1) ;         (* JNZ -> body again *)
+      (* UNTIL exits when the condition is TRUE, so the body runs again when
+         it is FALSE.  The condition is a 0/1 in AX and EmCmpAxi (0) has just
+         compared it with 0, so ZF=1 means "condition false" -- which is
+         exactly the case that loops, hence JZ.
+         This was JNZ, which compiled repeat-until as while-until: the body
+         ran once, the condition was tested, and it stopped.  t12_repeat is
+         `i:=0; repeat i:=i+1 until i>5' and it printed 1.
+         No patch slot: L1 is backwards and already known, so EmJcc returns
+         0 and there is nothing to SetPatTgt. *)
+      DropC (EmJcc (84H, L1)) ;        (* JZ -> body again *)
       DEC (brkN) ;
       DEC (brkN) ;
       i := brkSave [brkN] ;
       i := brkSave [brkN] ;
       WHILE i < exitCnt DO
       WHILE i < exitCnt DO
@@ -2830,6 +2879,7 @@ VAR nm : ARRAY [0..MaxName] OF CHAR ;
     isFunc : BOOLEAN ;
     isFunc : BOOLEAN ;
     saveNest, saveLoc, saveRes, saveF : CARDINAL ;
     saveNest, saveLoc, saveRes, saveF : CARDINAL ;
     saveLB, savePO : CARDINAL ;
     saveLB, savePO : CARDINAL ;
+    nestMark : CARDINAL ;             (* symTop just inside this procedure *)
     i : CARDINAL ;
     i : CARDINAL ;
 BEGIN
 BEGIN
    isFunc := curIsFunc ;
    isFunc := curIsFunc ;
@@ -2859,6 +2909,7 @@ BEGIN
    saveLB := locBytes ;
    saveLB := locBytes ;
    savePO := parmOff ;
    savePO := parmOff ;
    INC (lexnest) ;
    INC (lexnest) ;
+   nestMark := symTop ;          (* after the proc's own name, before its params *)
    locFree := 0FFFEH ;
    locFree := 0FFFEH ;
    locBytes := 0 ;
    locBytes := 0 ;
    parmOff := 4 ;
    parmOff := 4 ;
@@ -2926,6 +2977,8 @@ BEGIN
       symtab [old].fwd := TRUE ;
       symtab [old].fwd := TRUE ;
       symtab [old].defnd := (tok2 = TkExternal) ;
       symtab [old].defnd := (tok2 = TkExternal) ;
       IfMatchSemi () ;
       IfMatchSemi () ;
+      HideLocals (nestMark) ;      (* a FORWARD's parameters are not the
+                                      caller's to see either *)
       lexnest := saveNest ;
       lexnest := saveNest ;
       locFree := saveLoc ;
       locFree := saveLoc ;
       resultVar := saveRes ;
       resultVar := saveRes ;
@@ -2959,6 +3012,7 @@ BEGIN
       END ;
       END ;
       INC (i)
       INC (i)
    END ;
    END ;
+   HideLocals (nestMark) ;         (* parameters and locals stop here *)
    lexnest := saveNest ;
    lexnest := saveNest ;
    locFree := saveLoc ;
    locFree := saveLoc ;
    resultVar := saveRes ;
    resultVar := saveRes ;

+ 9 - 4
shell/tests/check_framedisp.py

@@ -60,9 +60,14 @@ t28_farparam.pas  seventy declared parameters, of which the call passes
                   are at +128 and +142, the first offsets the disp8 form
                   are at +128 and +142, the first offsets the disp8 form
                   cannot represent.
                   cannot represent.
 
 
-t28 is a compile-level fixture and is never executed: p63..p70 are declared
-but not passed, so at run time those reads would come from uninitialised
-stack.  What it establishes is the ENCODING, which is the thing that was
+t28 is executed as well as compiled, and that distinction matters for what its
+.out file may assert.  `g := p16 + p1' is deterministic (1 + 16 = 17) and is
+the printed value.  `unused := p63 + p70' reads stack garbage, because a call
+caps at 16 arguments and p17..p70 are never passed; it is computed and
+discarded, and its value is deliberately NOT in the .out file.  A fixture that
+printed it would be asserting a number that depends on what the caller left on
+the stack, which is a test of the harness rather than of the compiler.  What
+t28 establishes for p63/p70 is the ENCODING, which is the thing that was
 wrong.  Do not read t28 as a claim that a 70-argument call works.
 wrong.  Do not read t28 as a claim that a 70-argument call works.
 
 
 Usage: check_framedisp.py [-v]     (from shell/)
 Usage: check_framedisp.py [-v]     (from shell/)
@@ -109,7 +114,7 @@ CASES = [
     # (load).
     # (load).
     ("t27_localvar", "89", expected_local_offsets(5)),
     ("t27_localvar", "89", expected_local_offsets(5)),
     ("t27_localvar", "8B", expected_local_offsets(5)),
     ("t27_localvar", "8B", expected_local_offsets(5)),
-    # t28: `g := p63 + p70`.
+    # t28: `unused := p63 + p70` -- read but never printed, see the note above.
     ("t28_farparam", "8B", [expected_param_offsets(63),
     ("t28_farparam", "8B", [expected_param_offsets(63),
                             expected_param_offsets(70)]),
                             expected_param_offsets(70)]),
 ]
 ]

+ 17 - 9
shell/tests/fixtures/expected.tsv

@@ -20,6 +20,14 @@
 #   t25       a string literal used where a 16-bit word is wanted
 #   t25       a string literal used where a 16-bit word is wanted
 #   uierror   deliberate syntax error, pinned by the editor UI test
 #   uierror   deliberate syntax error, pinned by the editor UI test
 #
 #
+# The nine control-flow fixtures (t08 t09 t10 t11 t12 t13 t15 t27 t28) were
+# re-baselined upward when they were rewritten to PRINT their result, so each
+# has a .out file and is executed under qemu (run_com_exec.py) as well as
+# compiled here.  Before that they assigned to a variable and fell off the end:
+# they asserted that the compiler produced bytes, and nothing about behaviour.
+# t13 also gained a second procedure, which is why it is bigger than the sum of
+# its parts.  t08's data went 6 -> 7 because a CHAR variable joined the INTEGER.
+#
 # t02 t03 t05 t17 used to be ERR 102 as well: a multi-character literal was
 # t02 t03 t05 t17 used to be ERR 102 as well: a multi-character literal was
 # rejected because writeln had no string support.  They compile now, via the
 # rejected because writeln had no string support.  They compile now, via the
 # inline-string path (CALL wrtinl, then <length><chars> in the code stream -
 # inline-string path (CALL wrtinl, then <length><chars> in the code stream -
@@ -144,14 +152,14 @@ t04_var	OK	48	6
 t05_own_line_comment	OK	38	4
 t05_own_line_comment	OK	38	4
 t06_two_args	OK	52	4
 t06_two_args	OK	52	4
 t07_big	OK	102	6
 t07_big	OK	102	6
-t08_const	OK	35	6
-t09_if	OK	67	6
-t10_while	OK	75	6
-t11_for	OK	69	6
-t12_repeat	OK	72	6
-t13_proc	OK	56	6
+t08_const	OK	76	7
+t09_if	OK	189	6
+t10_while	OK	147	6
+t11_for	OK	142	6
+t12_repeat	OK	133	6
+t13_proc	OK	126	6
 t14_types	ERR	102	83
 t14_types	ERR	102	83
-t15_label	OK	38	6
+t15_label	OK	88	6
 t16_str1	OK	42	4
 t16_str1	OK	42	4
 t17_two_str	OK	45	4
 t17_two_str	OK	45	4
 t18_writeln_bare	OK	32	4
 t18_writeln_bare	OK	32	4
@@ -163,8 +171,8 @@ t23_str_empty	OK	36	4
 t24_str_quote	OK	41	4
 t24_str_quote	OK	41	4
 t25_str_as_value	ERR	102	48
 t25_str_as_value	ERR	102	48
 t26_str_mixed_args	OK	58	4
 t26_str_mixed_args	OK	58	4
-t27_localvar	OK	125	6
-t28_farparam	OK	126	6
+t27_localvar	OK	256	6
+t28_farparam	OK	153	8
 t29_readln	OK	82	7
 t29_readln	OK	82	7
 t30_forloop	OK	96	8
 t30_forloop	OK	96	8
 t31_procparam	OK	69	6
 t31_procparam	OK	69	6

+ 1 - 0
shell/tests/fixtures/t08_const.out

@@ -0,0 +1 @@
+49 A

+ 4 - 1
shell/tests/fixtures/t08_const.pas

@@ -4,6 +4,9 @@ const
   c = 'A' ;
   c = 'A' ;
 var
 var
   x : integer ;
   x : integer ;
+  y : char ;
 begin
 begin
-  x := n
+  x := n * n ;
+  y := c ;
+  writeln (x, ' ', y)
 end.
 end.

+ 3 - 0
shell/tests/fixtures/t09_if.out

@@ -0,0 +1,3 @@
+nonpos
+pos
+ge

+ 13 - 2
shell/tests/fixtures/t09_if.pas

@@ -2,8 +2,19 @@ program t09;
 var
 var
   x : integer ;
   x : integer ;
 begin
 begin
+  x := 0 ;
   if x > 0 then
   if x > 0 then
-    x := 1
+    writeln ('pos')
   else
   else
-    x := 2
+    writeln ('nonpos') ;
+  x := 5 ;
+  if x > 0 then
+    writeln ('pos')
+  else
+    writeln ('nonpos') ;
+  x := 5 ;
+  if x >= 5 then
+    writeln ('ge')
+  else
+    writeln ('lt')
 end.
 end.

+ 2 - 0
shell/tests/fixtures/t10_while.out

@@ -0,0 +1,2 @@
+10
+100

+ 6 - 1
shell/tests/fixtures/t10_while.pas

@@ -4,5 +4,10 @@ var
 begin
 begin
   x := 0 ;
   x := 0 ;
   while x < 10 do
   while x < 10 do
-    x := x + 1
+    x := x + 1 ;
+  writeln (x) ;
+  x := 100 ;
+  while x < 10 do
+    x := x + 1 ;
+  writeln (x)
 end.
 end.

+ 7 - 0
shell/tests/fixtures/t11_for.out

@@ -0,0 +1,7 @@
+1
+2
+3
+i=4
+3
+2
+1

+ 5 - 2
shell/tests/fixtures/t11_for.pas

@@ -2,6 +2,9 @@ program t11;
 var
 var
   i : integer ;
   i : integer ;
 begin
 begin
-  for i := 1 to 10 do
-    i := i
+  for i := 1 to 3 do
+    writeln (i) ;
+  writeln ('i=', i) ;
+  for i := 3 downto 1 do
+    writeln (i)
 end.
 end.

+ 2 - 0
shell/tests/fixtures/t12_repeat.out

@@ -0,0 +1,2 @@
+6
+99

+ 7 - 1
shell/tests/fixtures/t12_repeat.pas

@@ -5,5 +5,11 @@ begin
   i := 0 ;
   i := 0 ;
   repeat
   repeat
     i := i + 1
     i := i + 1
-  until i > 5
+  until i > 5 ;
+  writeln (i) ;
+  i := 9 ;
+  repeat
+    i := 99
+  until i > 5 ;
+  writeln (i)
 end.
 end.

+ 2 - 0
shell/tests/fixtures/t13_proc.out

@@ -0,0 +1,2 @@
+3
+12

+ 8 - 1
shell/tests/fixtures/t13_proc.pas

@@ -5,6 +5,13 @@ procedure show (a : integer) ;
 begin
 begin
   x := a
   x := a
 end ;
 end ;
+procedure add (a : integer, b : integer) ;
 begin
 begin
-  show (3)
+  x := x + a + b
+end ;
+begin
+  show (3) ;
+  writeln (x) ;
+  add (4, 5) ;
+  writeln (x)
 end.
 end.

+ 1 - 0
shell/tests/fixtures/t15_label.out

@@ -0,0 +1 @@
+5

+ 5 - 2
shell/tests/fixtures/t15_label.pas

@@ -4,6 +4,9 @@ label
 var
 var
   x : integer ;
   x : integer ;
 begin
 begin
-  goto 1 ;
-1: x := 1
+  x := 0 ;
+1: x := x + 1 ;
+  if x < 5 then
+    goto 1 ;
+  writeln (x)
 end.
 end.

+ 2 - 0
shell/tests/fixtures/t27_localvar.out

@@ -0,0 +1,2 @@
+10 20 30 40 50 150
+150

+ 4 - 2
shell/tests/fixtures/t27_localvar.pas

@@ -14,8 +14,10 @@ begin
   v3 := 30 ;
   v3 := 30 ;
   v4 := 40 ;
   v4 := 40 ;
   v5 := 50 ;
   v5 := 50 ;
-  g := v1 + v2 + v3 + v4 + v5
+  g := v1 + v2 + v3 + v4 + v5 ;
+  writeln (v1, ' ', v2, ' ', v3, ' ', v4, ' ', v5, ' ', g)
 end ;
 end ;
 begin
 begin
-  work
+  work ;
+  writeln (g)
 end.
 end.

+ 1 - 0
shell/tests/fixtures/t28_farparam.out

@@ -0,0 +1 @@
+17

+ 83 - 3
shell/tests/fixtures/t28_farparam.pas

@@ -1,10 +1,90 @@
 program t28;
 program t28;
 var
 var
   g : integer ;
   g : integer ;
-procedure far (p1 : integer, p2 : integer, p3 : integer, p4 : integer, p5 : integer, p6 : integer, p7 : integer, p8 : integer, p9 : integer, p10 : integer, p11 : integer, p12 : integer, p13 : integer, p14 : integer, p15 : integer, p16 : integer, p17 : integer, p18 : integer, p19 : integer, p20 : integer, p21 : integer, p22 : integer, p23 : integer, p24 : integer, p25 : integer, p26 : integer, p27 : integer, p28 : integer, p29 : integer, p30 : integer, p31 : integer, p32 : integer, p33 : integer, p34 : integer, p35 : integer, p36 : integer, p37 : integer, p38 : integer, p39 : integer, p40 : integer, p41 : integer, p42 : integer, p43 : integer, p44 : integer, p45 : integer, p46 : integer, p47 : integer, p48 : integer, p49 : integer, p50 : integer, p51 : integer, p52 : integer, p53 : integer, p54 : integer, p55 : integer, p56 : integer, p57 : integer, p58 : integer, p59 : integer, p60 : integer, p61 : integer, p62 : integer, p63 : integer, p64 : integer, p65 : integer, p66 : integer, p67 : integer, p68 : integer, p69 : integer, p70 : integer) ;
+  unused : integer ;
+(* Seventy parameters.  p63 and p70 sit at +128 and +142, the first offsets
+   an 8-bit displacement cannot represent, so their accesses must use the
+   4-byte mod=10 form -- that is what check_framedisp.py asserts.
+   A call caps at 16 arguments, so p17..p70 are never passed and reading
+   them returns stack garbage.  `unused' takes that sum and is never
+   printed: the point of the fixture is the ENCODING, and the printed
+   value must stay deterministic.  Do not read this as a claim that a
+   70-argument call works. *)
+procedure far (p1 : integer,
+  p2 : integer,
+  p3 : integer,
+  p4 : integer,
+  p5 : integer,
+  p6 : integer,
+  p7 : integer,
+  p8 : integer,
+  p9 : integer,
+  p10 : integer,
+  p11 : integer,
+  p12 : integer,
+  p13 : integer,
+  p14 : integer,
+  p15 : integer,
+  p16 : integer,
+  p17 : integer,
+  p18 : integer,
+  p19 : integer,
+  p20 : integer,
+  p21 : integer,
+  p22 : integer,
+  p23 : integer,
+  p24 : integer,
+  p25 : integer,
+  p26 : integer,
+  p27 : integer,
+  p28 : integer,
+  p29 : integer,
+  p30 : integer,
+  p31 : integer,
+  p32 : integer,
+  p33 : integer,
+  p34 : integer,
+  p35 : integer,
+  p36 : integer,
+  p37 : integer,
+  p38 : integer,
+  p39 : integer,
+  p40 : integer,
+  p41 : integer,
+  p42 : integer,
+  p43 : integer,
+  p44 : integer,
+  p45 : integer,
+  p46 : integer,
+  p47 : integer,
+  p48 : integer,
+  p49 : integer,
+  p50 : integer,
+  p51 : integer,
+  p52 : integer,
+  p53 : integer,
+  p54 : integer,
+  p55 : integer,
+  p56 : integer,
+  p57 : integer,
+  p58 : integer,
+  p59 : integer,
+  p60 : integer,
+  p61 : integer,
+  p62 : integer,
+  p63 : integer,
+  p64 : integer,
+  p65 : integer,
+  p66 : integer,
+  p67 : integer,
+  p68 : integer,
+  p69 : integer,
+  p70 : integer) ;
 begin
 begin
-  g := p63 + p70
+  g := p16 + p1 ;
+  unused := p63 + p70
 end ;
 end ;
 begin
 begin
-  far (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
+  far (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16) ;
+  writeln (g)
 end.
 end.

+ 199 - 0
shell/tests/nonvacuity.sh

@@ -34,6 +34,11 @@
 #                      triple-faults on the second call - so nothing short of
 #                      triple-faults on the second call - so nothing short of
 #                      running it, or of stating the register contract
 #                      running it, or of stating the register contract
 #                      explicitly, can see it.
 #                      explicitly, can see it.
+#   run_com_exec.py   behaviour:            catches a*b that emits an ADD, `>'
+#                      and `>=' swapped, REPEAT..UNTIL that stops after one
+#                      pass, and two procedures whose parameters collide.
+#                      All four sat in fixtures that COMPILED and were never
+#                      RUN, with every byte-level check green.
 #
 #
 # The mod=11 cases do not need a rebuild -- they read the probe sources
 # The mod=11 cases do not need a rebuild -- they read the probe sources
 # directly -- so they are cheap, and they are the ones that matter most: the
 # directly -- so they are cheap, and they are the ones that matter most: the
@@ -349,6 +354,200 @@ else
     fail=$((fail + 1))
     fail=$((fail + 1))
 fi
 fi
 
 
+# --- 3. the operator bugs the nine dead fixtures were hiding ------------
+echo
+echo "== the four bugs the nine never-executed fixtures were hiding"
+echo
+# A different KIND of case from everything above.  The others break the code
+# and assert a byte-level check notices; these break the code and assert a
+# BEHAVIOURAL check notices, which is the only kind that could have found them.
+# All four shipped with a green compile matrix, a passing .COM layout check, a
+# passing golden and a passing emitter audit:
+#
+#   OpMul = 1        `a * b` emitted ADD AX,CX.  `*' and `+' both numbered
+#                    their operator 1, and BinOpEmit cannot see which
+#                    precedence level called it, so every multiplication
+#                    dispatched to the addition.  The constant-folding arm was
+#                    correct, which is why `n * n' with n a CONST was right and
+#                    `a * a' with a a variable was not -- and t08_const is the
+#                    only fixture that ever multiplied.
+#   9Dh / 9FH        `>' got SETGE and `>=' got SETG: swapped, one letter
+#                    apart in the mnemonic.  Only a==b could see it.
+#   JNZ -> body      REPEAT..UNTIL looped back while the condition was TRUE,
+#                    which is WHILE, so the body ran once and stopped.
+#
+# They are mutated back to the original defect and run_com_exec.py must go red
+# on the exact fixture that pins the behaviour.  The fourth (HideLocals) is a
+# scoping bug rather than an operator bug; it was hiding in the same place.
+mutate_compiler () {   # reuse mutate's verified-change discipline on Compiler.mod
+    mf=Compiler.mod
+    msed=$1
+    cp "$SAVED_C" /tmp/opencode/nonvacuity.mut2.bak
+    sed -i "$msed" "$mf"
+    if cmp -s "$mf" /tmp/opencode/nonvacuity.mut2.bak; then
+        echo "  BROKEN CASE: the mutation did not change Compiler.mod"
+        echo "       sed: $msed"
+        echo "       the named code has probably been renamed or reformatted -"
+        echo "       fix this case, it is asserting nothing"
+        fail=$((fail + 1))
+        return 1
+    fi
+    return 0
+}
+
+# The SETcc swap and the HideLocals removal are done in python rather than
+# with sed: both need to match source text containing `*` and `(` in a way that
+# is tedious and fragile as a regex, and a case whose only failure mode is a
+# malformed sed is a case that silently asserts nothing.
+#
+# And a python helper fails in a way sed does not: a syntax error in the helper
+# is a non-zero exit, `if mutate_foo; then` is simply false, and the case is
+# SKIPPED -- with no failure counted and nothing on stdout but whatever python
+# printed.  That is how the SETcc case spent its first run: an apostrophe in an
+# assert message ("the `>' arm") closed the string early, python died, the
+# compiler was never broken, and the suite still reported 0 failed.  A skipped
+# case and a passing case look the same in the total.  So each helper below
+# fails LOUDLY: a non-zero exit from python is reported as a BROKEN CASE and
+# counted, never swallowed.
+#
+# Each one also counts its targets before replacing.  `assert s != before' only
+# says the file changed; with two edits it would pass if just one of them
+# landed, and with two identical HideLocals call sites it would happily delete
+# the wrong one -- still a changed file, still a working compiler, still green
+# for the wrong reason.
+mutate_cc_swap () {
+    if python3 - <<'PYX'
+p = 'Compiler.mod'
+s = open(p).read()
+GT = 'EmSetcc (9FH) ;          (* >  SETG  *)'    # the greater-than arm
+GE = 'EmSetcc (9DH) ;          (* >= SETGE *)'    # the greater-equal arm
+assert s.count(GT) == 1, 'expected 1 greater-than arm, found %d' % s.count(GT)
+assert s.count(GE) == 1, 'expected 1 greater-equal arm, found %d' % s.count(GE)
+s = s.replace(GT, GT.replace('9FH', '9DH'))
+s = s.replace(GE, GE.replace('9DH', '9FH'))
+open(p, 'w').write(s)
+PYX
+    then
+        return 0
+    fi
+    echo "  BROKEN CASE: the SETcc swap did not apply"
+    echo "       the two EmSetcc arms are probably renamed or reformatted -"
+    echo "       fix this case, it is asserting nothing"
+    fail=$((fail + 1))
+    return 1
+}
+
+mutate_no_hidelocals () {
+    if python3 - <<'PYX'
+p = 'Compiler.mod'
+s = open(p).read()
+# Two HideLocals calls exist.  Only the body-exit one may go: deleting the
+# FORWARD one instead would still change the file, still rebuild, and still
+# leave t13_proc compiling, so the case would go green for the wrong reason.
+HL = '   HideLocals (nestMark) ;         (* parameters and locals stop here *)\n'
+assert s.count(HL) == 1, 'expected 1 body-exit HideLocals, found %d' % s.count(HL)
+open(p, 'w').write(s.replace(HL, ''))
+PYX
+    then
+        return 0
+    fi
+    echo "  BROKEN CASE: HideLocals was not removed"
+    echo "       the call or its comment has probably been reformatted -"
+    echo "       fix this case, it is asserting nothing"
+    fail=$((fail + 1))
+    return 1
+}
+
+cp "$SAVED_C" Compiler.mod
+if mutate_compiler 's|^         op := OpMul ; DropCh|         op := OpAdd ; DropCh|'; then
+    if rebuild_compiler; then
+        expect_red "execution catches '*' emitting an ADD (t08_const, n*n)" \
+            "t08_const" python3 tests/run_com_exec.py t08_const
+        # t08 only ever multiplied two CONSTANTS, which is the one path that was
+        # never wrong, because BinOpEmit folds it.  So the case above is close
+        # to vacuous: it proves the mutation changed the binary, not that the
+        # emitted multiply is covered.  The check that matters needs a
+        # VARIABLE operand, and no shipped fixture has one -- which is why the
+        # bug survived at all.  So this writes a throwaway fixture that
+        # multiplies a variable, runs it, and asserts the multiply is right.
+        # The fixture is deleted afterwards; it is here to close the coverage
+        # hole, not to become a permanent test (that is what a real fixture
+        # with a `*' in it would be for).
+        cat > tests/fixtures/zzmul.pas <<'ZZEOF'
+program zzmul;
+var a : integer ;
+begin
+  a := 7 ;
+  writeln (a * 6)
+end.
+ZZEOF
+        printf '42\r\n' > tests/fixtures/zzmul.out
+        expect_red "execution catches '*' on a VARIABLE (the unfolded path)" \
+            "zzmul" python3 tests/run_com_exec.py zzmul
+        rm -f tests/fixtures/zzmul.pas tests/fixtures/zzmul.out
+    else
+        echo "  FAIL: the compiler would not rebuild with OpAdd for '*'"
+        fail=$((fail + 1))
+    fi
+fi
+cp "$SAVED_C" Compiler.mod
+
+cp "$SAVED_C" Compiler.mod
+if mutate_cc_swap; then
+    if rebuild_compiler; then
+        expect_red "execution catches '>' and '>=' swapped (t09_if)" \
+            "t09_if" python3 tests/run_com_exec.py t09_if
+    else
+        echo "  FAIL: the compiler would not rebuild with the SETcc swap"
+        fail=$((fail + 1))
+    fi
+fi
+cp "$SAVED_C" Compiler.mod
+
+cp "$SAVED_C" Compiler.mod
+if mutate_compiler 's|      DropC (EmJcc (84H, L1)) ;        (\* JZ -> body again \*)|      zj := EmJcc (85H, L1) ;|'; then
+    if rebuild_compiler; then
+        expect_red "execution catches REPEAT..UNTIL exiting after one pass (t12)" \
+            "t12_repeat" python3 tests/run_com_exec.py t12_repeat
+    else
+        echo "  FAIL: the compiler would not rebuild with the JNZ repeat"
+        fail=$((fail + 1))
+    fi
+fi
+cp "$SAVED_C" Compiler.mod
+
+# The fourth: sibling procedures shared one parameter namespace, because a
+# finished procedure's symbols were left at a level Search still accepts.
+# Removing HideLocals puts two procedures' `a : integer' back in collision.
+cp "$SAVED_C" Compiler.mod
+if mutate_no_hidelocals; then
+    if rebuild_compiler; then
+        expect_red "a duplicate parameter in two procedures is a compile error again" \
+            "ERROR 41" python3 tests/run_com_exec.py t13_proc
+    else
+        echo "  FAIL: the compiler would not rebuild without HideLocals"
+        fail=$((fail + 1))
+    fi
+else
+    echo "  FAIL: could not remove HideLocals to test the scoping fix"
+    fail=$((fail + 1))
+fi
+cp "$SAVED_C" Compiler.mod
+
+if rebuild_compiler; then
+    if python3 tests/run_com_exec.py >/dev/null 2>&1; then
+        echo "  ok: all 30 executed fixtures pass on the restored compiler"
+        pass=$((pass + 1))
+    else
+        echo "NOT RESTORED: run_com_exec.py is red after restoring Compiler.mod"
+        python3 tests/run_com_exec.py 2>&1 | grep -i fail | head -3 | sed 's/^/       /'
+        fail=$((fail + 1))
+    fi
+else
+    echo "NOT RESTORED: the compiler would not rebuild"
+    fail=$((fail + 1))
+fi
+
 echo
 echo
 echo "== the emitter-name audit of Compiler.mod (audit_helpers.py)"
 echo "== the emitter-name audit of Compiler.mod (audit_helpers.py)"
 # These need no rebuild: the audit reads the SOURCE, not the built object, so
 # These need no rebuild: the audit reads the SOURCE, not the built object, so