|
|
@@ -24,7 +24,7 @@ manual, not guessed.
|
|
|
| Execute the nine fixtures that only compiled | `v-TP3-DEAD-FIXTURES` | done, **30/30 run; four operator/scoping bugs found** |
|
|
|
| Runtime entries under qemu, and the register contract stated | `v-TP3-BP-CONTRACT` | done, **36/36 entry checks; `wrchar`/`wrbool` no longer destroy BP** |
|
|
|
| 8086-legal conditional branches and `SETcc` | `v-TP3-8086-LOWERING` | done, **the emitted code no longer contains an opcode the 8086 lacks** |
|
|
|
-| `CmdRun` (the `R` key), in-process 8086 interpreter | — | **not started** |
|
|
|
+| `CmdRun` (the `R` key) + `Exec86`, the in-process 8086 interpreter | `v-TP3-CMDRUN` | done, **a second execution oracle: 33/33 fixtures agree with qemu byte for byte, and `R` runs one inside the shell** |
|
|
|
|
|
|
Every row that names a tag has one, and every tag points at a commit on
|
|
|
`master`; verified by diffing the rows against `git tag -l`, which is how
|
|
|
@@ -71,6 +71,15 @@ The one diagnostic is `./Compiler.mod: ParseExpr: too many errors in pass 3`,
|
|
|
which is the expected phase-1 rollup that the recipe tolerates — not a real
|
|
|
error. `shell/Makefile` is the single authoritative build recipe.
|
|
|
|
|
|
+**Scratch and logs live in `TP3-comp/tmp/`, beside the tree that produced
|
|
|
+them** — never in `/tmp`. Every test that writes a build log, a saved copy of a
|
|
|
+mutated source, a probe binary or a corpus puts it there (`../tmp/`, since the
|
|
|
+shell scripts `cd` into `shell/` first), and the folder is gitignored, so a
|
|
|
+failed run's evidence sits next to the code it describes and cannot be
|
|
|
+committed by accident. What still touches the system temp area is only the
|
|
|
+per-run working directory that `tempfile.mkdtemp` / `mktemp -d` creates — an
|
|
|
+anonymous tree, not a named file anyone goes back and reads.
|
|
|
+
|
|
|
**`shell/build_tpshell.sh` is now a three-line wrapper around `make`**, and
|
|
|
that is a fix, not a refactor. It used to be a second hand-maintained copy of
|
|
|
the recipe, and a copy of a build recipe drifts — this one was wrong twice
|
|
|
@@ -85,6 +94,8 @@ knowledge of its own.
|
|
|
## What is verified, and how
|
|
|
|
|
|
Nothing here is "it compiles clean" — each claim below comes from a run.
|
|
|
+`tests/run_all.sh` runs every check below in one pass and prints
|
|
|
+`OVERALL: ALL PASS`, or it prints which one did not.
|
|
|
|
|
|
### Compiler front end — `shell/tests/run_compile_tests.sh`
|
|
|
|
|
|
@@ -104,13 +115,14 @@ pins the verdict *and the numbers* per fixture — verdict plus code size plus
|
|
|
data size, or error number plus position — and the runner compares. A wrong
|
|
|
error position or a program that lost six bytes now fails the suite instead of
|
|
|
needing a squint. It was checked for vacuousness by reverting the string
|
|
|
-scanner fix: 19/23 and exit 1, restored: 23/23 and exit 0.
|
|
|
+scanner fix: the suite went red with exit 1, and came back green only when the
|
|
|
+fix was restored.
|
|
|
|
|
|
-**30 of 33 fixtures compile clean**, up from 1 (the empty program) when the
|
|
|
+**33 of 36 fixtures compile clean**, up from 1 (the empty program) when the
|
|
|
direct harness was first built.
|
|
|
|
|
|
```
|
|
|
-compile matrix: 33 passed, 0 failed (of 33)
|
|
|
+compile matrix: 36 passed, 0 failed (of 36)
|
|
|
```
|
|
|
|
|
|
Compiling: `t01` minimal · `t04` var+assign+`writeln` · `t06` two args ·
|
|
|
@@ -123,17 +135,20 @@ Compiling: `t01` minimal · `t04` var+assign+`writeln` · `t06` two args ·
|
|
|
`t27` five locals · `t28` a 70-parameter declaration · `t29` `readln` from a
|
|
|
supplied input file · `t30` a counted `for` whose bounds come from an
|
|
|
expression · `t31` a value parameter *and* a call across statements ·
|
|
|
-`t32` `EXIT` out of a `for` body.
|
|
|
+`t32` `EXIT` out of a `for` body · `t33` all six comparisons · `t34` the
|
|
|
+operators nothing else uses (`div mod and or`, unary `-`, a variable `*`) ·
|
|
|
+`t35` `not`, both halves of TPSRC9's `neglevel` split.
|
|
|
|
|
|
Failing, all deliberately: `t14` `array [1..5] of integer` at its point of use
|
|
|
and `t25` a string literal used as a *value* (`s := 'hi'`) both → `ENoLib`
|
|
|
(102), the original's "not implemented" path; `uierror` is a deliberate syntax
|
|
|
error (41) used by the UI test.
|
|
|
|
|
|
-`run_com_tests.sh` additionally links **all 30 that compile** to a real `.COM`
|
|
|
+`run_com_tests.sh` additionally links **all 33 that compile** to a real `.COM`
|
|
|
and re-verifies the bytes with an independent Python checker that *measures*
|
|
|
-the layout instead of restating it: `30 checked, 0 failed`. That last part was
|
|
|
-itself a bug fix — see "the two restated constants" below.
|
|
|
+the layout instead of restating it: `independent .COM check: 33 checked, 0
|
|
|
+failed`. That last part was itself a bug fix — see "the two restated
|
|
|
+constants" below.
|
|
|
|
|
|
### Shell + editor UI — `shell/tests/uitest.py`
|
|
|
|
|
|
@@ -156,7 +171,7 @@ changing `Run`'s signature, so `Editor.def` stays additive.
|
|
|
feature: the latter move as the compiler grows, and a test whose
|
|
|
expectations drift with it stops being a test.
|
|
|
|
|
|
-### The encodings — six checks that can each go red
|
|
|
+### The encodings — checks that can each go red
|
|
|
|
|
|
Nothing above looks at *machine code*. It all stops at "the compiler produced
|
|
|
what it intended to produce", which is exactly where the bugs in this project
|
|
|
@@ -170,10 +185,11 @@ proves each one can go red.
|
|
|
|---|---|---|
|
|
|
| `probe/run_modrm19.py` | the mod=00/01/10 effective addresses, by **executing** 23 cases on a real 8086 under qemu and scanning for where the marker landed | mod=11 — see below |
|
|
|
| `probe/modrm11.py` | the mod=11 register identities, by **encoding** with GNU `as` and decoding with FCML, against hard-coded bytes | the table agreeing with itself |
|
|
|
-| `audit_helpers.py` | every one-line emitter in **both** `Runtime.mod` and `Compiler.mod` decodes to what its *name* says — **100/100**, from an inventory scanned independently of the parser | anything longer than one instruction |
|
|
|
+| `audit_helpers.py` | every one-line emitter in **both** `Runtime.mod` and `Compiler.mod` decodes to what its *name* says — **101/101**, from an inventory scanned independently of the parser | anything longer than one instruction |
|
|
|
| `check_runtime.py` + `runtime.golden` | the built runtime's code region (436 bytes, code ends at 405) sweeps cleanly through FCML, every entry and all branch targets land on an instruction boundary, and the whole disassembly is byte-for-byte the committed golden | whether the golden is *right* |
|
|
|
| `check_framedisp.py` | `[BP+off]` uses disp8 iff `off <= 127`, for locals (negative) and far parameters (>127) | which of the two encodings was chosen, if the other also works |
|
|
|
| `run_com_exec.py` | the **emitted image executes** and prints exactly the expected bytes | semantics the fixture never exercises |
|
|
|
+| `run_exec86.py` | the same image runs in a **second, independent 8086** and agrees with qemu byte for byte, 33/33 | the `.out` file it shares with `run_com_exec.py` — a wrong expectation fails both at once |
|
|
|
| `rt_exec.py` | the **runtime entries themselves** are called directly, one qemu boot per call, 35 cases plus a pre-flight, 36/36 — plus a `check_bp_contract` pre-flight over the built blob: an entry that borrows BP must open with `55 8B EC` (exact) and contain a `5D` (a screen, because `5D` is also a displacement byte) | whether the *caller's* half of a contract is right, where the only witness is a case that happens to make two calls in a row |
|
|
|
|
|
|
Two of these deserve the detail, because the reason they exist is the reason
|
|
|
@@ -233,15 +249,19 @@ by building it and requiring the check to stay green.
|
|
|
|
|
|
### Execution under qemu — `tests/run_com_exec.py`
|
|
|
|
|
|
-The sixth check is the one that cannot be written as a byte comparison, so it
|
|
|
-is also the one that finds the most: 31 fixtures are compiled to `.COM`, put on
|
|
|
-a floppy, booted, and their serial output compared to a committed `.out` file
|
|
|
-**exactly** — CRLF included — plus the exit code passed to `INT 21h AH=4Ch`.
|
|
|
+The check that cannot be written as a byte comparison, so also the one that
|
|
|
+finds the most: 33 fixtures are compiled to `.COM`, put on a floppy, booted,
|
|
|
+and their serial output compared to a committed `.out` file **exactly** — CRLF
|
|
|
+included — plus the exit code passed to `INT 21h AH=4Ch`.
|
|
|
|
|
|
```
|
|
|
-execution: 31 passed, 0 failed (of 31)
|
|
|
+execution: 33 passed, 0 failed (of 33)
|
|
|
```
|
|
|
|
|
|
+This is no longer the only execution oracle: `run_exec86.py` below runs the
|
|
|
+same 33 images a second time, in a different machine, and requires the two to
|
|
|
+agree byte for byte.
|
|
|
+
|
|
|
It also found bug 33 — the branch polarity inverted in *every* conditional in
|
|
|
*every* program — while the compile matrix and the `.COM` layout check were both
|
|
|
perfectly happy. That is the third time in a row that a fault passed every
|
|
|
@@ -270,16 +290,15 @@ there are two of them and why both were wrong in the same way.
|
|
|
|
|
|
### 8086 legality — `tests/check_8086.py`
|
|
|
|
|
|
-The twelfth and newest check, and the only one that asks a question about the
|
|
|
-*target* rather than about the compiler: **does the 8086 have this instruction
|
|
|
-at all?**
|
|
|
+The only check that asks a question about the *target* rather than about the
|
|
|
+compiler: **does the 8086 have this instruction at all?**
|
|
|
|
|
|
```
|
|
|
-8086 check: 26 comparison sites, 22 lowered to a Boolean value, 13 lowered to a branch
|
|
|
- value conditions : = x2 <> x2 < x5 >= x3 <= x2 > x8
|
|
|
+8086 check: 35 comparison sites, 31 lowered to a Boolean value, 13 lowered to a branch
|
|
|
+ value conditions : = x6 <> x2 < x5 >= x3 <= x2 > x13
|
|
|
branch conditions : IF / REPEAT x9 CASE x2 FOR downto x1 FOR to x3
|
|
|
runtime: 436 bytes, 219 swept, 0 0F-prefixed
|
|
|
- program code: 28 of 31 fixtures swept end to end, 2257 bytes
|
|
|
+ program code: 30 of 33 fixtures swept end to end, 2897 bytes
|
|
|
t33_cmpops: 13 comparisons matched against their source operators, in order
|
|
|
clause H: 8 of 8 fixtures matched the branch conditions read off their source
|
|
|
```
|
|
|
@@ -320,22 +339,91 @@ through were the same hole. **H** pins, per fixture, the conditions its branch
|
|
|
sites declare, read off the `.pas` sources; its need was *measured* (mutation M5
|
|
|
came back green before H existed) rather than anticipated.
|
|
|
|
|
|
-**What it does not do.** It cannot assert on the CASE arm's label immediate, and
|
|
|
-it never executes anything: it proves the opcodes are 8086 and that conditions
|
|
|
-are attached to the right constructs, but the control flow those bytes produce
|
|
|
-is still only checked by qemu on a 486. An in-process 8086 interpreter is the
|
|
|
-only thing that would close that, which is why `CmdRun` is next.
|
|
|
+**What it does not do.** It cannot assert on the CASE arm's label immediate,
|
|
|
+and it never executes anything: it proves the opcodes are 8086 and that
|
|
|
+conditions are attached to the right constructs, but the control flow those
|
|
|
+bytes produce is a question for execution — and until this milestone it could
|
|
|
+only be asked of qemu, on a machine with no 8086 model. It is now asked twice.
|
|
|
+
|
|
|
+### The second execution oracle — `tests/run_exec86.py`
|
|
|
+
|
|
|
+`shell/Exec86.mod` is a flat 8086 interpreter over the linked image, and this
|
|
|
+check runs every fixture through it as well as through qemu and requires the
|
|
|
+two to agree **byte for byte**:
|
|
|
+
|
|
|
+```
|
|
|
+exec86: 33 passed, 0 failed (of 33), cross-checked against qemu
|
|
|
+```
|
|
|
+
|
|
|
+Three assertions per fixture, in order of how much they are worth: the output
|
|
|
+matches the hand-derived `.out` exactly; the output matches what qemu printed
|
|
|
+for the same bytes; and the guest left through `INT 21h AH=4Ch` with code 0.
|
|
|
+The `.out` files are written from Pascal's semantics and are never blessed
|
|
|
+from a machine's output — `run_exec86.py` has no `--rebless` at all — so
|
|
|
+agreeing with qemu is a **third** opinion, not a second vote on the same one.
|
|
|
+
|
|
|
+It exists because the 8086-legality check could not close its own gap: qemu's
|
|
|
+lowest CPU model is a 486, where `0F 84` is an ordinary `JZ`, so an oracle
|
|
|
+built on qemu is structurally blind to that class of fault in *either*
|
|
|
+direction. This one was written against the 8086's own reference instead, and
|
|
|
+`nonvacuity.sh` proves it can go red on its own: inverting `JE` inside its
|
|
|
+`Cond` swaps the two arms of every `=` in every program, while the emitted
|
|
|
+bytes, the sizes and every byte-level check stay green — and qemu, executing
|
|
|
+the unchanged image, still agrees with itself.
|
|
|
+
|
|
|
+What it does not do: `FLAGS` are never written back, `PF`/`AF` are not
|
|
|
+maintained (`JP`/`JNP` fault saying so rather than answering a value nobody
|
|
|
+computed), no string instruction exists, a non-zero segment register faults,
|
|
|
+and execution outside the loaded image faults. Each is deliberate; `Exec86.mod`'s
|
|
|
+header states the measured instruction set it implements and refuses to guess
|
|
|
+past it, because an interpreter that guesses does not fail, it answers.
|
|
|
+
|
|
|
+### The `R` key, end to end — `tests/runtest.py`
|
|
|
+
|
|
|
+The only check that exercises `CmdRun`, and the only one whose evidence comes
|
|
|
+from a program nobody here has read: drive the shell through a pty, `W` to load
|
|
|
+`t34_arith`, `R`, and compare the **guest's own output** against that fixture's
|
|
|
+hand-derived `.out`.
|
|
|
+
|
|
|
+```
|
|
|
+UI TEST (R): t34_arith.pas
|
|
|
+------------------------------------------------------------
|
|
|
+R reported a successful compile PASS
|
|
|
+R reported poking the image at 0100h PASS
|
|
|
+guest ran to a clean AH=4Ch exit with code 0 PASS
|
|
|
+R reported a non-zero step count PASS
|
|
|
+guest output matches the hand-derived .out PASS
|
|
|
+ESC after the run returned to the main menu PASS
|
|
|
+shell exited cleanly (status 0) PASS
|
|
|
+R wrote no .COM (it runs the image where it already is) PASS
|
|
|
+------------------------------------------------------------
|
|
|
+child: EXIT 0
|
|
|
+RESULT: ALL PASS
|
|
|
+```
|
|
|
+
|
|
|
+The last assertion is the one no other check can make: `R` writes **no file**,
|
|
|
+so the `.COM` on disk must be exactly as it was before — nothing else in this
|
|
|
+project observes `R` at all. It is proved able to fail by neutering the poke
|
|
|
+loop's count: with `n = 0` nothing is copied in, `Run86` faults on its first
|
|
|
+step (*execution left the loaded image*), and the guest's `AH=4Ch` assertion
|
|
|
+goes red before a single instruction has run.
|
|
|
|
|
|
### Non-vacuity — `tests/nonvacuity.sh`
|
|
|
|
|
|
-Every assertion in this file is proved able to fail: **44 deliberate
|
|
|
+Every assertion in this file is proved able to fail: **52 deliberate
|
|
|
breakages, each asserted to turn exactly one named check red for the stated
|
|
|
-reason, then restored and re-asserted green.** Six break the runtime, five
|
|
|
-attack the mod=11 table (including restoring the exact wrong table this
|
|
|
-project once shipped), three target `EmBpDisp` — the truncation, the
|
|
|
-always-disp16 over-encoding that must *stay* green, and the restored source —
|
|
|
-six attack the helper audit, five attack the `.COM` layout checker, and five
|
|
|
-attack the 8086 lowering.
|
|
|
+reason, then restored and re-asserted green** — `non-vacuity: 52 ok, 0 failed`.
|
|
|
+They cover the runtime's emitter audit and its restored source, the mod=11
|
|
|
+table (including restoring the exact wrong table this project once shipped),
|
|
|
+the `[BP+off]` rule, the behavioural bugs, the emitter-name audit of
|
|
|
+`Compiler.mod`, the BP contract, the `.COM` layout checker, the 8086 lowering,
|
|
|
+the interpreter itself, and the `R` key.
|
|
|
+
|
|
|
+Eight of those 52 arrived with this milestone, and each one is the same shape:
|
|
|
+code that compiled clean and passed every byte-level check, until it was
|
|
|
+**run**. Two are behavioural (below), two come from the new interpreter, two
|
|
|
+pin the two new grammar rows the helper audit gained for `EmXorAl01`, and two
|
|
|
+are the `R` key's mutation and its restored green.
|
|
|
|
|
|
That last group is the newest and the least optional. Two of its five
|
|
|
(`M4`, `M5`) invert the branch polarity, which is a *legal* 8086 opcode
|
|
|
@@ -391,7 +479,7 @@ checker.
|
|
|
compile to `.COM` images, are booted on a floppy by a 512-byte hand-assembled
|
|
|
boot sector, and are compared **byte for byte** against the exact output the
|
|
|
fixture demands — `tests/run_com_exec.py`, wired into `run_all.sh`, 21/21 at
|
|
|
-that tag and 30/30 now.
|
|
|
+that tag and 33/33 now.
|
|
|
|
|
|
Everything below is about establishing what *can* be believed, because the
|
|
|
first attempt at this used an emulator that was wrong, and a wrong oracle is
|
|
|
@@ -607,12 +695,60 @@ audit reads the *name*. The emitters are now `MovAlArg2`/`MovAlArg4` and
|
|
|
it: a helper called `CmpArg4W0` that emitted the `+2` form fails the run with
|
|
|
*"step 2: displacement is 2, name says 4"*.
|
|
|
|
|
|
-### Still missing: `CmdRun`
|
|
|
+### The in-process interpreter — `shell/Exec86.mod`, and the `R` key
|
|
|
+
|
|
|
+TP3's `R` runs a `.COM` *in place*, without the DOS loader, from the same
|
|
|
+64 KB DOS would give it. `Exec86` is that machine: a flat 8086 over
|
|
|
+`ARRAY [0..65535] OF CARDINAL` of bytes (one element per byte), with three
|
|
|
+entry points and nothing else:
|
|
|
|
|
|
-The `R` key of the shell is still unimplemented. TP3's `R` runs a `.COM`
|
|
|
-*in place*, without the DOS loader, from the same 64 KB of memory; the
|
|
|
-honest way to do that is a small in-process 8086 interpreter over the
|
|
|
-image, cross-validated against qemu on the same bytes.
|
|
|
+```
|
|
|
+Clear86 wipe all 64 KB; AX..DI := 0; SP := 0FFFEh;
|
|
|
+ IP := 0100h; segments := 0; flags cleared;
|
|
|
+ loadHi := 0100h
|
|
|
+Poke86 (addr, value) mem[addr] := value MOD 256, and raise loadHi
|
|
|
+Run86 (VAR exitCode : CARDINAL;
|
|
|
+ VAR steps : LONGCARD) : CARDINAL
|
|
|
+ 0 = the guest halted through INT 21h AH=4Ch,
|
|
|
+ 1 = fault, 2 = the step limit was reached
|
|
|
+```
|
|
|
+
|
|
|
+`CmdRun` does what the original's `krungo` does with no loader: `Compile` →
|
|
|
+`Clear86` → poke `LinkSize()` bytes at `0100h` → `Run86` → report the status
|
|
|
+and the step count → `WaitEsc`. **No file is written**, so `R` is identical
|
|
|
+with Destination = Memory and Destination = `.COM`, and the guest's own
|
|
|
+`AH=4Ch` exit code is printed rather than discarded.
|
|
|
+
|
|
|
+The guest writes to this process's `fd 1` through the runtime's `INT 21h`
|
|
|
+`AH=02`/`AH=09`/`AH=08`, so its output lands between the two status lines
|
|
|
+instead of being collected and replayed: `Term` writes one byte per `write(2)`
|
|
|
+and buffers nothing, which is why the two streams stay in order.
|
|
|
+
|
|
|
+**Why `Clear86`/`Poke86`/`Run86` rather than `Clear`/`Poke`/`Run`.** ISO
|
|
|
+Modula-2 has no import renaming (`FROM M IMPORT x AS y` is rejected) and no
|
|
|
+procedure-local import, and the shell's flat namespace already contains
|
|
|
+`TextBuf.Clear` and `Editor.Run`. The `86` suffix is the fix, applied
|
|
|
+identically in `Exec86.def`, `Exec86.mod`, `tests/Exec86Run.mod` and the
|
|
|
+module-level import in `Shell.mod`.
|
|
|
+
|
|
|
+**What it deliberately does not do**, each stated in `Exec86.mod`'s own
|
|
|
+header: `FLAGS` are never written back, so `INT 21h` "preserves the flags" by
|
|
|
+construction (a real `INT` pushes them and `IRET` pops them, so the handler's
|
|
|
+`CLC`/`STC` are discarded — `bootcom.s` relies on that); `PF`/`AF` are not
|
|
|
+maintained, so `JP`/`JNP` fault instead of answering a value nobody computed;
|
|
|
+`DF`/`IF`/`TF` are not maintained and no string instruction exists, so nothing
|
|
|
+can read `DF`; a non-zero segment register faults **before every step**,
|
|
|
+because a non-zero segment would silently *alias* onto the same 64 KB instead
|
|
|
+of faulting; an unknown `INT 21h` function faults; and `MaxSteps = 2000000000`
|
|
|
+catches a runaway. Execution outside `[0100h, loadHi)` faults, which is what
|
|
|
+makes the `R` non-vacuity case fail on its very first step.
|
|
|
+
|
|
|
+The instruction set is not "the 8086" but the measured union of (a) every byte
|
|
|
+`Runtime.mod` emits — 219 instructions, swept by `check_8086.py` — and (b)
|
|
|
+every byte `Compiler.mod`'s `Em*` procedures can write (the generated region
|
|
|
+cannot be swept: inline string literals desynchronise a sweep, so the authority
|
|
|
+there is the source). Everything outside that union faults rather than
|
|
|
+guessing, because an interpreter that guesses does not fail — it answers.
|
|
|
|
|
|
## Components
|
|
|
|
|
|
@@ -627,7 +763,9 @@ old file to `.BAK` (unlink+rename, TP3's order); `D` is a DOS `*.*` glob
|
|
|
listing with `k bytes free`; `O` is the options submenu; `Q` confirms and
|
|
|
prompts to save when the text changed.
|
|
|
|
|
|
-`E` and `C` are wired. **`R` is a stub** — it prints "Interpreter pending".
|
|
|
+`E`, `C` and `R` are wired. `R` compiles, pokes the linked image into the
|
|
|
+in-process interpreter at `0100h`, runs it and reports the guest's exit status
|
|
|
+and the step count — see *The in-process interpreter* above.
|
|
|
|
|
|
### Editor — `shell/Editor.mod`
|
|
|
|
|
|
@@ -876,14 +1014,21 @@ Details that are deliberate, not incidental:
|
|
|
|
|
|
## Honest limitations
|
|
|
|
|
|
-- **`CmdRun` — the `R` key — is still a stub.** The compiler's *output* now
|
|
|
- executes (30 fixtures, exact output, exit codes), but the shell cannot run a
|
|
|
- `.COM` in place. The linker writes a real `.COM` and the boot sector runs one
|
|
|
- under qemu; nothing in the host program yet interprets 8086 code. So the
|
|
|
- user's route to seeing output is "compile, then run under qemu", not "press
|
|
|
- `R`". TP3's `R` runs in the same 64 KB with no DOS loader, which is why this
|
|
|
- is an interpreter and not a `system()` call.
|
|
|
-- **Every fixture that compiles is now also run.** 30 of 33 execute; the 3 that
|
|
|
+- **A multi-argument call whose earlier argument is a computed value is still
|
|
|
+ wrong, and is not claimed to be fixed.** `SaveLeft` parks a kind-2 operand
|
|
|
+ (a value that exists only in `AX`) across the parse of the *other* operand of
|
|
|
+ a binary operator, which is what fixed `(p > q) or (q > p)`. The three call
|
|
|
+ parsers never park an argument that has already been parsed, so in
|
|
|
+ `f (a > b, x)` the parse of `x` overwrites `a > b`'s value before the call is
|
|
|
+ emitted, and `f (a > b, c > d)` passes the second comparison twice.
|
|
|
+ Reproduced exactly as written here; the fix belongs to the call path and was
|
|
|
+ out of scope for the operator fix.
|
|
|
+- **`runtest.py` runs one fixture through `R`, not 33.** It drives `t34_arith`
|
|
|
+ through the pty because a pty test is expensive and this one's job is to
|
|
|
+ prove the *path* exists (compile → poke → run → report → no file written),
|
|
|
+ which it does with eight assertions. The other 32 are covered by
|
|
|
+ `run_exec86.py`, which calls the same interpreter directly.
|
|
|
+- **Every fixture that compiles is now also run.** 33 of 36 execute; the 3 that
|
|
|
do not are `t14` and `t25` (`ENoLib`, by design) and `uierror` (a deliberate
|
|
|
syntax error). The gap this replaces was nine fixtures that compiled and were
|
|
|
*never executed* — `t08` const, `t09` if/then/else, `t10` while, `t11` for/to,
|
|
|
@@ -938,7 +1083,7 @@ Details that are deliberate, not incidental:
|
|
|
(`7x 03 E9`, searching for the `E9`) and by `t22_case`'s execution, but the
|
|
|
label immediate itself is unchecked.
|
|
|
- **The runtime's entries are now each called directly, and two of its
|
|
|
- invariants are stated rather than implied.** 436 bytes, 14 entries, 100
|
|
|
+ invariants are stated rather than implied.** 436 bytes, 14 entries, 101
|
|
|
emitter helpers decoded against their own names across both modules, the
|
|
|
whole code region golden-pinned, all branch targets on instruction
|
|
|
boundaries — and 35 cases that call the entries one at a time under qemu
|
|
|
@@ -953,9 +1098,11 @@ Details that are deliberate, not incidental:
|
|
|
hard-coded — but the two `RT_SZ` constants that *were* hard-coded and had
|
|
|
drifted (see the execution section) are the precedent for why this one gets
|
|
|
stated every time.
|
|
|
-- **30 fixtures is a small sample of Pascal.** They cover `var`, `const`,
|
|
|
+- **33 executed fixtures is a small sample of Pascal.** They cover `var`, `const`,
|
|
|
`if`, `while`, `for`, `repeat`, `case` over scalars, procedures with value
|
|
|
- parameters, `goto`/`label`, string literals and `readln`. They do **not**
|
|
|
+ parameters, `goto`/`label`, string literals, `readln`, all six comparisons,
|
|
|
+ and the arithmetic and logical operators `* + - div mod and or not` on
|
|
|
+ **variables**. They do **not**
|
|
|
cover nested procedures, recursion, `var` parameters, `with`, records, sets,
|
|
|
files, reals, or any type wider than 2 bytes — all still `ENoLib`. A green
|
|
|
execution matrix says nothing about those.
|
|
|
@@ -1299,8 +1446,8 @@ fault than the previous thirty-one and is worth setting out at length.
|
|
|
|
|
|
What makes this worth a section rather than a bullet is that **everything
|
|
|
was green while it was true.** The compile matrix passed. The `.COM` layout
|
|
|
- checker passed. The runtime golden passed. The emitter audit passed. Thirty
|
|
|
- fixtures booted in qemu and printed exactly their hand-derived expected
|
|
|
+ checker passed. The runtime golden passed. The emitter audit passed. Every
|
|
|
+ fixture booted in qemu and printed exactly its hand-derived expected
|
|
|
bytes. Two reasons, and the second is the one to remember:
|
|
|
|
|
|
- **qemu-system-i386 has no 8086 model.** Its lowest is 486, where
|
|
|
@@ -1384,15 +1531,64 @@ its branch sites declare, **read off the `.pas` sources** and written out with
|
|
|
the reasoning beside each row — a table measured from the image would agree with
|
|
|
any behaviour including a wrong one.
|
|
|
|
|
|
-Five mutations are now permanent cases in `tests/nonvacuity.sh` (44 ok, 0
|
|
|
-failed, up from 39): M1 and M2 restore each original defect, M3 swaps `>`/`>=`,
|
|
|
+Five mutations are now permanent cases in `tests/nonvacuity.sh` (`52 ok, 0
|
|
|
+failed` in total across every section; these five were `44 ok` when added):
|
|
|
+M1 and M2 restore each original defect, M3 swaps `>`/`>=`,
|
|
|
M4 inverts the branch polarity everywhere, M5 inverts it for IF and CASE only.
|
|
|
M5's first run was the one that came back green, and that is the case's whole
|
|
|
reason for existing.
|
|
|
|
|
|
+### Then two fixtures were written for the untested operators, and two more appeared
|
|
|
+
|
|
|
+34. **A computed left operand was destroyed while the right one was parsed.**
|
|
|
+ `t34_arith` exists because `div`, `mod`, `and`, `or`, unary `-` and a
|
|
|
+ *variable* `*` had never been executed by anything — `t08` was the only
|
|
|
+ fixture that multiplied, and it multiplied two **constants**, which
|
|
|
+ `BinOpEmit` folds away without emitting an instruction at all. Its
|
|
|
+ `and`/`or` lines are a second first: `(p > q) and (q > p)` puts a value
|
|
|
+ that exists only in `AX` on *both* sides of an operator, and nothing kept
|
|
|
+ the left one alive while the right one was parsed. The `or` line printed
|
|
|
+ `FALSE` where Pascal says `TRUE`.
|
|
|
+
|
|
|
+ `SaveLeft` pushes it the moment the operator is recognised (`kind := 4`)
|
|
|
+ and `LoadPair` then materialises `AX = left`, `CX = right` in whichever of
|
|
|
+ three shapes the situation needs; the third is the original path and is
|
|
|
+ still correct for its case. Both are named in `Compiler.mod` because the
|
|
|
+ shape table *is* the fix — an inline "just reload it" at one call site
|
|
|
+ would not survive the next operator.
|
|
|
+
|
|
|
+ This bug is also why the fix has a stated boundary: the **call** path has
|
|
|
+ the same hole and does not have it fixed (`f (a > b, x)`). A fix that
|
|
|
+ claims "computed operands" while only covering binary operators is worse
|
|
|
+ than one that writes its edge down; see Honest limitations.
|
|
|
+
|
|
|
+35. **`not` was lowered identically for booleans and integers, so every
|
|
|
+ boolean negation was wrong.** TPSRC9's `neglevel` picks the instruction
|
|
|
+ from the operand's *type* before it emits anything — `NOT AX` (`F7 D0`)
|
|
|
+ for an integer, `XOR AL,#01` (`34 01`) for a boolean, error 47 for
|
|
|
+ anything else — and this compiler emitted `NOT AX` for both. So
|
|
|
+ `not (a = 17)` computed `0FFFEh`, and the runtime's `wrbool` tests
|
|
|
+ `[BP+4] <> 0`, which reads `0FFFEh` as TRUE. Pascal says FALSE.
|
|
|
+
|
|
|
+ **Exec86 and qemu agreed with each other and both disagreed with Pascal**,
|
|
|
+ which is what pins the fault on the compiler rather than on either
|
|
|
+ interpreter: two machines built independently cannot share a bug in an
|
|
|
+ emitter neither of them ever reads. It also made the new oracle worth
|
|
|
+ having — the same disagreement would have been invisible in a suite whose
|
|
|
+ only second opinion was a different execution of the same bytes.
|
|
|
+
|
|
|
+ The fix is `neglevel`'s own split, not a special case: `ParseNeg` dispatches
|
|
|
+ on `r.cls`, so `t35_not` carries *both* arms — `not a` has to stay `NOT AX`
|
|
|
+ and must not be dragged to `XOR AL,#01` by a fix aimed at booleans. A
|
|
|
+ narrow row in the helper audit pins the new emitter by its name: a helper
|
|
|
+ called `XorAl01` that emits an immediate of `02` fails with *"immediate is
|
|
|
+ 2, name says 1"*, and the matching opcode row refuses a byte no name
|
|
|
+ claims — moving the `34H` to `35H` fails with *"no name pattern accepts
|
|
|
+ it"*, so the emitter cannot silently become a different instruction.
|
|
|
+
|
|
|
## The bug family, stated once
|
|
|
|
|
|
-Nine of the thirty-two are the *same* bug in different clothes: **loading the
|
|
|
+Nine of the thirty-five are the *same* bug in different clothes: **loading the
|
|
|
address where the value was wanted, or picking the register one byte or one
|
|
|
letter away from the right one.** `EmPushVarAddr` had the right bytes for the
|
|
|
wrong register. `LdAlBx` and `MovAlBl` are one letter apart. `MovAh0` and
|
|
|
@@ -1469,34 +1665,43 @@ independently-scanned inventory at all.
|
|
|
|
|
|
## Next steps
|
|
|
|
|
|
-1. **`CmdRun`** as an in-process 8086 interpreter — the `R` menu key, and a
|
|
|
- fallback executor for environments with no DOS. Validate it against qemu on
|
|
|
- the *same images*, so the two oracles check each other. Cross-validation is
|
|
|
- the point: an interpreter that agrees with qemu on 31 fixtures is far more
|
|
|
- evidence than either alone. It is also the only candidate for an **8086**
|
|
|
- execution oracle, since qemu cannot be one.
|
|
|
-2. **String *variables*** — `s : string`, `s := 'hi'`, `writeln(s)`. The
|
|
|
+1. **Close the third restated `RT_SZ`.** `tests/check_framedisp.py` hard-codes
|
|
|
+ `RT_SZ = 391` where the runtime now measures 436, so `img[RTSZ:]` begins 61
|
|
|
+ bytes inside the runtime tail and the check passes by luck. `run_com_tests.sh`
|
|
|
+ and `comtest.py` carried the same constant and were fixed by *measuring* the
|
|
|
+ header from its own signature; this one needs that shared helper
|
|
|
+ (`tests/comimage.py`) and, being new, its own non-vacuity case — a green
|
|
|
+ nobody has ever seen red is the failure mode this project keeps
|
|
|
+ rediscovering, and this is the last known instance of it.
|
|
|
+2. **The multi-argument kind-2 clobber.** `f (a > b, x)` passes a wrong first
|
|
|
+ value, and `f (a > b, c > d)` passes the second comparison twice.
|
|
|
+ `SaveLeft` parks a computed operand across the parse of the *other* operand
|
|
|
+ of a binary operator; the three call parsers park nothing. Fixture first, so
|
|
|
+ the bug is red before the fix (see "Honest limitations").
|
|
|
+3. **String *variables*** — `s : string`, `s := 'hi'`, `writeln(s)`. The
|
|
|
encoding blocker is gone (`EmBpDisp`); what is left is a length word, an
|
|
|
assignment path, and a `WrStr` entry (TPSRC4 `xwrtstr`). `IoCall` currently
|
|
|
refuses with `ENoLib`.
|
|
|
-3. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from
|
|
|
+4. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from
|
|
|
RESUME-TP3.md §3.11), range/index checks (`TU_RANGE_CHECK`,
|
|
|
`TU_INDEX_CHECK`), typed constants (RESUME-TP3.md §3.14), `array` at its
|
|
|
point of use (`t14`), `case` with subrange labels.
|
|
|
-4. **`readln` of a `BYTE`** calls `rdint`, which stores 2 bytes and overflows
|
|
|
+5. **`readln` of a `BYTE`** calls `rdint`, which stores 2 bytes and overflows
|
|
|
into the next variable. TP3 has a separate `xrdbyte`; a `TU_RdByte` entry is
|
|
|
the fix. No fixture exists yet, which is why it has not been done — write
|
|
|
the fixture first, so the bug is red before the fix.
|
|
|
-5. Make the 4 KiB code window an enforced limit rather than a documented one:
|
|
|
+6. Make the 4 KiB code window an enforced limit rather than a documented one:
|
|
|
report an error when `pc` reaches `dc`, instead of writing over the data.
|
|
|
-6. Both spellings of a multi-name declaration. `var i, c : integer;` is
|
|
|
+7. Both spellings of a multi-name declaration. `var i, c : integer;` is
|
|
|
error 1 at the comma and needs two `var` lines; `procedure f (a : integer;
|
|
|
b : integer)` is error 1 at the semicolon and needs a comma. Neither is
|
|
|
wrong Pascal, so a program that compiles under one compiler may not under
|
|
|
another. A parameter may also not shadow a global (`DupTest` rejects any
|
|
|
name `Search` finds at any level), which Pascal allows.
|
|
|
-7. Harden the program-header parameter loop against non-advancing input
|
|
|
+8. Harden the program-header parameter loop against non-advancing input
|
|
|
(`program p(1;)`) with a `BOOLEAN` flag — **not** `EXIT`, which ICEs gm2.
|
|
|
-8. FreeDOS (`freedos.qcow2`, FD14-LiveCD) is still untried. Not needed for any
|
|
|
- claim above, but it is the only way to get a *real* DOS as a third opinion
|
|
|
- on the `INT 21h` shim.
|
|
|
+9. FreeDOS (`freedos.qcow2`, FD14-LiveCD) is still untried. Now that `R` runs
|
|
|
+ the image in-process, a real DOS is no longer needed for any claim above —
|
|
|
+ but it is still the only way to get a third opinion on the `INT 21h` shim,
|
|
|
+ and `Exec86` deliberately implements only the three functions the runtime
|
|
|
+ calls.
|