# TP3-comp — state summary Recreating **Turbo Pascal 3.0** (shell / editor / compiler / interpreter) in **GNU Modula-2** (`gm2 -fiso -Wall`). Faithful to the original: screen layout, keys and behaviour are reconstructed from the disassembled TP3.0 source in `Resources/turbopascal3source/TP3/` (TPSRC1-10) and the reference manual, not guessed. ## Milestones | Milestone | Tag | State | |---|---|---| | TP3.0 main-menu shell | `v0.1-shell` | done | | WordStar-style editor | `v0.2-editor` | done | | Shell/editor polish, Ctrl-K-D / Ctrl-K-X quit | `v-TP3-SHELL-EDITOR-QUIT` | done | | Compiler skeleton + build recipe | `v-TP3-SHELL-COMPILES` | done | | Parser `Skip` bug class (9 sites) | `v-TP3-PARSER-FIXES` | done | | Standard procedures + `rel16` fix | `v-TP3-STDPROCS` | done | | Runtime library + 8086 execution harness | `v-TP3-RUNTIME-BLOB` | assembled, executed under qemu | | Inline string literals (`writeln('hi')`) | `v-TP3-STRLITERAL` | done, executed | | Linker: real DOS `.COM` writer + independent byte checker | `v-TP3-COM-IMAGE` | done, executed | | Measured encodings: ModR/M table, runtime audit, golden disassembly, `[BP+off]` | `v-TP3-MEASURED-EMITTERS` | done, executed | | Execution: a boot sector, qemu, and a claim about behaviour | `v-TP3-EXECUTION` | done, **21/21 fixtures run, exact output** | | Runtime entries under qemu, and the register contract stated | `v-TP3-BP-CONTRACT` | done, **36/36 entry checks; `wrchar`/`wrbool` no longer destroy BP** | | `CmdRun` (the `R` key), in-process 8086 interpreter | — | **not started** | ## Build ```sh cd shell && make # → shell/tpshell ``` Whole-program link **must** be two-phase; a single `gm2 -o` pass 3 silently caps identifier/error counts on a compiler-sized program: ```sh gm2 -fiso -c Compiler.mod # phase A gm2 -fiso -fgen-module-list=modules.lst -o /dev/null \ Compiler.mod Term.o TextBuf.o Posix.o Editor.o # phase B1 (rc=1 expected) gm2 -fiso -fuse-module-list=modules.lst -o tpshell \ Shell.mod Compiler.mod Term.o TextBuf.o Posix.o Editor.o # phase B2 ``` Current clean build: `make clean && make` → rc=0, `tpshell` **163616 bytes**. The one diagnostic is `./Compiler.mod: ParseExpr: too many errors in pass 3`, which is the expected phase-1 rollup that the recipe tolerates — not a real error. `shell/Makefile` is the single authoritative build recipe. **`shell/build_tpshell.sh` is now a three-line wrapper around `make`**, and that is a fix, not a refactor. It used to be a second hand-maintained copy of the recipe, and a copy of a build recipe drifts — this one was wrong twice over: it ran `gm2 -c Posix.mod`, but `Posix` is a *foreign C module* built by `cc -c Posix.c` and there is no `Posix.mod`, so it died on the third module every time; and it printed phase 2's return code and then carried on regardless, so a failed link was reported as a success whenever an older `tpshell` was still lying around. The copy that is wrong is the one nobody runs, which is how it survived. The wrapper keeps the old invocation working and holds no build knowledge of its own. ## What is verified, and how Nothing here is "it compiles clean" — each claim below comes from a run. ### Compiler front end — `shell/tests/run_compile_tests.sh` `tests/CompileTest.mod` links `Compiler` + `TextBuf` + `Posix`, reads fixture paths from **stdin**, loads each exactly the way `LoadWorkFile` does (LF→CR normalisation, `^Z` ends the text), calls `Compile`, and prints a verdict plus a source excerpt with a caret at `errPos`. No pty, instant. ``` cd shell && tests/run_compile_tests.sh # all fixtures cd shell && tests/run_compile_tests.sh /some/dir # another fixture set printf '@dump\ntests/fixtures/t19_int1.pas\n' | ./compiletest # hex-dump the image ``` **The matrix asserts; it does not just count.** `tests/fixtures/expected.tsv` pins the verdict *and the numbers* per fixture — verdict plus code size plus data size, or error number plus position — and the runner compares. A wrong error position or a program that lost six bytes now fails the suite instead of needing a squint. It was checked for vacuousness by reverting the string scanner fix: 19/23 and exit 1, restored: 23/23 and exit 0. **30 of 33 fixtures compile clean**, up from 1 (the empty program) when the direct harness was first built. ``` compile matrix: 33 passed, 0 failed (of 33) ``` Compiling: `t01` minimal · `t04` var+assign+`writeln` · `t06` two args · `t07` 10 assignments · `t08` const · `t09` if/then/else · `t10` while · `t11` for/to · `t12` repeat/until · `t13` procedure + value param · `t15` label + goto · `t16` `writeln('a')` · `t18` bare `writeln` · `t19` `writeln(1)` · `t20` 3 string args · `t21` mixed args · `t22` `case` with two labels · `t23` `writeln('')` · `t24` `writeln('don''t')` · `t26` mixed scalar/string args · `t02`/`t03`/`t05`/`t17` multi-char literals · `t27` five locals · `t28` a 70-parameter declaration · `t29` `readln` from a supplied input file · `t30` a counted `for` whose bounds come from an expression · `t31` a value parameter *and* a call across statements · `t32` `EXIT` out of a `for` body. Failing, all deliberately: `t14` `array [1..5] of integer` at its point of use and `t25` a string literal used as a *value* (`s := 'hi'`) both → `ENoLib` (102), the original's "not implemented" path; `uierror` is a deliberate syntax error (41) used by the UI test. `run_com_tests.sh` additionally links **all 30 that compile** to a real `.COM` and re-verifies the bytes with an independent Python checker that *measures* the layout instead of restating it: `30 checked, 0 failed`. That last part was itself a bug fix — see "the two restated constants" below. ### Shell + editor UI — `shell/tests/uitest.py` Drives a real pty (`tests/ptyharness.py` supplies read-until-quiet, key sending and a small VT100 emulator) and asserts the TP3 **compile-error jump**: `W` load → `C` compile → `ESC` → editor opens with the cursor on the error position → `Ctrl-K D` back to the menu → `Q` exit 0. **10/10 pass**, and the reported error number is 41 (not 0 — this is what caught the `errNo := errNo` self-assignment where `Compile`'s formal shadowed the module variable). The jump mirrors original TP3 `kcwait` + `editor2` (TPSRC5:333-336, :919): `BX:=txerrpos; DEC BX; JMP editor2`, and `editor2` does `ADD BX,txbeg; INC BX` — the `DEC`/`INC` cancel, so the net is `txbeg+txerrpos` = our 0-based `errPos`. Armed as a sticky position (`Editor.GotoOffset`) rather than by changing `Run`'s signature, so `Editor.def` stays additive. `uitest.py` uses a fixture with a deliberate **syntax** error (`x := 1 + ;` → error 41, line 9 col 12) rather than a missing library feature: the latter move as the compiler grows, and a test whose expectations drift with it stops being a test. ### The encodings — six checks that can each go red Nothing above looks at *machine code*. It all stops at "the compiler produced what it intended to produce", which is exactly where the bugs in this project live: a wrong ModRM byte is not a compile error and not a wrong code size, it is a perfectly well-formed instruction that does something else. So the encodings get their own stack, and it is built so that each check fails on a *different* class of mistake. `tests/run_all.sh` runs all of them; `tests/nonvacuity.sh` proves each one can go red. | check | what it asserts | what it cannot see | |---|---|---| | `probe/run_modrm19.py` | the mod=00/01/10 effective addresses, by **executing** 23 cases on a real 8086 under qemu and scanning for where the marker landed | mod=11 — see below | | `probe/modrm11.py` | the mod=11 register identities, by **encoding** with GNU `as` and decoding with FCML, against hard-coded bytes | the table agreeing with itself | | `audit_helpers.py` | every one-line emitter in **both** `Runtime.mod` and `Compiler.mod` decodes to what its *name* says — **100/100**, from an inventory scanned independently of the parser | anything longer than one instruction | | `check_runtime.py` + `runtime.golden` | the built runtime's code region (436 bytes, code ends at 405) sweeps cleanly through FCML, every entry and all branch targets land on an instruction boundary, and the whole disassembly is byte-for-byte the committed golden | whether the golden is *right* | | `check_framedisp.py` | `[BP+off]` uses disp8 iff `off <= 127`, for locals (negative) and far parameters (>127) | which of the two encodings was chosen, if the other also works | | `run_com_exec.py` | the **emitted image executes** and prints exactly the expected bytes | semantics the fixture never exercises | | `rt_exec.py` | the **runtime entries themselves** are called directly, one qemu boot per call, 35 cases plus a pre-flight, 36/36 — plus a `check_bp_contract` pre-flight over the built blob: an entry that borrows BP must open with `55 8B EC` (exact) and contain a `5D` (a screen, because `5D` is also a displacement byte) | whether the *caller's* half of a contract is right, where the only witness is a case that happens to make two calls in a row | Two of these deserve the detail, because the reason they exist is the reason they are hard. **`audit_helpers.py` exists because a wrong ModRM that still decodes is invisible.** The mistake this project actually makes is not a malformed instruction — it is `MovSiBx` emitting `89 DC`, which decodes perfectly as `MOV SP,BX`, and `CmpSiBx` emitting `39 DC`, which decodes as `CMP SP,BX`. Both shipped for a long time. A structural check passes them. A golden passes them. Only asking "does this byte sequence mean what this procedure is called?" fails, which is what the audit does: it disassembles each one-line emitter and compares the decode against the name. Five bugs came out of it in one pass. And then the audit itself turned out to have **three independent ways of silently dropping subjects**, which is worse, because a check that quietly checks nothing looks exactly like a check that passes: 1. It swept `Runtime.mod` only — while `EmXchgAxCx` lives in `Compiler.mod` and was wrong (`93h` = `XCHG AX,BX`) for its entire life, with a correct byte count and a green matrix. The fix sweeps both modules. 2. The procedure-header regex required a *non-empty* parameter list, so it matched **0 of `Compiler.mod`'s 22 emitters**. 3. The coverage list was *derived from the parser's own output*. That is vacuous: any input that stops the parser also deletes the helper from the inventory, so the audit reports "100% covered" for a file it read nothing from. The inventory is now a deliberately dumb `^PROCEDURE\s+(\w+)` scan with no parameter, `BEGIN` or comment awareness, and the audit fails if a documented emitter is missing from it. `EmitData` is the one named exclusion, with the reason recorded in the source. `audit_vx_helpers` had a smaller version of the same disease: it *printed* `len(VX_KINDS)` = "4 Vx subjects" for `Compiler.mod`, which has none. It now reports the number actually checked. **`modrm11.py` is deliberately not self-referential.** The obvious way to check a ModR/M table is to write the table in assembly and assemble it — but that can never fail, because editing the assembly makes `as` faithfully re-encode the new claim and the two then agree again. That failure mode was found by corrupting the `.s` and watching the check stay green. Two things close it: an `EXPECT` byte sequence hard-coded independently of the `.s` text, and four *anchor* encodings that are spelled as literal `.byte` directives because `as` would never choose them for a mnemonic (`83 C4 08` ADD SP,8 · `83 C6 02` ADD SI,2 · `8B EC` MOV BP,SP · `8B E5` MOV SP,BP). No table shifted by one cell can satisfy all four. **`check_framedisp.py` exists because the bug it guards was accidentally correct.** The old code truncated the displacement with `off MOD 100H`, always emitting disp8. Locals are allocated *downward* from `0FFFEh`, so a local's offset is negative and −32768..+127 — the whole range where truncating to a byte happens to be right. The bug was only visible above +127, reachable from the 63rd parameter onward, and no fixture had one. The fix is `disp8` iff `off <= 127` else `disp16`, with no overflow branch at all: disp16 covers the entire 16-bit range as a signed value. The check also asserts the *rule* rather than one encoding — always-disp16 is accepted, and `nonvacuity.sh` proves that by building it and requiring the check to stay green. ### Execution under qemu — `tests/run_com_exec.py` The sixth check is the one that cannot be written as a byte comparison, so it is also the one that finds the most: 21 fixtures are compiled to `.COM`, put on a floppy, booted, and their serial output compared to a committed `.out` file **exactly** — CRLF included — plus the exit code passed to `INT 21h AH=4Ch`. ``` execution: 21 passed, 0 failed (of 21) ``` The two fixtures it found nothing in are the interesting ones: `t31_procparam` and `t32_forexit` were the two most expensive bugs in the project, and neither was visible as a wrong byte count. See `overProc` below. **The `.COM` layout constants are measured, not restated.** `run_com_tests.sh` and `comtest.py` both used to hard-code `RT_SZ = 391` against a runtime that had since grown to 432 bytes at the time, so they read the program header 41 bytes early and reported **30 false failures** — a red suite that meant nothing, which is the most expensive kind of red. Both now locate the header by its own signature (`hdrFlag = 1`, `hdrDS == hdrOff + 1000h + bias`, `hdrHeap > hdrDS`, `hdrCS` leaves room, `initmem` length at `ENT_SZ`) and *derive* `rtSz`, `prologAt` and `dataBase` per file: ``` measured runtime size: 436 bytes (header at image offset 439) ``` A restated constant that has drifted is worse than a derived one, and the two checkers had drifted from each other as well as from the runtime — which is why there are two of them and why both were wrong in the same way. ### Non-vacuity — `tests/nonvacuity.sh` Every assertion in this file is proved able to fail: **28 deliberate breakages, each asserted to turn exactly one named check red for the stated reason, then restored and re-asserted green.** Six break the runtime, five attack the mod=11 table (including restoring the exact wrong table this project once shipped), three target `EmBpDisp` — the truncation, the always-disp16 over-encoding that must *stay* green, and the restored source — six attack the helper audit, and five attack the `.COM` layout checker. Two properties of the harness itself are enforced, because both had already gone wrong silently: - **A baseline assertion runs first**, so a case that is *already* red is reported as `NOT NON-VACUOUS` and distinguished from one that *went* red. Without it, a harness broken by an earlier case would make every later case look like a success. - **Every source mutation goes through `mutate`, which asserts the file actually changed.** Four cases were found to be dead this way: a `sed` that matched nothing, a helper that had been renamed, a helper that had been reformatted, and a checker that correctly stayed green because the breakage it looked for was no longer the breakage the checker hunts. Two of the four (`MovAlDh`, `StBxDl`) were repaired rather than deleted. The one that produced the most information was restoring the original shifted mod=11 table: it turns **three** cells red rather than one, because the error is invisible at code 100 and only visible from 101 down. See below. ## The executor problem (SOLVED — images boot and run) The whole point of a Pascal→8086 compiler is that the output *runs*, and for the first four milestones it did not: **no compiled image and no runtime entry had ever been executed on a CPU**, correct or otherwise. Everything in the checks above was a claim about bytes. Whether the bytes work was the untested part, and it was the only part that could not be closed by writing another checker. **It is now closed, and the milestone is `v-TP3-EXECUTION`.** 21 fixtures compile to `.COM` images, are booted on a floppy by a 512-byte hand-assembled boot sector, and are compared **byte for byte** against the exact output the fixture demands — `tests/run_com_exec.py`, wired into `run_all.sh`, 21/21. Everything below is about establishing what *can* be believed, because the first attempt at this used an emulator that was wrong, and a wrong oracle is worse than none: it cannot distinguish "my codegen is broken" from "the machine is broken". **Unicorn 2.1.4 cannot be used for this.** `UC_MODE_16` mis-decodes 16-bit ModRM memory operands. The measurement, by loading a byte-pattern image so a load reveals its own effective address, then executing a single `LEA AX,[r+disp8]` and reading AX back with every register set to a distinct value: ``` mod=01, disp8=4 got 8086 says 8D 40 04 LEA AX,[BX+4] AX=0d04 0x504 (= BX+SI+4) WRONG 8D 41 04 LEA AX,[BX+SI+4] AX=0e04 0xd04 WRONG 8D 42 04 LEA AX,[BX+DI+4] AX=0f04 0xe04 WRONG 8D 43 04 LEA AX,[BP+4] AX=1004 0x704 WRONG 8D 45 04 LEA AX,[DI+4] AX=0904 0x904 right 8D 46 04 LEA AX,[BP+4] AX=0704 0x704 right 8D 47 04 LEA AX,[DI+4] AX=0504 0x904 WRONG mod=00 / mod=10 direct disp16 8B 1E 00 20 MOV BX,[2000] BX=1234 right 8D 06 34 12 LEA AX,[1234] AX=1234 right ``` So `rm=5` and `rm=6` decode correctly but `rm=0,1,2,3,7` do not, and only *base-register-free* addressing (direct `disp16`) is trustworthy. That rules Unicorn out as an oracle for exactly the instruction forms generated code is made of — `LEA AX,[BP+d]`, `MOV AX,[SI+d]`, `LODSW`-style loops, everything with a frame pointer. It cannot distinguish "my codegen is wrong" from "the emulator is wrong", which makes it worse than no emulator at all. `pip install --upgrade unicorn` resolves to the same 2.1.4, so this is not avoidable by upgrading. **This is no longer the whole picture: qemu-system-i386 is now a trusted oracle, and it was trusted by measurement, not by reputation.** `modrm19.s` assembles with GNU `as`, is wrapped in a 512-byte boot sector, and is booted under `/usr/bin/qemu-system-i386` with the serial port captured to a file. For each of 24 ModR/M encodings it stores a marker through the encoding under test and then *scans memory* for where the word landed, with `BX=1000 DI=2000 SI=0030 BP=0040` so every candidate address is distinct. The answers come out as raw offsets, so this is arithmetic, not a judgement call, and the 23 cells it covers are the project's ground truth for effective addressing. It is also how we know **qemu's 8086 is right where Unicorn's is wrong**, on the same instruction class, by the same method. Two facts about the tooling came out of that work and are worth recording because both were believed wrong at first: - **`objdump -D -b binary -m i8086` disassembles 16-bit code correctly.** An earlier note in this file said there was no usable 16-bit disassembler available; that was wrong, and the cost of believing it was a hand-derived ModR/M table. FCML (`fcml-disasm -m16`) is the other decoder and the two are cross-checked by `tests/fcml_vs_objdump.py`. Note `fcml-disasm` linear-sweeps and aborts (rc=134) on inputs of 16 bytes or more through the Debian wrapper, which is why `tests/disasm16.py` windows input at 15. - **A qemu execution probe for mod=11 is structurally impossible**, not merely awkward. The comparison register is itself a candidate target, and SP is destroyed by the next `call` before any check can run — the first version of that probe pushed its return address through `SS:0xBEEF`, so the evidence it was about to collect had already been overwritten. It also "cleared AX because AX is never an r/m target", which is true of the wrong table and false of the right one. So mod=11 is measured by *encoding* instead, with `as` as an oracle independent of both the runtime and qemu. Also ruled out, for the record: - **DOSBox-X 2025.02.01** (installed, `/usr/bin/dosbox-x`, plain root-owned ELF, not a snap) starts cleanly headless under `SDL_VIDEODRIVER=dummy SDL_AUDIODRIVER=dummy`, but its `-c`/autoexec commands never observably execute: a `md` never appeared on the host, and a `.COM` that creates `OUT.TXT` via `INT 21h AH=3Dh/40h` never produced the file. Shell `>` is intercepted by dosbox-x's own wrapper (`SHELL:Redirect output to out.txt`). A `.BAT` route timed out. The user has since installed **FreeDOS** (`freedos.qcow2`, FD14-LiveCD) which is very likely the answer to this, and untried. ### The boot sector — `tests/exec/bootcom.s` The missing piece is now written. A **512-byte boot sector** assembled with GNU `as` and `objcopy`-ed onto a 1.44 MB floppy image: it loads sector 0, reads the `.COM` off the same floppy with `INT 13h AH=02h` (disk geometry `C0 H2 S18`, 512-byte sectors, so `.COM` byte *n* is at disk offset `512 + n` — **file offset 512 maps to memory `0x100`**), sets `DS=ES=0`, `SS=2000h`, `SP=2004h`, builds the three GDT descriptors at `2000h/2002h/2004h` (code, data, stack) in the way a `.COM` expects, installs an `INT 21h` shim that routes `AH=02h/09h/4Ch/08h` to the serial port at `COM1`, and `JMP 0000:0100`. qemu is invoked as `qemu-system-i386 -fda disk.img -serial out.txt`, and the fixture's expected text is compared to `out.txt` exactly — including CRLF, and including the exit code the program passes to `INT 21h AH=4Ch`. The `INT 21h` shim has **two distinct epilogues for one hook**, which is a fact about the 8086 and not a design choice: `AH=08h` (read with no echo, EOF) must be able to return `CF=1` with `AL=0`, so it exits through a different path (`.Ldone8`) from the `AH=02h/09h/4Ch` case. And the runtime's `INCUR` is an **index into the input buffer**, not a pointer, so the shim's EOF comparison is an index comparison — getting that wrong produces an "EOF at the first character" bug that looks exactly like a broken `readln`. `run_com_exec.py --show` prints the serial file, so a failing fixture can be told apart from a broken harness without re-deriving anything. ### The runtime, entry by entry — `tests/rt_exec.py` `run_com_exec.py` proves *emitted programs* behave. This harness proves the **runtime entries themselves** do, which is a different question: a program never calls `rdint` without a `readln` behind it, never calls `rdbool` twice in a row, and never exercises the `INT16` extremes individually. 35 cases, one fresh qemu boot each, plus a pre-flight that runs before any of them: `initmem` (data poisoned to `AA` first, so a zero is a *result* and not a leftover), `wrint` ×10 including both `INT16` extremes, `wrchar` ×2, `wrbool` ×3, `wrln`, `stackchk`, a composed `writeln(42) writeln TRUE` sequence, `rdint` ×6 against supplied input, `rdchar` ×2, `rdbool` ×6, `rdint` at EOF, and `wrtinl`. Three properties are load-bearing, and each of them was chosen against the easier alternative: - **One boot per case.** A machine that has already run a case has already run a runtime entry, and if that entry corrupts something the next case inherits it. About a tenth of a second a boot buys a machine that has provably never executed anything, and 35 of them cost under four. - **One case record, written twice.** `rt_exec.py` and `exec/rtdrv.s` each state the 40-byte layout, and the cases only pass if the two agree on every offset. A driver that silently read the wrong field would still boot, still run, and still print — so the duplication is the check. - **A BP-contract pre-flight, before any machine starts.** See below. The harness reuses `tests/exec/bootcom.s`, the same 512-byte boot sector `run_com_exec.py` uses, so the boot machinery is written once. #### Two expectations were wrong, and both were wrong in the harness's favour Recording this because the instinct on a red test is to suspect the code, and in both cases the code was right: - `rdchar` stores **one** byte (`StDiDl` = `MOV [DI],DL`), so the old check dumped a two-byte word and compared it against `ord(c)`. It could never pass. The case now reports two bytes and expects the character followed by the `0xEE` poison still sitting in the second. - `rdint` at end of input stores **nothing** — TP3's `xrdint` returns to `rnerr` without touching the variable, and `EmitRdInt`'s own comment says so. The old expectation said the variable was set to zero. `wrtinl` also needed a harness change rather than a machine change: that entry's argument is not a stack word, the caller must place a length byte and the characters at the **return address**, so testing it also tests the encoding contract between `Compiler.IoCall` and `Runtime.EmitWrInl`. #### What execution found that no byte check could **A 10-byte corruption of the BIOS data area.** The boot sector DMA'd the image straight to `0000:0100`, which is what DOS does — and `0400h-04FFh` is the BDA, where SeaBIOS keeps live state. The transfer destroys it, SeaBIOS writes part of it back *after* the DMA, and ten bytes of BIOS data end up on top of the image. The read sets `CF=0` and returns success. The image is the right length in the right place and ten bytes of it are wrong. It was found by a 19-byte probe that read `0440h` as its first instruction after the jump and got the BIOS's own bytes back, and then localised by dumping the whole loaded image against a pattern: one 10-byte run wrong inside an otherwise byte-perfect sector, which no partial-read or sector-count bug can produce. The fix is in `bootcom.s`: read to `8000h` — clear of the IVT, the BDA, SeaBIOS's stack at `700h` and the ROM window at `C000h` — and then `REP MOVSW` down to `0100h`. The copy is executed code, so it is the last thing that touches the image and no BIOS call follows it. **`CmpAl (20)` was decimal twenty.** `rdint`'s lead-in skip is `CmpAl (20H)`, `JBE` — skip everything at or below a space. Written as `20`, Modula-2 read it as decimal and emitted `3C 14`, so a leading space was never skipped and the scan ended with nothing read. Every other magic number in `Runtime.mod` is now written as hex (`MovAh (02H)`, `CmpAl (0DH)`, `CmpAl (1AH)`), because a literal that *reads* like hex but is decimal is silent. **`wrchar` and `wrbool` destroyed BP.** Both borrow BP to reach their argument — `[SP]` cannot be encoded in 16-bit mode, so BP stands in — and neither saved it. BP is the one register an entry may keep, because the driver keeps its cursor into the case record there. So `wrchar` sent the *next* call to a garbage address, the machine triple-faulted, SeaBIOS rebooted, and the run printed the record header twice and hung. Symptom: `record was never closed (EOT)`. **The interesting part is that every existing check called that shape correct.** The bytes were well formed, the size did not change, the golden matched, all branch targets were on instruction boundaries, and the name audit said every helper emitted what its name said. `check_runtime.py`'s entry golden had *blessed* it in five bytes: `"wrchar": "8B EC 8A 46 02 89 EC"`. A positional golden blesses whatever is there. So the rule is now stated, in two places that can disagree: - `check_runtime.py`'s entry goldens carry the `PUSH BP` and the `POP BP`. What the bytes must be. - `rt_exec.py` has a `check_bp_contract` pre-flight over the built blob. What the bytes must *mean*. Its two halves are not equally strong and the code says so: "must start with `55 8B EC`" is exact, and "must contain a `5D`" is a *screen*, because `5D` is also a displacement byte and this code does not disassemble. Requiring the `POP` to be contiguous with anything else is not an improvement — `wrbool` legitimately closes its frame after the `INT 21h`, and a rule insisting on `89 EC 5D` fails a correct entry, which is worse because it teaches a reader to distrust the check. And a third consequence, in the emitters themselves: reaching the argument at `[BP+2]` versus `[BP+4]` is a one-byte difference that reads as a plausible character, and the first version of the fix passed that displacement as a Modula-2 **parameter** — which is invisible to `audit_helpers.py`, because the audit reads the *name*. The emitters are now `MovAlArg2`/`MovAlArg4` and `CmpArg2W0`/`CmpArg4W0`, with the displacement in the name, so the audit pins it: a helper called `CmpArg4W0` that emitted the `+2` form fails the run with *"step 2: displacement is 2, name says 4"*. ### Still missing: `CmdRun` The `R` key of the shell is still unimplemented. TP3's `R` runs a `.COM` *in place*, without the DOS loader, from the same 64 KB of memory; the honest way to do that is a small in-process 8086 interpreter over the image, cross-validated against qemu on the same bytes. ## Components ### Shell — `shell/Shell.mod`, `Term.mod`, `Posix.c`, `TextBuf.mod` Exact TP3.0 screen (`Logged drive`, `Active directory`, `Work file`, `Main file`, `Edit Compile Run Save` / `Dir Quit compiler Options`, `Text: n bytes`, `Free: n bytes`, `>`), command letters drawn bold. Keys `L A W M E C R S D O Q`; any other key redraws the menu, like TP3. `W` auto-`.PAS` with `Loading`/`New File`; `S` writes `^Z` EOF and rotates the old file to `.BAK` (unlink+rename, TP3's order); `D` is a DOS `*.*` glob listing with `k bytes free`; `O` is the options submenu; `Q` confirms and prompts to save when the text changed. `E` and `C` are wired. **`R` is a stub** — it prints "Interpreter pending". ### Editor — `shell/Editor.mod` WordStar-style full-screen editing over `TextBuf`. Status line `Line n Col n Insert/Overwrite Indent X:FILENAME`; text on rows 2-24. Movement (`^S/^D/^E/^X/^A/^F/^R/^C/^W/^Z`, `^Q S/D/E/X/R/C/B/K/P`, arrows, PgUp/PgDn/Home/End), editing (`^V` insert/overtype, `^G` delete char, backspace joins lines, `^T`/`^Y` word/line, `^Q-Y` to EOL, `^N`/CR break, TAB auto-indent to the word start above), block (`^K B/T/H` mark, word, show, `^K C/V/Y` copy/move/delete, `^K R/W` file read/write), search (`^Q-F`, `^Q-A`, `^L` repeat; options B/G/n/U/W, N = no-confirm; `^A` any char, CR LF matches a line break), `^P` literal control char, `^U`/ESC aborts a prompt. `^K-D` returns to the shell with the text still in memory and `changed` reported. Disk format matches TP3: CRLF plus trailing `^Z`, load normalises, save re-expands. Detail: `TP3-EDITOR.md`, `TP3-EDITOR-PSEUDOCODE.md`. ### Compiler — `shell/Compiler.mod` Single-pass Pascal → 8086, following TPSRC6 `turbo` / TPSRC7-10: one pass over the shared `TextBuf` emitting machine code into `cbuf`, a patch list for forward references, TP3-style error reporting (number + relative position), and code/data size accounting. Emitted image is a byte array (mode word, CS/DS, size words, `CALL initmem`, `MOV BP,SP`, generated code). Working subset (v0.5): integer/char/boolean/byte scalars, constants with folding, globals, locals, value parameters, procedures and scalar-result functions, `ARRAY[const..const]` with constant indexing, control flow, `GOTO`/`EXIT`, the standard procedures `WRITE`, `WRITELN`, `READ`, `READLN`, `HALT`, and **inline string literals** as `WRITE`/`WRITELN` arguments. **Standard procedures** are `KBuiltin`, not `KProc`, because they are not called generically. TP3 (TPSRC8 `pwriteln`/`pwrloop`/`prdtyped`) does *not* pass a descriptor to the runtime: it inspects each argument's class and emits a *different call per type*, so formatting is fixed at compile time and the runtime only ever sees a value. `IoCall` mirrors that — one call per argument, then a final call for the line break: ``` writeln(1) MOV AX,1 ; PUSH AX ; CALL 20H ; ADD SP,2 ; CALL 40H writeln('a') MOV AX,'a' ; PUSH AX ; CALL 28H ; ADD SP,2 ; CALL 40H writeln('hi') ; CALL 70H ; 02 'h' 'i' ; CALL 40H readln(x) LEA AX,[0104]; PUSH AX ; CALL 48H ; ADD SP,2 ; CALL 60H ``` (Those `TU_*` names are the compiler's own; the offsets behind them are now **assigned from `Runtime.RT_Entry`** rather than written down, so the placeholder-versus-real distinction is gone. The third line is the inline-literal form, which differs in kind: no value is pushed and the `ADD SP,2` is absent, because the length and the characters *are* the argument.) `READ`/`READLN` push the *address* so the runtime can store (`EmPushVarAddr`: `LEA AX,[BP+off]` via `EmBpDisp` for locals, `8D 06 off` for globals); a non-variable argument is `ETypeErr` (56), as in TP3. Detail: `TP3-COMPILER.md`. ### Runtime library — `shell/Runtime.mod`, `shell/Runtime.def` The 8086 runtime, **assembled byte by byte from Modula-2** — no external assembler and no checked-in binary, so the tree stays self-contained and the entry offsets are *derived* rather than guessed. Each emitter is one instruction with its ModRM byte spelled out in a comment so the encoding can be checked by hand against an 8086 table. This is the same approach `Compiler.mod` already takes (`Ebyte`/`Eword`/`EmCall`). It mirrors the original's own mechanism: TPSRC7 `copyrt` copies the runtime into the front of the code buffer (`SI=DI=0`, `REPZ MOVSB`) and `pc` is then initialised past it (`MOV pc,#$2D7C`). Same shape here, and now actually done: `pc := RT_Size`, `dc := RT_Size + 1000H`, so the image is `[runtime][program header][program code]` and every emitted address is image-absolute. **No relocation pass is needed** — worth having paid for, since a linker that has to walk fixups is a linker that can get them wrong. Current blob: **391 bytes**, 14 entries, 37 branch targets, offsets read back out of the assembled bytes by `tests/check_runtime.py` rather than asserted by hand: | entry | offset | entry | offset | entry | offset | |---|---|---|---|---|---| | `initmem` | 0 | `wrint` | 36 | `rdint` | 167 | | `progend` | 28 | `wrchar` | 97 | `rdchar` | 268 | | `stackchk` | 35 | `wrbool` | 109 | `rdbool` | 289 | | `halt` | 28 | `wrreal` | 132 | `rdln` | 335 | | | | `wrln` | 140 | `wrtinl` | 148 | The `TU_*` constants in `Compiler.mod` are **assigned from `Runtime.RT_Entry` in `Inittur`**, not written down, so a runtime edit that moves an entry cannot leave the compiler calling the old address. `progend` and `halt` deliberately share one address (`XOR AX,AX / MOV AH,4C / INT 21h / RET`): the compiler already zeroes AX before `progend` and discards the `HALT` argument at compile time, so both leave with exit code 0. Conventions, matching `Compiler.IoCall` exactly: | entry | argument | notes | |---|---|---| | `WrInt/WrChar/WrBool/WrReal` | one 16-bit **value** on the stack | caller pops | | `RdInt/RdChar/RdBool` | one **address** on the stack | caller pops | | `WrLn/RdLn/StackChk` | nothing | | | `InitMem` | `AX` = offset of the program header | a *register*, not a stack word | | `ProgEnd/Halt` | nothing | exits, code 0 | | `WrInl` | **nothing** — reads its own text via `POP BX` | see below | `InitMem` receives the program-header offset **in AX** (not on the stack), reads the data base and end out of the header, and zeroes that range, because Pascal leaves globals undefined. `StackChk` is a bare `RET` — range and stack checking aren't compiled in yet, and the call site sits mid-expression, so it must not touch a register. **Five emitter bugs were fixed here in one session, all found by the `audit_helpers.py` name-vs-decode pass described above**, and all of them were invisible to everything else that was already in place: | emitter | emitted | actually was | correct | |---|---|---|---| | `MovSiBx` | `89 DC` | `MOV SP,BX` | `89 DE` | | `CmpSiBx` | `39 DC` | `CMP SP,BX` | `39 DE` | | `MovSiAx` | `8B C0` | `MOV AX,AX` (a no-op) | `8B F0` | | `initmem` zeroing loop | `MovAxDx` | loaded a value it then discarded | `XOR AX,AX` | | `initmem` header read | `+8` | `hdrMax` | `+6` (`hdrHeap`) | The third is the instructive one, because it is the same error pointed the other way. `8B C0` reads as `MOV AX,AX` under the shifted ModR/M table this project shipped, and the fix looked like it should be the byte that table said was SI. It is not: for opcode `8B` the **reg field is the destination**, so `8B F0` (reg=110=SI, r/m=000=AX) is `MOV SI,AX`, which is what the name asks for. Getting the direction backwards nearly caused a correct fix to be reverted. Two label-name plus fixup list: `rel8`, `rel16` and runtime-data addresses are all patched after the blob is placed, so nothing depends on a hand-computed displacement. **It has still never executed.** The encodings are now audited, golden-pinned and non-vacuity-proved, which is a much stronger static claim than "it assembles" — but a static check cannot tell you the code *works*, only that it is what was intended. See the executor section. ### The program image — `shell/Linker.mod` The runtime is copied to the **front** of the code buffer, then the program header, then the program. The header is our own format at `rtSz`: | off | field | | |---|---|---| | +0 | `hdrFlag` | 1, "header present" | | +2 | `hdrCS` | | | +4 | `hdrDS` | | | +6 | `hdrHeap` | = `dc`, the end of the data area | | +8 | `hdrMax` | | | +10… | | max-open-files, input buffer, output buffer words | `InitMem` gets the header offset in AX and reads `+6` for the heap limit; the independent checker in `run_com_tests.sh` restates these offsets as its own constants and asserts `initmem`'s SI displacements equal them, assertion by assertion, rather than asking the compiler where it thinks the header is. `CmdCompile` honours the Destination option: 0 = memory, 1 = `.COM`, 2 = `.CHN` (refused). A `.COM` is named after its source with the extension swapped at the last dot, padded with a zero gap to `max(pc, dc)`; its stack sits at the segment top (`SS = SP = CS:FFFE`), which is where DOS puts it. **Known limit:** the data area starts at a fixed `rtSz + 1000H` (391 + 4096 = 4481), so a program whose code exceeds 4 KiB runs into its own data. Every fixture is at 4491 or 4493 bytes. Documented rather than fixed, because the original has the same fixed-offset behaviour. ### String literals — `writeln('hi')` `writeln('toto')` was the last thing standing between the front end and a hello-world: string literals had no encoding at all, so anything past one character was `ENoLib`. The encoding is not invented — it is the original's. TPSRC8 `pwrinlin` peeks at the character after the literal: if it is `,` or `)` the literal is a *WRITE argument*, not an expression, and it emits (TPSRC10 `estring`) the length byte and the characters **into the code stream** right behind the call: ``` CALL wrtinl ... ``` TPSRC4 `xwrtinl` is what makes that self-delimiting: `POP BX` takes the return address — which *is* the address of the length byte — and the entry ends with `JMP BX`, returning to just past the last character. So the literal needs no terminator, no length table, and **nothing at all in the data segment**. The arithmetic confirms it: `t26` emits `02 68 69` inline and its data size is unchanged from a program with no strings at all. `wrtinl` is 19 bytes at offset 148 (`5B` POP BX · `31 C9` XOR CX,CX · `8A 0F` MOV CL,[BX] · `43` INC BX · `B4 02` MOV AH,2 · `E3 07` JCXZ to the end label · `8A 07` MOV AL,[BX] · `CD 21` · `43` · `E2 F9` LOOP · `FF E3` JMP BX) — hand-checked once, and now also covered by the golden disassembly and the audit. **A string literal is a value in exactly one place: a `WRITE`/`WRITELN` argument.** Everywhere else it is a hard error, and it is enforced in a single place — `LoadAtom` — because assignment, `IF`, `WHILE`, `FOR`, `REPEAT`, `CASE`, array subscripts and every operator all reach their operand through `LoadAtom`, and none of them can use a counted string where a 16-bit word is expected. `ParseFactor` therefore *marks* a literal (`kind = 3`) rather than rejecting it, and `IoCall` handles it before `LoadAtom` is ever reached. The alternative — letting it through and producing a machine word that happens to be a pointer — would be a silently wrong program; `t25` pins the error instead. Literal text has to survive from the scan to `IoCall`, since the parser does not yet know it is writing rather than computing, so it is collected into a pool as it is read: `strPool[0..4095]`, `strOff`/`strLen[0..255]`, `strTop`, `strCnt`, plus `StrNew`/`StrPut`. The pool is reset in `Inittur`, so it is per-compilation. Details that are deliberate, not incidental: - **`''` is a zero-length literal**, reaching the runtime's `JCXZ` path. It used to be the scalar 39, so `writeln('')` printed a quote mark. - **`'don''t'`** — the doubled quote becomes one character; `t24` pins `len 5`. - **A literal of 256 characters or more is `EConstRange` (45), not truncated.** The length is one byte, so 300 characters would go out behind a length of 44 and the runtime would print 44 of them and silently drop the rest. TP3 strings are at most 255 characters, so refusing is the faithful answer. - **A string *variable* is `ENoLib`, not wrong code.** `IoCall` knows the difference and refuses. `EmPushVarAddr`'s local form is fixed now (see bug 4 below), so the blocker is no longer the encoding — it is that there is no `string` type, no length word, no assignment path and no `WrStr` entry. ## Honest limitations - **`CmdRun` — the `R` key — is still a stub.** The compiler's *output* now executes (21 fixtures, exact output, exit codes), but the shell cannot run a `.COM` in place. The linker writes a real `.COM` and the boot sector runs one under qemu; nothing in the host program yet interprets 8086 code. So the user's route to seeing output is "compile, then run under qemu", not "press `R`". TP3's `R` runs in the same 64 KB with no DOS loader, which is why this is an interpreter and not a `system()` call. - **21 of 33 fixtures execute.** 3 do not compile (`t14` and `t25` by design as `ENoLib`, `uierror` deliberately), leaving **9 that compile and are never run**: `t08` const · `t09` if/then/else · `t10` while · `t11` for/to · `t12` repeat/until · `t13` procedure + value param · `t15` label + goto · `t27` five locals · `t28` the 70-parameter declaration. Those are not incidental omissions — they are the *control-flow* fixtures, and `t27`'s five locals are precisely the `[BP+off]` paths this project got wrong twice. They have byte-level checks and no behavioural check at all. Writing nine `.out` files is cheap and is the highest-value step after `rt_exec.py`; the byte counts in `expected.tsv` will then have a behavioural counterpart. - **The runtime's entries are now each called directly, and two of its invariants are stated rather than implied.** 436 bytes, 14 entries, 100 emitter helpers decoded against their own names across both modules, the whole code region golden-pinned, all branch targets on instruction boundaries — and 35 cases that call the entries one at a time under qemu (`rt_exec.py`, 36/36). What is *not* covered is what a direct call cannot see: `wrtinl` is checked, but only from the harness's side of the calling contract, and the nine fixtures that are compiled but never executed (above) are still reached only through the 21 that are. - **The pushback slot's address is a moving target.** It sits at `rtSz + LoadBias + dataAt + D_PUSH`, so every runtime growth moves it, and every address derived from it must be recomputed. It is computed, not hard-coded — but the two `RT_SZ` constants that *were* hard-coded and had drifted (see the execution section) are the precedent for why this one gets stated every time. - **21 fixtures is a small sample of Pascal.** They cover `var`, `const`, `if`, `while`, `for`, `repeat`, `case` over scalars, procedures with value parameters, `goto`/`label`, string literals and `readln`. They do **not** cover nested procedures, recursion, `var` parameters, `with`, records, sets, files, reals, or any type wider than 2 bytes — all still `ENoLib`. A green execution matrix says nothing about those. - **`wrreal` is a deliberate stub.** It writes the literal text `?REAL?` — the string lives in the runtime's own data block at `D_REAL=24`, which is what makes it a real 9-byte routine rather than a trap. Reals are not formatted yet, so `writeln(1.5)` "works" and prints nonsense. A trap would be louder; neither choice is a real answer, and this is now reachable code rather than an unreachable one, which raises the stakes on the choice. - **Code above 4 KiB overruns the data area.** The data base is fixed at `rtSz + 1000H` = 4432 and a `.COM` is padded to `max(pc, dc)`, so the fixed 4 KiB code window is real and not advisory. The largest fixture (`t32_forexit`, 135 bytes of code) is nowhere near it, so nothing has hit this and nothing tests it. `rtSz` also *moves* every time the runtime grows, so the window shrinks silently — another derived value that must never be restated. - **`[SP]` cannot be encoded on an 8086, and the fix is a shape change.** `EmMovAxSp`/`EmMovCxSp` now emit `POP reg` / `PUSH reg` — two instructions where there used to be one, so anything that assumed a one-instruction emitter has to be revisited. The audit handles them as a documented *sequence*; if another one appears, the sequence machinery is where it goes, not a name hack. - **String *literals* work; string *variables* do not.** A literal in a `WRITE`/`WRITELN` argument list is emitted inline and needs no runtime support beyond `wrtinl`. Declaring `s : string`, assigning to it and printing it are all `ENoLib` — there is no `string` type, no length word, no assignment path, no `WrStr` entry. The `chr` flag on `ERes` is what keeps `writeln('a')` calling the *character* writer instead of the integer writer; without it the compiler emitted the integer path and printed 97 while the test still said OK. - **Comma-separated names are not supported.** `var i, c : integer;` is a parse error. Pre-existing, unrelated to any of the above, and still open. - **`readln` of a `BYTE` is a latent 1-byte overflow.** It calls `rdint`, which stores a 2-byte word, so the high byte lands on the next variable. TP3 has a separate `xrdbyte` for exactly this. No fixture declares a `BYTE`, which is the only reason this has never been observed. Write the fixture first. - **The program-header parameter loop is unguarded against non-advancing input.** `program p(1;)` would loop forever. The fix needs a `BOOLEAN` flag and **must not** use `EXIT`, which ICEs gm2 in pass 3 (see the pitfalls list). Not yet done. - **Not implemented** (all `ENoLib`): real, set, record, file, string *variables*, and any type wider than 2 bytes. `with` is `ENoLib`. `case` *is* implemented (cascade `CMP`/`JNZ` per label, per RESUME-TP3.md §3.6) but only over scalar labels — subrange labels and label lists are untested. - gm2 string-literal → `ARRAY OF CHAR` assignment copies the literal *plus a NUL* and leaves the tail untouched, so NUL-terminated tables are safe — this was checked, not assumed. ## Three bugs only a byte-level dump could find None of these produced a compile error, and two produced *passing* tests. All three were found by hex-dumping the emitted image and decoding it by hand — code sizes looked perfectly plausible throughout. 1. **`rel16` off by 2 in every direct CALL/JMP.** `EmCall`/`EmJmpNear`/ `EmJcc` computed the displacement from `pc` at a point where `pc` already pointed past the opcode and *at* the displacement field; x86 measures from the end of the instruction (`pc + 2`). `ResolvePatches`, on the forward-patched path, was already right — which is why forward gotos looked fine and backward ones did not. 2. **`EmMovAxSp` emitted `8B 04` with no SIB byte.** ModRM `04` means "a SIB byte follows", so the `CMP AX,imm16` of the *next* instruction was eaten as that SIB byte, and the intended `MOV AX,[SP]` became `MOV AX,[BP+DI+disp]`. Every `case` label comparison therefore vanished and `case` compiled to a chain of loads from garbage addresses — while the fixture reported OK. Correct encoding is `8B 44 24 00` (`[SP]` cannot use mod=00, that computes `BP+SP`). 3. **The dump tool's own offset column was wrong.** It printed 4 *nibbles* through a helper that formats a **byte**, so every row was labelled 16× too large (`0010` shown as `00000100`). A misleading tool is worse than none — it corrupts any offset arithmetic done from its output. ## Bugs found by actually running code Executing the hand-assembled runtime — even under a broken emulator — and running the emitted images back through the harness paid for itself immediately, because a wrong encoding *executes* rather than failing to assemble. Sixteen so far, none of which a compiler diagnostic would ever have reported. 1. **`B()` silently truncated multi-byte opcodes.** `PROCEDURE B` emits exactly one byte and masks with `MOD 100H`, so `B (8BE4H)` — a two-byte opcode passed as one literal — emitted just `E4`. `MOV BP,SP` was missing from every frame in the runtime, so `BP` stayed 0 and *every* `BP`-relative access read address 4. Found by decoding the hex dump; the byte is gone, not wrong. 2. **`MOV BP,SP` encoded as `8B E4`, which is `MOV SP,SP`** — a no-op. The ModRM byte is `mod·64 + reg·8 + rm`, and I mis-derived it. `Compiler.mod`'s `EmMovBpSp` had it right all along (`8B 0CH`); only the new module was wrong. This is why the encoding is now written as three explicit `B` calls with the arithmetic in a comment rather than as one hex literal. 3. **`InitMem` read its argument from `[SP]`** — the return address. The convention is a register (`AX`), unlike the per-argument I/O entries which do take a stack word. Caught because the data area was never cleared. 4. **`EmPushVarAddr` computed the wrong base register for locals, and truncated the displacement** (pre-existing; **now fixed**). It emitted `8D 46 disp`, which is `LEA AX,[SI+disp8]`, but intended `LEA AX,[BP+disp8]` = `8D 45 disp` — so `read` into a *local* had always addressed the wrong cell, silently. It also masked the offset with `off MOD 100H`, losing displacements above 255. The global form `8D 06 off` (`LEA AX,[disp16]`) was always correct. The fix is `EmBpDisp`, one procedure that owns the choice: disp8 iff `off <= 127`, else disp16, with no overflow branch, because disp16 covers the whole 16-bit range as a signed value and every real offset is either negative (locals, allocated down from `0FFFEh`) or small-positive. It is now shared by `EmLoadVar`, `EmStoreVar` and `EmPushVarAddr` rather than written three times, and guarded by `check_framedisp.py`. The old truncation was *accidentally correct* across −32768..+127, which is the whole range where every real variable lives — so the bug was unreachable from any fixture that existed. `t28` exists to make it reachable. 5. **A string literal was eating the rest of the source.** Every multi-character literal reported its error at *exactly* `Length()` — one past the last character of the buffer — so the editor landed past the final `.` of the program. The scanner loop tested `CurCh # quote`, but its "closing quote detected" branch consumed *two* characters (the content character **and** the quote), so the cursor moved past the quote and the next condition test saw the character *after* the literal, was satisfied, and scanned on to end-of-buffer. The `IF` after the loop that was meant to consume the closing quote was unreachable for any string of two or more characters — which is exactly why `writeln('a')` always worked and `writeln('hi')` never did. Worse than a bad caret: it destroyed the parse, so anything after a literal was consumed as string contents and a genuine later error was misattributed to end-of-file. 6. **Every program lost 6 bytes to an uninitialised flag.** The `FOR` over `IoCall`'s argument list needed a "did this argument push a value" flag, and it was never set, so the first argument's `CALL` and its `ADD SP,2` were skipped. Nothing looked wrong — a slightly smaller image looks *more* plausible, not less. Only `expected.tsv` pinning code sizes caught it. 7. **`writeln('hi')` emitted `02 69 00`** — `i` then NUL. `StrNew` recorded the first character of a literal but did not advance `strTop`, so the first `StrPut` landed on top of the seeded character and overwrote it. `t17_two_str` is what pinned it down: its third emitted character was `e`, the *second* literal's character, which had been written into that slot. The last three were each found by a different means — (5) by noticing that every error position was exactly the buffer length, (6) by `expected.tsv`, (7) by hex-dumping the image — and it is worth being precise about why all three were invisible to a check that only asks "does it compile": (5) still produced a plausible error *number*, (6) a plausible code *size*, and (7) a plausible *character*. Each is precisely the shape of bug a compile-only fixture ships. 8. **The `[BP+off]` displacement was truncated to a byte** (see 4 above). Found by reading `EmLoadVar` and asking what `off MOD 100H` means for a negative local offset — at which point the answer is "correct by accident, and unreachable from any fixture that exists", which is the most expensive kind of wrong. 9–13. **Five runtime emitters were one ModRM byte off** — `MovSiBx` `89 DC`, `CmpSiBx` `39 DC`, `MovSiAx` `8B C0`, `initmem`'s zeroing loop, and `initmem`'s header word. Tabulated with their correct encodings in the runtime section above. All five decoded cleanly, all five passed a structural check, and all five passed a golden disassembly. They were found by the one check that asks a question the bytes can answer on their own: *does this decode to what this procedure is called?* 14. **The ModR/M table in the runtime's own documentation was wrong**, and it had been wrong since the runtime was written. It read `CX DX BX SP BP SI DI BX` — the correct list with `AX` dropped off the front and a duplicate `BX` invented at the end. Every code was therefore one too low except `100`, which lands on `SP` either way, so the error was invisible at exactly the cell anyone would check first. This was a *documentation* bug only: the emitters that followed the wrong table emitted `89 DE`/`39 DE`/ `8B F0`, which are right. The table is now measured, not remembered — see `tests/probe/README.md`, which is the fuller account. 15. **`DataBytes()` returned `dc`, the absolute end of the data area, not a size.** Every program over-reported by 256, and every `expected.tsv` row had been baselined to agree. The field is documented as "emitted data size in bytes", so 4 is right and 260 was wrong. Re-baselining the whole matrix is exactly the move that can turn a red suite green by hiding a bug, so it was done *with* the semantic argument above written into the file, and the 6-byte rows (the fixtures declaring one global) are the ones that carry the claim. 16. **`build_tpshell.sh` was a broken duplicate of the Makefile** — it built `Posix` as if it were Modula-2, and ignored its own link's return code. It was never run, because the Makefile is what everyone runs, and a build script nobody runs is documentation. The specific lesson: when two things must agree, keep one. ### Then the image actually ran, and eleven more appeared The sixteen above were found with byte dumps, structural checks and one broken emulator. Booting the emitted `.COM` under qemu and comparing **exact output** is a different kind of instrument: it catches bugs whose every byte is locally correct and whose only defect is that the program goes somewhere else. Eleven more, and the two worst in the project are here. 17. **`FOR` was emitted as a post-test loop** — the increment sat outside the body and the test came after it, so the body ran once before the first comparison. TP3's own `emitfor` (TPSRC1, `DoEmit`) tests *before* the body and increments *inside* it. Every `for` fixture printed one line too many, and only `run_com_exec.py` could say so: the byte count was unchanged, the control flow was well-formed, and the matrix stayed green. 18. **`EXIT` inside a `for` body jumped to the increment, not to the exit.** `t32_forexit` therefore re-tested the condition and could re-enter the body. Worse, the handler had an `EmAddSp (2)` left over from when it exited a `with`-style scope, which unbalanced the argument-cleanup stack. The patch-list sweep also had to be rewritten from `FOR i := a TO exitCnt - 1` to `WHILE i < exitCnt` — `exitCnt` is a `CARDINAL`, so an *empty* range means `TO 65535`, and a program with no `EXIT` in the loop patched 65532 slots. That is a MODULA-2 idiom trap, not a codegen bug. 19. **The procedure-skip jump was missing, so procedure bodies executed as part of the main body.** The compiler emits a procedure's body *between* the caller's prologue and its own main body, so a jump over it is required. `DeclaresProc ()` now answers "does this program declare any procedure?" by a save/restore-`srcPos` lookahead, and `Compile` emits `EmJmpNear (0)` after the prolog when the answer is TRUE, patching it with `SetPatTgt (overProc, pc)` after `DefPart`. Four fixture code sizes grew by exactly 3 bytes — the `E9` plus its rel16 — and each re-baseline is documented in `expected.tsv` rather than waved through. 20. **And then the jump's patch slot could not be told from "no jump".** `EmJmpNear (target)` returns a **patch slot index**, and **slot 0 is a legitimate slot**. The sentinel was `overProc := 0`, so `IF overProc # 0 THEN SetPatTgt (...)` *skipped the patch* for a program whose procedure-skip jump happened to be the first patch in the image. The jump kept its placeholder target 0, so it landed at image offset 0 — the entry jump — and the program looped forever, printing nothing. The fix is a separate `hasProc : BOOLEAN`, not a magic slot number. `t31_procparam` *hung* rather than failed, which is how it was found: a fixture that never terminates is a louder signal than one that prints the wrong thing — but only if the harness has a timeout, and only if somebody reads a timeout as information. The general lesson is the one this project keeps re-learning: **a plausible placeholder is indistinguishable from a real value.** Slot 0, `0` as "no target", `0` as "no flag set" — each was correct until the first case where the real value was 0. A separate `BOOLEAN` has no collision to have. 21. **`IF MatchKey (tok) AND (tok = TkElse)` consumed the token it was rejecting.** `AND` is not short-circuit in Modula-2, and a lookahead built out of a *matching* primitive is a parser bug, not a lookahead. `MatchKey` advanced past the keyword, so an `else` that should have been left for the enclosing statement was eaten. The shape that works is `PeekKw (tok)` for the question and `DropB (MatchKey (tok))` for the commit — and `IF ` may not be the last thing in a compound, which is a separate ISO rule that this hit too. 22. **A `getbyte` with no pushback slot lost one character per call.** TP3's `getbyte` has a *char pre-read flag*: it reads the next character while looking for digits and then hands it back. The first port had no such slot, so `rdint` consumed the delimiter and the following statement lost its first character. The fix is a one-character pushback slot `D_PUSH` plus `ungetch`, and the slot's *address moves whenever the runtime grows* (`rtSz + LoadBias + dataAt + D_PUSH` — at 436 bytes, image `0x1B4`, memory `0x2B4`, one pad byte before the program header at `0x1B7`). It is computed, never hard-coded: the same class of restated-constant bug as `RT_SZ` above. 23. **`rdint` did not mirror TP3's `xrdint`/`readnum`.** TP3 checks for `^Z` *first*, then skips characters `<= 20h`, then an optional sign, then digits, then **pushes the terminator back**. Ours skipped leading whitespace only and did not push the terminator back, so two `readln`s misbehaved. `MUL r/m16` also writes `DX`, so the running digit is parked in `DI` — an encoding fact that has to be written down, or the next person "simplifies" it back. 24. **`INT 21h AH=02h` was given the character in `DL`.** It takes it in `AL`. Four emitters (`MovAlD`, `MovAlSi`, `MovAlArg`, `MovAlDl`) were wrong together, so every character written was the high half of something else — and the *count* of bytes written was right, which is why a byte-counting check passed. 25. **`EmMovAxSp` / `EmMovCxSp` tried to encode `[SP]`,** which the 8086 has no ModRM form for. They now emit `POP reg` / `PUSH reg` — a two-instruction shape, so the audit treats them as a *sequence* rather than trying to match a mnemonic. 26. **There is no `[BX]` in mode 16.** `mod=00 rm=111` is `[BX+SI]`, not `[BX]`. A store through a pointer was writing to `BX+SI`. Stores now go through `DI` or `SI`, and absolute data addresses use `mod=00 rm=110` = ModRM `1E`. Worth stating as a rule because the *name* `[BX]` appears in half the 8086 documentation ever written. 27. **`EmMoveAxDx` emitted `92h` — which is `XCHG AX,DX`.** Behaviour was identical either way, so nothing was broken; only the *name* lied, and a name that lies is how the next one becomes a silent wrong value. It is now `EmXchgAxDx`. The same pass audited the emitter *vocabulary* and renamed the ambiguous ones so the distinction survives: `MovBxImm`/`MovBxVx` (ADDRESS) vs `LdBxVx` (CONTENTS) vs `StVxBx`; `MovAlBl` (`8A C3`, register) vs `LdAlBx` (`8A 07`, memory); `MovAh0` → `{0xB4}` vs `MovAl0` → `{0xB0}`. ## The bug family, stated once Nine of the twenty-seven are the *same* bug in different clothes: **loading the address where the value was wanted, or picking the register one byte or one letter away from the right one.** `EmPushVarAddr` had the right bytes for the wrong register. `LdAlBx` and `MovAlBl` are one letter apart. `MovAh0` and `MovAl0` are one letter apart and their opcodes differ by `04h`. The response to that is not more checking — it is that the *names* now carry the distinction, so the next one is a compile error instead of a silent wrong value. **Check emitted bytes against intended semantics whenever an emitter is added**, and let the audit do it for one-liners. And a second family, on the harness side rather than the compiler's: **a check that quietly checks nothing looks exactly like a check that passes.** The audit that swept one module, the regex that matched no emitters, the coverage list derived from the parser's own output, the four `nonvacuity` mutations that matched nothing, and two hard-coded `RT_SZ` constants — six instances, all green, all worthless. That is why this file has a non-vacuity section and an independently-scanned inventory at all. ## gm2 / ISO Modula-2 pitfalls hit along the way - **Two-phase link** (above) — a single whole-program pass 3 caps identifiers/errors silently. - `EXIT` inside the program-header parameter `WHILE` **ICEs gm2** in pass 3 (`ExitStatement → PopExit → M2StackWord_PopWord → invalidloc`). Extracting the loop into its own procedure did *not* help. Use a `BOOLEAN` "advanced" flag, never `EXIT`. (`LOOP`+`EXIT` is fine, and is used widely.) - A bare `HALT` aborts under `-fiso` (SIGABRT, exit 134); `HALT (0)` is correct. - `CHAR` is not the ZType: `ch = 09H` must be `ORD (ch) = 09H`. - gm2's ISO `SYSTEM` exports `ORD` but **not** `Ord` — the import is case-sensitive here despite gm2's usual case-insensitivity, so `FROM SYSTEM IMPORT Ord` fails with "unknown symbol" while plain `ORD (c)` compiles. Write `ORD`, never `Ord`. - ISO forbids dropping a function result in a statement → the `DropCh`/ `DropB`/`DropC` discard helpers wrap ~39 call sites. - `AND`/`OR`/`NOT` on 16-bit `CARDINAL` → `BitAnd`/`BitOr`/`BitNot`. - `PROCEDURE f : T` is invalid; the file's convention is `PROCEDURE f () : T`. - ISO will not index a plain string constant as an array (needs an array constructor) — compute hex digits with `CHR` instead. - Foreign modules (`FOR "C"`) must use `ADDRESS`, not Modula-2 pointer types; cast at the call site. `Posix` has no `Posix.mod` (do not try to rebuild it) and no `argv` binding. - termios raw mode: every newline is an explicit `CR LF`. - Foreign modules and `Posix` cannot pass `argv`; the test harness reads fixture paths from **stdin** instead. - `subprocess.run(input=…)` needs **bytes**, not `str`, or it raises inside Python rather than reporting the real error. - `as` always picks opcode `89` for a register-to-register `mov`, so it will never emit `8B EC` for the mnemonic `mov bp,sp`. Test anchors that must assert a specific opcode have to be written as literal `.byte`. - FCML prints immediates with an `h` suffix (`add sp,8h`), and the Debian `fcml-disasm` wrapper aborts with rc=134 on inputs of 16 bytes or more. ## Reference material - `TP3-COMPILER.md` — the compiler: what was changed, the `Skip` bug class, the `rel16` off-by-2, standard procedures, current matrix. - `TP3-EDITOR.md` / `TP3-EDITOR-PSEUDOCODE.md` — the original TP3 editor, from TPSRC5/TPSRC6. - `RESUME-TP3.md` — book-derived reference for the 8086 code TP3 generates per Pascal construct (types, skeletons, arithmetic, IF/CASE/REPEAT/WHILE/ FOR, procedures, parameters, functions, I/O, typed constants, absolutes), plus the `TU_*` runtime entry list. - `shell/tests/probe/README.md` — **read this before touching any emitter.** What each ModR/M artifact establishes, which oracle measures which half of the table, why the mod=11 execution probe is structurally impossible, and the shifted table this project shipped, in full. - `shell/tests/run_all.sh` / `nonvacuity.sh` headers — what runs, in what order, and which breakage is supposed to turn which check red. - `Resources/turbopascal3source/TP3/` — TPSRC1-10, the disassembled original. **This is the ground truth**; when our behaviour and the book disagree, the disassembly wins. - `Resources/coeur-tp-ocr/` — OCR of "Au coeur de Turbo Pascal". ## Next steps 1. **`CmdRun`** as an in-process 8086 interpreter — the `R` menu key, and a fallback executor for environments with no DOS. Validate it against qemu on the *same images*, so the two oracles check each other. Cross-validation is the point: an interpreter that agrees with qemu on 21 fixtures is far more evidence than either alone. 2. **String *variables*** — `s : string`, `s := 'hi'`, `writeln(s)`. The encoding blocker is gone (`EmBpDisp`); what is left is a length word, an assignment path, and a `WrStr` entry (TPSRC4 `xwrtstr`). `IoCall` currently refuses with `ENoLib`. 3. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from RESUME-TP3.md §3.11), range/index checks (`TU_RANGE_CHECK`, `TU_INDEX_CHECK`), typed constants (RESUME-TP3.md §3.14), `array` at its point of use (`t14`), `case` with subrange labels. 4. **`readln` of a `BYTE`** calls `rdint`, which stores 2 bytes and overflows into the next variable. TP3 has a separate `xrdbyte`; a `TU_RdByte` entry is the fix. No fixture exists yet, which is why it has not been done — write the fixture first, so the bug is red before the fix. 5. Make the 4 KiB code window an enforced limit rather than a documented one: report an error when `pc` reaches `dc`, instead of writing over the data. 6. Comma-separated names: `var i, c : integer;`. 7. Harden the program-header parameter loop against non-advancing input (`program p(1;)`) with a `BOOLEAN` flag — **not** `EXIT`, which ICEs gm2. 8. FreeDOS (`freedos.qcow2`, FD14-LiveCD) is still untried. Not needed for any claim above, but it is the only way to get a *real* DOS as a third opinion on the `INT 21h` shim.