Make the emitted code 8086 code, and prove it can go red

EmJcc emitted `0F 8x rel16' and EmSetcc emitted `0F 9x rel8' (SETcc).  `0F'
is a 386-and-later opcode-escape prefix and the 8086 -- the machine this
compiler exists to emit for -- has none.  So every relational operator and
every conditional branch in the compiler produced an ILLEGAL INSTRUCTION, not
merely wrong code.  That had been true since the code generator was written,
and the whole repository was green throughout:

  - the compile matrix passed
  - the .COM layout checker passed
  - the runtime golden passed
  - the emitter audit passed
  - 30 fixtures booted in qemu and printed their hand-derived output exactly

Two reasons, and the second is the one worth keeping.  qemu-system-i386 has no
8086 model -- its lowest is 486, where `0F 84' is an ordinary JZ -- so the
execution oracle cannot see this class of fault at all, ever.  And FCML, this
project's INDEPENDENT disassembler, has a 386 for its -m16 mode, so the one
tool whose job is to say "this is not a real instruction" was guaranteed to
agree with the bug.  An independent checker that shares the subject's blind
spot is worse than none: it turns an unknown into a false assurance.

The replacement is TP3's own idiom, read off the disassembly rather than
invented.  TPSRC8 ~246-295 lays IF/WHILE/REPEAT out as a SHORT Jcc of
displacement 3 stepping over a 3-byte EJMP; TPSRC9 ~412-424 (`flgbool') turns a
comparison into a boolean with `MOV AX,#0001 ; <Jcc> +1 ; DEC AX'.  Both are
8086 code and both are shorter than what they replace.  The condition lives in
the opcode's LOW NIBBLE, so `70H + cc' carries the identical condition and all
seven EmJcc sites and all six EmSetcc arms go on passing the byte they always
passed, unchanged.  The flags survive, which the FOR test needs: it emits CMP
and then Jcc with nothing in between.

One thing is deliberately NOT the same as TPSRC8, and it cost a round of
"every conditional is inverted": TP3's brnchop is taken when the condition is
TRUE and steps over the EJMP; the nibbles these call sites pass are taken when
the condition is FALSE (IF's `EmJcc (84H)' is JZ patched to the ELSE).  EmJcc
jumps TO its target, so stepping over an EJMP and falling into the destination
runs the two the wrong way round, and the byte has to be the negation.  That
was bug 33, and only the EXECUTION suite caught it -- third instance in a row
of a fault that passed every byte-level check in the project.

tests/check_8086.py is the check this forced into existence.  Its design is
mostly a list of what does not work:

  - a linear sweep of the code region is not a sound oracle, because inline
    string literals are emitted into the code stream: t09_if decodes 16 real
    instructions and then dies on `FE E9 0D 00', which is ASCII;
  - a bare `3D' anchor is unsound -- it matches displacement and immediate
    bytes as readily as opcodes, and using one produced two false alarms (one
    inside a CALL displacement in t11_for, one inside a string in t21_mixed)
    before the three-byte `3D 00 00' form replaced it.

So it anchors on COMPARISON SITES rather than instruction boundaries: `3B C1'
(EmCmpAxCx) and `3D 00 00' (EmCmpAxi (0)) are the only sequences that can
precede a lowering.  Branches are additionally found by shape (`7x 03 E9',
searching for the E9) so the CASE arm -- whose EmCmpAxi carries a label rather
than 0 -- is covered too.  Where a region sweeps clean, the sweep also asserts
no `0F', and the fraction it reached (28 of 31) is printed rather than implied.

"Is this an 8086 shape" is much weaker than it looks -- a SETG where a SETGE
belongs is still a fine shape -- so two clauses compare against SOURCE:

  G  t33_cmpops' 13 comparisons against the operators in source order, which
     catches a `>'/`>=' swap (the message shows the swap).  That fixture
     exists partly for this: `=', `<>' and `<=' had NEVER been used as a
     comparison anywhere in the suite, and `>'/`>=' had already been swapped
     once, so the coverage hole and the bug it let through were one hole.
  H  each fixture's branch conditions against its .pas.  H's need was
     MEASURED, not anticipated: dropping the polarity inversion for the IF and
     CASE sites only left the check GREEN while every conditional took the
     wrong path, because IF declares nibble 4, CASE declares 5, and they
     negate into each other.  The whole-suite version was caught only by luck
     (FOR declares C and F, whose negations D and E are declared for nothing).

Also in this commit:

  - t33_cmpops, closing the `='/`<>'/`<=' coverage hole.  Its .out is
    hand-derived from the Pascal and was written BEFORE the machine ran.
  - expected.tsv re-baselined for the nine control-flow fixtures, every one
    upward by exactly +1 per lowered site.  The site counts were counted out
    of the linked images and each row written only after its delta matched its
    count; the arithmetic is spelled out in the file.  Data sizes did not move.
  - nonvacuity.sh 39 -> 44 cases, adding M1/M2 (each original defect restored),
    M3 (the >/>= swap), M4 (polarity inverted everywhere) and M5 (polarity
    inverted for IF and CASE only, the one that came back green).

The new non-vacuity helpers check the REBUILD rather than assuming it.  The
8086 group's first draft called `make', which does not build `comtest', so the
stale compiler was measured and M1 was reported as vacuous -- and a helper that
exits non-zero makes `if mutate_x; then' false, so the case is silently SKIPPED,
which looks exactly like a pass.

Verified from a clean `make': run_all.sh 12/12 ALL PASS (compile matrix 34/34,
.COM linker 31/31, execution 31/31, runtime entries 36/36); nonvacuity 44 ok,
0 failed.  Emitted bytes confirmed by disassembly: IF `3D 00 00 75 03 E9 0D
00', value `3B C1 B8 01 00 7F 01 48', FOR `3B C1 7E 03 E9 17 00'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

README.md

@@ -22,8 +22,8 @@ In short, current as of `v-TP3-DEAD-FIXTURES`:
 
 - The shell and the WordStar-style editor are done.
 - The compiler front end and 8086 code generator are done for the language
-  subset the fixtures cover, and the emitted `.COM` images **run**: 30 fixtures
-  boot in `qemu-system-i386` and print exactly their expected bytes, and 35
+  subset the fixtures cover, and the emitted `.COM` images **run**: 31 fixtures
+  boot in `qemu-system-i386` and print exactly their expected bytes, and 36
   further cases call the runtime's own entries directly.
 - **The shell's `R` key is still a stub.** There is no in-process 8086
   interpreter, so nothing runs a `.COM` from inside `tpshell` itself; execution

SUMMARY.md

@@ -23,6 +23,7 @@ manual, not guessed.
 | Execution: a boot sector, qemu, and a claim about behaviour | `v-TP3-EXECUTION` | done, **21/21 fixtures run, exact output** |
 | Execute the nine fixtures that only compiled | `v-TP3-DEAD-FIXTURES` | done, **30/30 run; four operator/scoping bugs found** |
 | Runtime entries under qemu, and the register contract stated | `v-TP3-BP-CONTRACT` | done, **36/36 entry checks; `wrchar`/`wrbool` no longer destroy BP** |
+| 8086-legal conditional branches and `SETcc` | `v-TP3-8086-LOWERING` | done, **the emitted code no longer contains an opcode the 8086 lacks** |
 | `CmdRun` (the `R` key), in-process 8086 interpreter | — | **not started** |
 
 Every row that names a tag has one, and every tag points at a commit on
@@ -219,14 +220,19 @@ by building it and requiring the check to stay green.
 ### Execution under qemu — `tests/run_com_exec.py`
 
 The sixth check is the one that cannot be written as a byte comparison, so it
-is also the one that finds the most: 30 fixtures are compiled to `.COM`, put on
+is also the one that finds the most: 31 fixtures are compiled to `.COM`, put on
 a floppy, booted, and their serial output compared to a committed `.out` file
 **exactly** — CRLF included — plus the exit code passed to `INT 21h AH=4Ch`.
 
 ```
-execution: 30 passed, 0 failed (of 30)
+execution: 31 passed, 0 failed (of 31)
 ```
 
+It also found bug 33 — the branch polarity inverted in *every* conditional in
+*every* program — while the compile matrix and the `.COM` layout check were both
+perfectly happy. That is the third time in a row that a fault passed every
+byte-level check in this file and was caught only here.
+
 The two fixtures it found nothing in are the interesting ones: `t31_procparam`
 and `t32_forexit` were the two most expensive bugs in the project, and neither
 was visible as a wrong byte count. See `overProc` below.
@@ -248,18 +254,83 @@ A restated constant that has drifted is worse than a derived one, and the two
 checkers had drifted from each other as well as from the runtime — which is why
 there are two of them and why both were wrong in the same way.
 
+### 8086 legality — `tests/check_8086.py`
+
+The twelfth and newest check, and the only one that asks a question about the
+*target* rather than about the compiler: **does the 8086 have this instruction
+at all?**
+
+```
+8086 check: 26 comparison sites, 22 lowered to a Boolean value, 13 lowered to a branch
+            value conditions  : = x2  <> x2  < x5  >= x3  <= x2  > x8
+            branch conditions : IF / REPEAT x9  CASE x2  FOR downto x1  FOR to x3
+            runtime: 436 bytes, 219 swept, 0 0F-prefixed
+            program code: 28 of 31 fixtures swept end to end, 2257 bytes
+            t33_cmpops: 13 comparisons matched against their source operators, in order
+            clause H: 8 of 8 fixtures matched the branch conditions read off their source
+```
+
+It exists because bugs 32 and 33 got past everything else, and because **no
+execution oracle can exist for this**: qemu 10.0.11's lowest CPU model is 486,
+so `0F 84` is an ordinary `JZ` there and always was. Nor could the
+disassembler help — FCML's `-m16` mode is a 386, so the project's independent
+checker was guaranteed to agree with the bug.
+
+The design is mostly a list of things that do **not** work:
+
+- **A linear sweep of the code region is not a sound oracle.** Inline string
+  literals are emitted into the code stream, so the sweep desynchronises and
+  then aborts on text — `t09_if` decodes 16 real instructions and dies on
+  `FE E9 0D 00`, which is ASCII.
+- **A bare `3D` anchor is unsound.** It matches displacement and immediate bytes
+  as readily as opcodes, and using one produced **two false alarms**: a `3D`
+  inside a `CALL` displacement in `t11_for`, and one inside a string in
+  `t21_mixed`. The anchor is the three-byte `3D 00 00` or nothing.
+
+So it **anchors on comparison sites instead of instruction boundaries**: `3B C1`
+(`EmCmpAxCx`) and `3D 00 00` (`EmCmpAxi (0)`) are the only sequences that can
+precede a lowering, and all seven `EmJcc` sites and the `EmSetcc` site sit
+immediately after one — which was verified by reading each site, not inferred.
+Branches are *additionally* found by shape (`7x 03 E9`, searching for the
+`E9`), which is what covers the CASE arm whose `EmCmpAxi` carries a label rather
+than 0. Where a region sweeps clean the sweep also asserts no `0F`, and the
+fraction it reached (28 of 31) is **printed rather than implied**.
+
+"Is this an 8086 shape" is a much weaker question than it looks — a `SETG` where
+a `SETGE` belongs is still a fine shape — so two further clauses compare against
+**source**. **G** matches `t33_cmpops`'s 13 comparisons against the operators in
+source order, which catches a `>`/`>=` swap. That fixture exists partly for this:
+`=`, `<>` and `<=` had never been used as a comparison anywhere in the suite, and
+`>`/`>=` had already been swapped once, so the coverage hole and the bug it let
+through were the same hole. **H** pins, per fixture, the conditions its branch
+sites declare, read off the `.pas` sources; its need was *measured* (mutation M5
+came back green before H existed) rather than anticipated.
+
+**What it does not do.** It cannot assert on the CASE arm's label immediate, and
+it never executes anything: it proves the opcodes are 8086 and that conditions
+are attached to the right constructs, but the control flow those bytes produce
+is still only checked by qemu on a 486. An in-process 8086 interpreter is the
+only thing that would close that, which is why `CmdRun` is next.
+
 ### Non-vacuity — `tests/nonvacuity.sh`
 
-Every assertion in this file is proved able to fail: **28 deliberate
+Every assertion in this file is proved able to fail: **44 deliberate
 breakages, each asserted to turn exactly one named check red for the stated
 reason, then restored and re-asserted green.** Six break the runtime, five
 attack the mod=11 table (including restoring the exact wrong table this
 project once shipped), three target `EmBpDisp` — the truncation, the
 always-disp16 over-encoding that must *stay* green, and the restored source —
-six attack the helper audit, and five attack the `.COM` layout checker.
+six attack the helper audit, five attack the `.COM` layout checker, and five
+attack the 8086 lowering.
+
+That last group is the newest and the least optional. Two of its five
+(`M4`, `M5`) invert the branch polarity, which is a *legal* 8086 opcode
+sequence, so no shape-based check can see it and only a source-derived
+expectation can. **`M5` came back green on its first run**, which is why
+clause H of `check_8086.py` exists at all; see bug 33 above.
 
-Two properties of the harness itself are enforced, because both had already
-gone wrong silently:
+Four properties of the harness itself are enforced, because all four had
+already gone wrong silently:
 
 - **A baseline assertion runs first**, so a case that is *already* red is
   reported as `NOT NON-VACUOUS` and distinguished from one that *went* red.
@@ -271,6 +342,23 @@ gone wrong silently:
   been reformatted, and a checker that correctly stayed green because the
   breakage it looked for was no longer the breakage the checker hunts. Two
   of the four (`MovAlDh`, `StBxDl`) were repaired rather than deleted.
+- **The python mutation helpers fail LOUDLY.** A non-zero exit from a helper
+  makes `if mutate_foo; then` false, so the case is silently **skipped** — and
+  a skipped case and a passing case are indistinguishable in the total. This
+  is not hypothetical: the `SETcc` case lost its first run to an apostrophe in
+  an `assert` message that closed a Python string, python died, the compiler
+  was never broken, and the suite still reported 0 failed. Each helper now
+  echoes a `BROKEN CASE` line and increments `fail`.
+- **Each helper counts its targets before replacing.** `assert s != before`
+  only proves the file changed; with two edits it passes if either landed, and
+  with two identical `HideLocals` call sites it would delete the wrong one —
+  still a changed file, still a working compiler, still green for the wrong
+  reason. So `assert s.count(X) == 1` everywhere.
+- **A failed rebuild is a FAILURE, not a skip.** The 8086 group's first draft
+  rebuilt with `make` but ran `check_8086.py`, which links against the `comtest`
+  binary that `make` does not build — so the *stale* compiler was measured and
+  the first mutation was reported as VACUOUS. The helper now checks the build
+  and says so.
 
 The one that produced the most information was restoring the original shifted
 mod=11 table: it turns **three** cells red rather than one, because the error
@@ -803,16 +891,24 @@ Details that are deliberate, not incidental:
   a global `x` is rejected. Pascal allows the shadow and the inner one wins.
   `HideLocals` fixed siblings colliding with each other; this is the
   parent-vs-child direction of the same question and is untouched.
-- **The branch and compare lowering is 386 code on an 8086 target.** `EmJcc`
-  emits `0F 8x rel16` and `EmSetcc` emits `0F 9x` (SETcc). Neither exists on
-  an 8086. TP3 uses `JZ rel8` over a 3-byte `EJMP`, with `excond` materialising
-  the *negated* boolean (`TPSRC8` ~244-300). Every relational operator and every
-  conditional branch in the compiler is affected, so this is not one call site
-  but the whole idiom. **No test can see it**: qemu-i386 defaults to a
-  post-386 CPU, so every image in `run_com_exec.py` runs correctly on hardware
-  that did not exist when TP3 shipped. Catching it needs `-cpu 8086` (untried) or
-  a rewrite to the `excond` idiom plus a short/near branch policy — a milestone
-  with a wide golden blast radius, deliberately not folded into this one.
+- **`qemu-system-i386` cannot execute 8086 code, so the 8086 checks are
+  static and stay static.** The lowest CPU model qemu 10.0.11 offers is 486
+  (`-cpu help` confirms it; there is no 8086 model to ask for). Every image in
+  `run_com_exec.py` therefore runs on hardware that did not exist when TP3
+  shipped, and no amount of fixture work changes that. This is why
+  `tests/check_8086.py` exists and why it had to be written as a byte-level
+  question rather than a behavioural one — and why its clauses G and H compare
+  against *source* rather than against another run of the machine. It remains a
+  weaker instrument than execution: it proves the opcodes are 8086 and that the
+  conditions are attached to the right constructs, but the control flow those
+  bytes produce is still only checked by qemu on a 486.
+- **The 8086 check cannot assert on the CASE arm's comparison.** `CASE`
+  compares against a *label*, so its `EmCmpAxi` is `3D lo hi` with a non-zero
+  immediate, and no sound anchor can find it — a bare `3D` also matches
+  displacement and immediate bytes, which produced two false alarms before the
+  3-byte `3D 00 00` form replaced it. The arm is covered by *shape*
+  (`7x 03 E9`, searching for the `E9`) and by `t22_case`'s execution, but the
+  label immediate itself is unchecked.
 - **The runtime's entries are now each called directly, and two of its
   invariants are stated rather than implied.** 436 bytes, 14 entries, 100
   emitter helpers decoded against their own names across both modules, the
@@ -1160,15 +1256,115 @@ emitter audit.
     `symtab[old].resvar` holds an index and a function's result variable is one
     of the entries being hidden.
 
-The theme is worth stating because it is the same theme as bug 27: **a name that
-does not distinguish two things makes the next mistake invisible.** `*` and `+`
-were both `1`; `>` and `>=` were two hex bytes; two procedures' `a` was one
-symbol. In each case the fix is to make the distinction part of the name or the
-surrounding text, not to fix the value and leave the ambiguity in place.
+### Then the emitted code stopped being 8086 code, and two more appeared
+
+Thirty-two bugs. Bug 32 had been present since the code generator was written
+and every check in the project was blind to it, which is a more interesting
+fault than the previous thirty-one and is worth setting out at length.
+
+32. **Every conditional branch and every comparison in every compiled program
+    was an illegal instruction on the target CPU.** `EmJcc` emitted
+    `0F 8x rel16` (`Jcc` near) and `EmSetcc` emitted `0F 9x rel8` (`SETcc`).
+    `0F` is a 386-and-later opcode-escape prefix. The 8086 — the machine TP3
+    targets, and the machine this compiler exists to emit for — has no such
+    prefix. So the code was not wrong, it was not *code*.
+
+    What makes this worth a section rather than a bullet is that **everything
+    was green while it was true.** The compile matrix passed. The `.COM` layout
+    checker passed. The runtime golden passed. The emitter audit passed. Thirty
+    fixtures booted in qemu and printed exactly their hand-derived expected
+    bytes. Two reasons, and the second is the one to remember:
+
+    - **qemu-system-i386 has no 8086 model.** Its lowest is 486, where
+      `0F 84` is an ordinary `JZ`. So the execution oracle cannot see this class
+      of fault, ever — not with a better fixture, not with a longer run. The
+      suite that had found bugs 28–31 could not find this one by construction.
+    - **FCML is this project's *independent* disassembler, and FCML's 16-bit
+      mode is a 386.** The one tool whose entire job is to say "this is not a
+      real instruction" was architecturally guaranteed to agree with the bug.
+      An independent checker that shares the subject's blind spot is worse than
+      no checker, because it converts an unknown into a false assurance.
+
+    The fix is TP3's own idiom, read off the disassembly of the original rather
+    than invented. `TPSRC8` ~246-295 lays IF/WHILE/REPEAT out as `MOV AL,brnchop
+    ; MOV AH,#$03 ; CALL eword ; PUSH pc ; CALL ejump` — a **short** `Jcc` of
+    displacement 3, stepping over a 3-byte `EJMP`. And `TPSRC9` ~412-424
+    (`flgbool`) lays a comparison-to-Boolean out as `MOV AX,#0001 ; <Jcc> +1 ;
+    DEC AX`. Both are 8086 code, both are shorter than what they replace, and
+    the condition is carried in the opcode's **low nibble**, which is why
+    `70H + cc` reproduces the identical condition and all seven `EmJcc` sites
+    and all six `EmSetcc` arms go on passing the byte they always passed. The
+    flags survive, which the FOR test needs: it emits `CMP` then `Jcc` with
+    nothing in between.
+
+33. **The first attempt at that port inverted every conditional in every
+    program.** `EmJcc` *jumps to* its target, but the 8086 shape *steps over*
+    the `EJMP` — so writing `7x 03` and falling into the destination runs the
+    two the wrong way round. TP3's `brnchop` is the branch taken when the
+    condition is TRUE; the nibbles these call sites pass are the branch taken
+    when the condition is **FALSE** (IF's `EmJcc (84H)` is `JZ` patched to the
+    `ELSE`, so it must fire when the test failed). The two spellings are
+    opposite, and the naive port took the wrong one. `t09_if` printed
+    `pos / nonpos / lt` for a program whose hand-derived output is
+    `nonpos / pos / ge`, and `t12_repeat` looped forever.
+
+    Only the **execution** suite noticed. The compile matrix and the `.COM`
+    layout check were both perfectly happy with every conditional inverted —
+    third instance in a row of a byte-level-clean, behaviour-wrong fault.
+    `JccShortInv` now does the negation (`n XOR 1`, spelled `n + 1 - 2*(n MOD 2)`
+    because gm2 under `-fiso` has no XOR on integers at all), and it is one named
+    function rather than open-coded at two call sites.
+
+Fixing them needed a check that asks a question nothing else was asking:
+**does the target CPU have this opcode at all?** `tests/check_8086.py`.
+
+Its design is mostly about what *cannot* be done. A linear sweep of the code
+region is not a sound oracle here: inline string literals are emitted into the
+code stream, so the sweep desynchronises and then aborts on text — `t09_if`
+decodes 16 real instructions and dies on `FE E9 0D 00`, which is ASCII. So the
+check **anchors on comparison sites instead of boundaries**: `3B C1`
+(`EmCmpAxCx`) and the three-byte `3D 00 00` (`EmCmpAxi (0)`) are the only
+sequences that can precede a lowering, and every one of the seven `EmJcc` sites
+and the one `EmSetcc` site sits immediately after one. A **bare `3D` anchor is
+unsound** — it matches displacement and immediate bytes, and using one produced
+two false alarms (a `3D` inside a `CALL` displacement in `t11_for`, one inside a
+string in `t21_mixed`) before it was replaced by the 3-byte form. Where a
+fixture's region *does* sweep clean, the sweep additionally asserts no `0F`, and
+the fraction it reached is printed (28 of 31) rather than implied. The CASE arm,
+whose `EmCmpAxi` carries a label rather than 0, is found by *shape* (`7x 03 E9`,
+searching for the `E9`) — but the check cannot assert on the label immediate
+itself, and `t22_case`'s execution is what covers that.
+
+The check also carries two clauses about **which condition** each site means,
+because "is this an 8086 shape" is a much weaker question than it looks. A
+`SETG` where a `SETGE` belongs is still a perfectly good shape. Clause G matches
+the 13 comparisons of `t33_cmpops` against the operators **in source order**,
+which is what catches a `>`/`>=` swap (mutation M3, red with the swap visible in
+the message: `… Dh Dh Fh Fh Dh` where the source asks `… Fh Fh Dh Dh Fh`). That
+fixture exists partly for this: `=`, `<>` and `<=` had **never been used as a
+comparison anywhere in the suite**, and `>`/`>=` had already been swapped once,
+so the coverage hole and the bug it let through were the same hole.
+
+Clause H exists because clause H's need was **measured, not anticipated**. As
+first written the check was blind to M5 — the polarity inversion of bug 33
+applied to the IF and CASE sites only. IF declares nibble 4, CASE declares 5,
+they negate into each other, and both are declared, so nothing complained while
+every conditional took the wrong path. The whole-suite version (M4) was caught
+only by luck: FOR declares `C` and `F`, whose negations `D` and `E` are declared
+for nothing at all. So clause H pins, per fixture, the multiset of conditions
+its branch sites declare, **read off the `.pas` sources** and written out with
+the reasoning beside each row — a table measured from the image would agree with
+any behaviour including a wrong one.
+
+Five mutations are now permanent cases in `tests/nonvacuity.sh` (44 ok, 0
+failed, up from 39): M1 and M2 restore each original defect, M3 swaps `>`/`>=`,
+M4 inverts the branch polarity everywhere, M5 inverts it for IF and CASE only.
+M5's first run was the one that came back green, and that is the case's whole
+reason for existing.
 
 ## The bug family, stated once
 
-Nine of the thirty-one are the *same* bug in different clothes: **loading the
+Nine of the thirty-two are the *same* bug in different clothes: **loading the
 address where the value was wanted, or picking the register one byte or one
 letter away from the right one.** `EmPushVarAddr` had the right bytes for the
 wrong register. `LdAlBx` and `MovAlBl` are one letter apart. `MovAh0` and
@@ -1245,43 +1441,34 @@ independently-scanned inventory at all.
 
 ## Next steps
 
-1. **8086 branch and compare lowering.** `EmJcc` emits `0F 8x rel16` and
-   `EmSetcc` emits `0F 9x`; neither instruction exists on an 8086. Every
-   relational operator and every conditional branch depends on them. TP3's
-   idiom is `excond` materialising the negated boolean plus `JZ rel8` over a
-   3-byte `EJMP` (`TPSRC8` ~244-300), so this needs a boolean-materialisation
-   strategy and a short/near branch policy, not two opcode substitutions — and
-   it will move nearly every byte in the golden. **First, try
-   `qemu-system-i386 -cpu 8086`** on the existing 30 images: if that turns this
-   class red, it is a one-line addition to `rt_exec.py` and it makes every
-   later fix provable instead of argued.
-2. **`CmdRun`** as an in-process 8086 interpreter — the `R` menu key, and a
+1. **`CmdRun`** as an in-process 8086 interpreter — the `R` menu key, and a
    fallback executor for environments with no DOS. Validate it against qemu on
    the *same images*, so the two oracles check each other. Cross-validation is
-   the point: an interpreter that agrees with qemu on 30 fixtures is far more
-   evidence than either alone.
-3. **String *variables*** — `s : string`, `s := 'hi'`, `writeln(s)`. The
+   the point: an interpreter that agrees with qemu on 31 fixtures is far more
+   evidence than either alone. It is also the only candidate for an **8086**
+   execution oracle, since qemu cannot be one.
+2. **String *variables*** — `s : string`, `s := 'hi'`, `writeln(s)`. The
    encoding blocker is gone (`EmBpDisp`); what is left is a length word, an
    assignment path, and a `WrStr` entry (TPSRC4 `xwrtstr`). `IoCall` currently
    refuses with `ENoLib`.
-4. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from
+3. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from
    RESUME-TP3.md §3.11), range/index checks (`TU_RANGE_CHECK`,
    `TU_INDEX_CHECK`), typed constants (RESUME-TP3.md §3.14), `array` at its
    point of use (`t14`), `case` with subrange labels.
-5. **`readln` of a `BYTE`** calls `rdint`, which stores 2 bytes and overflows
+4. **`readln` of a `BYTE`** calls `rdint`, which stores 2 bytes and overflows
    into the next variable. TP3 has a separate `xrdbyte`; a `TU_RdByte` entry is
    the fix. No fixture exists yet, which is why it has not been done — write
    the fixture first, so the bug is red before the fix.
-6. Make the 4 KiB code window an enforced limit rather than a documented one:
+5. Make the 4 KiB code window an enforced limit rather than a documented one:
    report an error when `pc` reaches `dc`, instead of writing over the data.
-7. Both spellings of a multi-name declaration. `var i, c : integer;` is
+6. Both spellings of a multi-name declaration. `var i, c : integer;` is
    error 1 at the comma and needs two `var` lines; `procedure f (a : integer;
    b : integer)` is error 1 at the semicolon and needs a comma. Neither is
    wrong Pascal, so a program that compiles under one compiler may not under
    another. A parameter may also not shadow a global (`DupTest` rejects any
    name `Search` finds at any level), which Pascal allows.
-8. Harden the program-header parameter loop against non-advancing input
+7. Harden the program-header parameter loop against non-advancing input
    (`program p(1;)`) with a `BOOLEAN` flag — **not** `EXIT`, which ICEs gm2.
-9. FreeDOS (`freedos.qcow2`, FD14-LiveCD) is still untried. Not needed for any
+8. FreeDOS (`freedos.qcow2`, FD14-LiveCD) is still untried. Not needed for any
    claim above, but it is the only way to get a *real* DOS as a third opinion
    on the `INT 21h` shim.

TP3-COMPILER.md

@@ -137,13 +137,13 @@ byte is a perfectly well-formed instruction that does something else, so no
 amount of inspecting the compiler's intent will catch it.  This harness closes
 that gap the only way it can be closed -- by running the thing.
 
-For each of 30 fixtures it compiles a `.COM`, writes a 1.44 MB floppy with the
+For each of 31 fixtures it compiles a `.COM`, writes a 1.44 MB floppy with the
 image in it, boots `qemu-system-i386 -fda disk.img -serial out.txt`, and
 compares `out.txt` to the committed `tests/fixtures/tNN.out` **exactly** --
 CRLF included -- plus the exit code the program hands to `INT 21h AH=4Ch`.
 
 ```
-cd shell && python3 tests/run_com_exec.py            # all 21
+cd shell && python3 tests/run_com_exec.py            # all 31
 cd shell && python3 tests/run_com_exec.py --show     # print the serial file
 cd shell && python3 tests/run_com_exec.py --rebless  # rewrite the .out files
 ```
@@ -394,8 +394,8 @@ backward ones did not.  All three now use `pc + 2`.
 This was silently harmless while nothing executed the image, which is exactly
 why it survived so long.  Verified on `t10_while`: the `JZ` forward patch lands
 on the instruction after the loop, and the `JMP` back-edge now lands exactly on
-the loop head instead of 2 bytes into it -- and `t10_while` is one of the nine
-fixtures that *still* do not execute, so the claim is byte-level only.
+the loop head instead of 2 bytes into it.  (That claim was byte-level only at
+the time; the fixtures execute now, which is what later found bugs 28-33.)
 
 ## Codegen bug class: a patch slot of 0 is not "no patch"
 
@@ -421,6 +421,75 @@ plausible placeholder is indistinguishable from a real value.**  `0` as "no
 target", `-1` as "unbounded", `0` as "flag not set" are all correct until the
 first real value is 0.  Reach for a `BOOLEAN`; it has no collision to have.
 
+## Codegen bug: every conditional branch and comparison was 386 code
+
+`EmJcc` emitted `0F 8x rel16` and `EmSetcc` emitted `0F 9x rel8` (`SETcc`).
+`0F` is a 386-and-later opcode-escape prefix.  **The 8086 -- the machine TP3
+targets, and the machine this compiler exists to emit for -- has none.**  Every
+relational operator and every conditional branch in the compiler depended on
+these two emitters, so this was not two call sites but the whole idiom, and the
+code was not wrong: it was not *code*.
+
+Everything in this repository was green while that was true -- compile matrix,
+`.COM` layout checker, runtime golden, emitter audit, and 30 fixtures booting in
+qemu and printing exactly their hand-derived expected bytes -- for two reasons
+worth carrying forward:
+
+1. **`qemu-system-i386` has no 8086 model.**  Its lowest is 486, where
+   `0F 84` is an ordinary `JZ`.  The execution oracle cannot see this class of
+   fault, ever, and no better fixture changes that.
+2. **FCML is the project's *independent* disassembler and its `-m16` mode is a
+   386.**  The one tool whose job is to say "this is not a real instruction"
+   was architecturally guaranteed to agree with the bug.
+
+The replacement is TP3's own idiom, read off the disassembly of the original:
+
+```
+TPSRC8 ~246-295     IF / WHILE / REPEAT
+                    MOV AL,brnchop ; MOV AH,#$03 ; CALL eword
+                    PUSH pc ; CALL ejump
+
+TPSRC9 ~412-424     flgbool: a comparison turned into a BOOLEAN
+                    MOV AX,#0001 ; <Jcc> +1 ; DEC AX
+```
+
+Both are 8086 code and both are shorter than what they replace.  The condition
+lives in the opcode's **low nibble** -- `70H + cc` reproduces exactly the
+condition `0F 8x`/`0F 9x` carried -- which is why all seven `EmJcc` sites and
+all six `EmSetcc` arms go on passing the byte they always passed, unchanged.  The
+flags survive, which the `FOR` test needs: it emits `CMP` then `Jcc` with
+nothing in between.
+
+The one thing that is **not** the same as `TPSRC8`, and cost a round of "every
+conditional is inverted": TP3's `brnchop` is the branch taken when the condition
+is TRUE, and TP3 steps over the `EJMP` when it is taken.  Here the call sites
+pass the branch taken when the condition is **FALSE** -- `IF`'s `EmJcc (84H)` is
+`JZ` patched to the `ELSE`, so it must fire when the test failed.  `EmJcc` jumps
+*to* its target, and stepping over an `EJMP` and falling into the destination is
+the wrong way round, so the byte must be the *negation*.  `JccShortInv` does it
+(`n XOR 1`, spelled `n + 1 - 2 * (n MOD 2)` because gm2 under `-fiso` has no XOR
+on integers at all), in one named function rather than open-coded twice.  The
+control flow that comes out is identical to the `0F 8x` form; only which of the
+pair is spelled differs.
+
+`tests/check_8086.py` is the check this forced into existence.  Its design is
+mostly a list of what does **not** work: a linear sweep of the code region
+desynchronises on inline string literals (`t09_if` decodes 16 real instructions
+and dies on ASCII), and a bare `3D` anchor matches displacement and immediate
+bytes as readily as opcodes -- which produced two false alarms before the
+three-byte `3D 00 00` form replaced it.  So it anchors on *comparison sites*
+(`3B C1`, `3D 00 00`), which are the only sequences that can precede a lowering,
+and finds branches additionally by shape (`7x 03 E9`) so the `CASE` arm -- whose
+`EmCmpAxi` carries a label rather than 0 -- is covered too.  Two further clauses
+compare against **source**, because "is this an 8086 shape" is much weaker than
+it looks and a `SETG` where a `SETGE` belongs is still a fine shape: `G` pins
+`t33_cmpops`'s 13 comparisons against the operators in source order (which is
+what catches a `>`/`>=` swap), and `H` pins each fixture's branch conditions
+against its `.pas`.  `H` exists because `H`'s need was **measured**: dropping
+the polarity inversion for the `IF` and `CASE` sites only left the check green
+while every conditional in every program took the wrong path, because `IF`
+declares nibble 4, `CASE` declares 5, and they negate into each other.
+
 ## Codegen: procedure bodies are emitted inside the caller's main body
 
 The compiler emits a procedure's body *between* the caller program's prologue

shell/Compiler.mod

@@ -496,21 +496,79 @@ BEGIN
    RETURN 0
 END EmJmpNear ;
 
-PROCEDURE EmJcc (cc : BYTE ; target : CARDINAL) : CARDINAL ;
-(* 0F 8x rel16 near conditional; target = 0 => forward. *)
-VAR rel, p : CARDINAL ;
+PROCEDURE JccShort (cc : BYTE) : BYTE ;
+(* The 8086 SHORT Jcc opcode for a condition nibble.  70h..7Fh is exactly
+   70h + nibble:  70 JO  71 JNO  72 JB  73 JAE  74 JE  75 JNE  76 JBE  77 JA
+   78 JS  79 JNS  7A JP  7B JNP  7C JL  7D JGE  7E JLE  7F JG.
+   So 70H + cc is the same condition the 386-only `0F 8x rel16' (for a Jcc) or
+   `0F 9x' (for a SETcc) encoded, which is what lets the seven EmJcc sites and
+   the six EmSetcc arms go on passing the low byte they always passed. *)
+VAR n : CARDINAL ;
 BEGIN
-   Ebyte (0FH) ;
-   Ebyte (cc) ;
-   IF target = 0 THEN
-      Eword (0) ;
-      p := nPatch ;
-      AddPatch (pc - 2, 0) ;
-      RETURN p
-   END ;
-   rel := (target + 10000H - (pc + 2)) MOD 10000H ;   (* see EmCall *)
-   Eword (rel) ;
-   RETURN 0
+   n := VAL (CARDINAL, cc) MOD 10H ;
+   RETURN VAL (BYTE, 70H + n)
+END JccShort ;
+
+PROCEDURE JccShortInv (cc : BYTE) : BYTE ;
+(* The same, for a jump that is taken when the condition does NOT hold.
+   EmJcc needs this one and EmSetcc needs the other, and the difference is the
+   whole bug, so it is worth being explicit about where it comes from: the low
+   bit of a Jcc code IS the negation bit.  4/5, C/D, E/F, 2/3, 6/7, A/9, B/8 and
+   0/1 are the eight (condition, its negation) pairs, so negating a condition is
+   `n XOR 1' and nothing more - `JE' and `JNE' are 0x74 and 0x75.
+
+   Gm2 under -fiso has no XOR on integers at all: BITAND and BAND are both
+   syntax errors, and arithmetic on a BYTE operand is rejected too, which is
+   why every operand here goes through VAL.  n + 1 - 2*(n MOD 2) is XOR 1 for a
+   four-bit n and it lives in one named place rather than open-coded, because
+   an open-coded negation at two call sites is how they end up disagreeing. *)
+VAR n : CARDINAL ;
+BEGIN
+   n := VAL (CARDINAL, cc) MOD 10H ;
+   RETURN VAL (BYTE, 70H + n + 1 - 2 * (n MOD 2))
+END JccShortInv ;
+
+PROCEDURE EmJcc (cc : BYTE ; target : CARDINAL) : CARDINAL ;
+(* A conditional branch, 8086 style.  `cc' is the condition nibble, and it is
+   exactly the low byte of the `0F 8x rel16' this used to emit.
+
+   That was a real fault and nothing in the build could see it: `0F' is a
+   386-and-later opcode prefix and the 8086 has none, so EVERY conditional
+   branch in EVERY compiled program was an illegal instruction on the machine
+   TP3 targets.  FCML decoded it happily, because FCML's -m16 mode is 386 --
+   and FCML is this project's independent disassembler, so the one tool that
+   could have objected was the one guaranteed to agree.  qemu-system-i386 has
+   no 8086 model either; its lowest is 486.  So the compile succeeded, the
+   .COM linked, the layout checked, the golden held and all 30 fixtures ran to
+   the right answers, all at once, with the bug in.
+
+   TPSRC8 lays IF, WHILE and REPEAT out as
+
+       MOV AL,brnchop ; MOV AH,#$03 ; CALL eword ; PUSH pc ; CALL ejump
+
+   i.e. a SHORT Jcc of displacement 3, stepping over a 3-byte EJMP.  That is
+   the shape here too, and EmJmpNear already owns the displacement arithmetic
+   and the patch slot, so it is three lines and there is no second copy of
+   that rule.
+
+   The one thing that is NOT the same as TPSRC8, and cost a round of "every
+   conditional is inverted" (t09_if printed pos/nonpos/lt for a program that
+   must print nonpos/pos/ge): TP3's brnchop is the branch taken when the
+   condition is TRUE, and TP3 steps over the EJMP when it is taken.  Here `cc'
+   is the branch taken when the condition is FALSE -- IF's `EmJcc (84H)' is
+   JZ, patched to the ELSE, so it must fire when the test failed.  EmJcc jumps
+   to the target, it does not step over it, so stepping over an EJMP and then
+   falling into the destination is the wrong way round: the byte has to be
+   JccShortInv, not JccShort.  The control flow that comes out is identical to
+   the `0F 8x' form this replaces; only which of the pair is spelled differs.
+
+   The flags survive, and the FOR test needs them to: it emits CMP and then
+   Jcc with nothing in between, so anything that wrote a flag here would
+   break the loop.  Jcc and EJMP both leave the flags alone. *)
+BEGIN
+   Ebyte (JccShortInv (cc)) ;        (* Jcc_s, taken when cc does NOT hold *)
+   Ebyte (03H) ;                     (* rel8: step over the 3-byte EJMP *)
+   RETURN EmJmpNear (target)         (* target = 0 => forward, see above *)
 END EmJcc ;
 
 PROCEDURE ResolvePatches () ;
@@ -678,9 +736,48 @@ BEGIN
 END EmCmpAxi ;
 
 PROCEDURE EmSetcc (cc : BYTE) ;
-BEGIN
-   Ebyte (0FH) ; Ebyte (cc) ; Ebyte (0C0H) ;
-   EmMovAh0 ()
+(* Flags -> a Boolean in AX, on an 8086.  `cc' is the SETcc opcode's low byte
+   (94H = E, 95H = NE, 9CH = L, 9DH = GE, 9EH = LE, 9FH = G), i.e. the same
+   condition nibble EmJcc takes.
+
+   This used to emit `0F cc C0' - SETcc - which is 386-and-later, and then a
+   MOV AH,0.  The 8086 cannot read its flags as a value at all, so there was
+   nothing else to fall back on.
+
+   TPSRC9's flgbool is the fallback, and emits exactly this for exactly this
+   case (CH = 04h, a comparison whose result is wanted as a value rather than
+   as a branch):
+
+       CALL ecode ; B $03,$B8,$01,$00   ->  MOV AX,#0001
+       MOV AL,brnchop ; CALL ebyte     ->  JNZ +1
+       CALL ecode ; B $02,$01,$48      ->  DEC AX
+
+   AX stays 1 because the DEC was stepped over, and becomes 0 because it ran.
+   So the shape is one MOV, one short Jcc whose displacement is the length of
+   the DEC, and the DEC - which is EmJcc's shape with a different displacement,
+   and the reason both are two instructions and a byte.
+
+   The polarity is the opposite of EmJcc's, and deliberately so: here the jump
+   must be taken when the comparison is TRUE, because what is being asked is
+   "is this comparison true", and the nibble the six ParseCmp arms pass is the
+   comparison's own opcode.  So this is JccShort and EmJcc is JccShortInv --
+   see EmJcc for why the difference is there at all.  TP3's flgbool writes JNZ
+   for the same reason; in the one case IT reaches flgbool from, the boolean is
+   sitting in AX rather than in the flags, so JNZ is how it says "AX is
+   non-zero".
+
+   AH comes out 0 for free, which is why the EmMovAh0 this used to end with is
+   gone: 6 bytes here where the old sequence was 5.  The FLAGS do not survive,
+   which the old SETcc did - and nothing reads them.  Every conditional branch
+   in the compiler is preceded by its own CMP (see EmJcc's note on the FOR
+   test), and a comparison's value is consumed either as an AX operand or by
+   the test that follows it; the 30 executed fixtures are what holds that
+   down, not this comment. *)
+BEGIN
+   Ebyte (0B8H) ; Eword (1) ;                           (* MOV AX,#0001 *)
+   Ebyte (JccShort (cc)) ;            (* taken when the comparison HOLDS *)
+   Ebyte (01H) ;                     (* rel8: step over the DEC AX *)
+   Ebyte (48H)                                          (* DEC AX *)
 END EmSetcc ;
 
 PROCEDURE EmIncAx () ;

shell/tests/check_8086.py

@@ -0,0 +1,544 @@
+#!/usr/bin/env python3
+"""check_8086.py -- require that the emitted image contains no opcode the 8086
+lacks, and that what replaced the two illegal ones is TP3's shape.
+
+The bug
+-------
+`EmJcc` emitted `0F 8x rel16' and `EmSetcc` emitted `0F 9x' (SETcc).  Both are
+386-and-later: on an 8086 the byte `0F' is not an opcode prefix at all, so
+every conditional branch and every comparison *value* in every compiled program
+was an illegal instruction on the machine TP3 targets.
+
+Nothing in the build could see it, and each thing that might have failed is
+worth naming:
+
+  * it compiled, because the compiler only ever writes bytes;
+  * FCML decoded it happily, because FCML's -m16 mode is 386 -- and FCML is
+    this project's independent disassembler, so the one tool that could have
+    objected was the one tool guaranteed to agree;
+  * the .COM linked and its layout checked, because `0F 84 lo hi' is a
+    perfectly well-formed 4-byte displacement field;
+  * the runtime golden did not move, because the runtime emits no 0F;
+  * and all 30 fixtures ran to the right answers under qemu-system-i386,
+    whose lowest CPU model is 486.  There is no `-cpu 8086'.
+
+That last one is why this file exists rather than a one-line change to the
+harness.  The gap was invisible to the oracle, not absent.
+
+Why the two regions are treated differently
+------------------------------------------
+The runtime's code region (bytes 0..code-end of the runtime blob) is pure
+code, so it can be swept exhaustively and the sweep must complete.  That is a
+hard guarantee: no 0F-prefixed instruction anywhere in the runtime.
+
+The generated program's code region is NOT pure code -- inline string literals
+are emitted into it, after the code, and a linear sweep desynchronises on them
+and then reports an undefined opcode in the middle of a string.  t09_if is the
+demonstration: the sweep decodes 16 real instructions and then dies at the
+bytes `FE E9 0D 00', which are ASCII text, not code.  So a sweep of the
+program region cannot answer "is this 8086-legal", and a check that believed it
+would be worse than no check.
+
+What is sound instead is to name the SITES rather than the boundaries.  Both
+illegal opcodes were emitted in answer to exactly one thing -- the result of a
+comparison -- and a comparison is always introduced by one of two sequences
+this compiler emits and nothing else emits:
+
+    3B C1      EmCmpAxCx,  CMP AX,CX
+    3D lo hi   EmCmpAxi,   CMP AX,imm16
+
+Every one of the seven EmJcc call sites and the single EmSetcc call site sits
+immediately after one of those, which is checked by reading each site rather
+than assumed (see the site table in Compiler.mod).  So this check asserts:
+
+  A. the runtime's code region sweeps clean and holds no 0F-prefixed opcode;
+  B. every `3B C1` in every fixture's code region is followed by one of exactly
+     two 8086-legal shapes --
+       B8 01 00  7X 01  48      a Boolean VALUE: MOV AX,1 ; Jcc +1 ; DEC AX
+       7X 03  E9                a BRANCH:          Jcc +3 ; EJMP
+     which are EmSetcc and EmJcc respectively;
+  C. every `3D lo hi' is followed by the BRANCH shape, because all five
+     EmCmpAxi sites are IF/WHILE/REPEAT/CASE tests;
+  D. every condition nibble the compiler's two tables declare must appear at
+     least once across the suite.
+
+D is what stops the check being vacuous, and it is why t33_cmpops exists: when
+this was first written, measuring the emitted nibbles showed `=', `<>' and
+`<=' were never used in a comparison anywhere in the suite.  A check that only
+requires "some condition was lowered" would have been satisfied by the three
+that were covered.
+
+The shapes in B and C are hand-derived from the original compiler, not read
+back out of this compiler's output:
+
+  TPSRC8 246-295   IF / WHILE / REPEAT are each
+                     MOV AL,brnchop ; MOV AH,#$03 ; CALL eword
+                     PUSH pc ; CALL ejump
+                   i.e. a SHORT Jcc of displacement 3 stepping over a 3-byte
+                   EJMP.  brnchop is the condition's own opcode, so the short
+                   jump is taken straight to the target.
+
+  TPSRC9 412-424   flgbool turns a comparison's flags into a value with
+                     MOV AX,#0001 ; <Jcc> +1 ; DEC AX
+                   AX stays 1 because the DEC was stepped over.
+
+Both are `short Jcc ; one byte ; something`, which is why the displacement is
+3 in one case and 1 in the other and why both are two instructions and a byte.
+
+Usage: check_8086.py [-v]     (from shell/)
+"""
+
+import collections
+import glob
+import os
+import re
+import subprocess
+import sys
+import tempfile
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+SHELL = os.path.dirname(HERE)
+sys.path.insert(0, HERE)
+import disasm16  # noqa: E402
+
+COMTEST = os.path.join(SHELL, "comtest")
+
+# Declared by Compiler.mod, restated here rather than asked of the code under
+# test, and cross-checked against what the suite emits.  These are the low
+# nibbles of the `0F 9x' SETcc opcodes ParseCmp passes to EmSetcc:
+#   = 94H  <> 95H  < 9CH  > 9FH  >= 9DH  <= 9EH
+# EmSetcc's job is to answer "is this comparison true", so its Jcc is the
+# comparison's OWN opcode and these keys are what appears in the image.
+SETCC_NIBBLES = {0x4: "=", 0x5: "<>", 0xC: "<", 0xD: ">=", 0xE: "<=",
+                 0xF: ">"}
+# ... and of the `0F 8x' Jcc opcodes the seven EmJcc sites pass:
+#   IF 84H  REPEAT 84H  CASE 85H  FOR 8CH (downto) / 8FH (to)
+# EmJcc JUMPS TO the target while these are "taken when the condition is
+# false" (IF's JZ is patched to the ELSE, so it must fire when the test
+# failed), and the Jcc-over-EJMP shape steps over the EJMP when it is TAKEN.
+# Those two things are opposite, so the byte in the image is the negation of
+# the nibble declared here: the Jcc code's low bit IS the negation bit, and
+# negating a condition is `n XOR 1' (JE/JNE are 74h/75h).  Hence the XOR below,
+# and hence clause E is stated on the emitted byte rather than on these keys.
+JCC_NIBBLES = {0x4: "IF / REPEAT", 0x5: "CASE", 0xC: "FOR downto",
+               0xF: "FOR to"}
+
+
+def negated(nib):
+    """The Jcc nibble the image will carry for a site that declares `nib'."""
+    return nib ^ 1
+
+LOAD_BIAS = 0x100        # Runtime.def: a .COM's byte 0 lands at DS:0100h
+CMP_AX_CX = b"\x3b\xc1"  # EmCmpAxCx
+CMP_AX_ZERO = b"\x3d\x00\x00"   # EmCmpAxi (0)
+
+
+def probe_runtime():
+    """(blob, code_end) from the existing rt_exec probe, so this check does
+    not restate Runtime.RT_Size or where the code stops."""
+    out = subprocess.run([sys.executable, os.path.join(HERE, "rt_exec.py"),
+                          "--probe"], capture_output=True, text=True,
+                         cwd=SHELL)
+    if out.returncode != 0:
+        sys.stderr.write(out.stdout + out.stderr)
+        raise SystemExit("FAIL: rt_exec.py --probe failed")
+    size = code_end = None
+    for line in out.stdout.splitlines():
+        if line.endswith("bytes") and size is None:
+            size = int(line.split()[0])
+        if line.startswith("code ends at"):
+            code_end = int(line.split()[3])
+    if size is None or code_end is None:
+        raise SystemExit("FAIL: could not read the runtime size from the probe")
+    # the probe prints a hex dump of the blob; rebuild it from the .COM-free
+    # dump lines so this check needs no second source of the runtime bytes
+    blob = bytearray()
+    for line in out.stdout.splitlines():
+        parts = line.split()
+        # one offset word then 16 two-digit hex bytes
+        if len(parts) == 17 and all(len(p) == 2 for p in parts[1:]):
+            try:
+                blob += bytes(int(p, 16) for p in parts[1:])
+            except ValueError:
+                pass
+    return bytes(blob), code_end, size
+
+
+def sweep(code, base=0):
+    """Linear sweep.  Returns (instructions, offset_it_stopped_at_or_None).
+    An instruction is (offset, opcode_byte, length)."""
+    out = []
+    pc = 0
+    while pc < len(code):
+        text, length = disasm16.decode(code[pc:], base + pc)
+        if length == 0:
+            return out, pc
+        out.append((pc, code[pc], length))
+        pc += length
+    return out, None
+
+
+def find_all(hay, needle, start=0):
+    i = start
+    while True:
+        i = hay.find(needle, i)
+        if i < 0:
+            return
+        yield i
+        i += 1
+
+
+def is_value_shape(nxt):
+    """B8 01 00 7X 01 48 -- EmSetcc: MOV AX,#0001 ; Jcc +1 ; DEC AX"""
+    return (len(nxt) >= 6 and nxt[0] == 0xB8 and nxt[1] == 0x01
+            and nxt[2] == 0x00 and 0x70 <= nxt[3] <= 0x7F
+            and nxt[4] == 0x01 and nxt[5] == 0x48)
+
+
+def is_branch_shape(nxt):
+    """7X 03 E9 -- EmJcc: Jcc +3, stepping over a 3-byte EJMP"""
+    return (len(nxt) >= 3 and 0x70 <= nxt[0] <= 0x7F
+            and nxt[1] == 0x03 and nxt[2] == 0xE9)
+
+
+# Pascal relational operator -> the condition nibble the 8086 short Jcc must
+# carry for that operator to be answered correctly.  `=' is JE (74h), and so
+# on down the 70h..7Fh table.  Restated here, and checked against what
+# Compiler.mod's ParseCmp table passes to EmSetcc, so the two cannot drift.
+OP_NIBBLE = {"=": 0x4, "<>": 0x5, "<": 0xC, "<=": 0xE, ">": 0xF, ">=": 0xD}
+
+# The fixture whose SOURCE ORDER of operators is compared against the order
+# the compiler emitted them in.  This is the clause that catches a swap: the
+# clauses above only ask "is this an 8086 shape", and a shape with the wrong
+# nibble is still a shape.  It has to be a fixture whose every comparison is a
+# value (so every one is an EmSetcc site, in source order) and which uses all
+# six operators -- hence t33_cmpops, which exists partly for this.
+ORDER_FIXTURE = "t33_cmpops"
+RE_WRITELN_OP = re.compile(
+    r"writeln\s*\(\s*\w+\s*(=|<>|<=|>=|<|>)\s*\w+\s*\)")
+
+# H: for each fixture that HAS a branch, the multiset of conditions its branch
+# sites declare, read off the .pas source by hand, with the reading spelled
+# out in BRANCH_WHY so a later reader can check the reasoning rather than
+# trust it.  Declared nibbles, i.e. before EmJcc's inversion, so 4 = IF/WHILE/
+# REPEAT, 5 = CASE, C = FOR downto, F = FOR to.
+BRANCH_SITES = {
+    "t09_if":     [4, 4, 4],     # three `if ... then ... else'
+    "t10_while":  [4, 4],        # two `while ... do'
+    "t11_for":    [15, 12],      # one `for .. to' (F), one `for .. downto' (C)
+    "t12_repeat": [4, 4],        # two `repeat .. until'
+    "t15_label":  [4],           # one `if x < 5 then goto 1'
+    "t22_case":   [5, 5],        # `case x of' with two arms, both fall to end
+    "t30_forloop": [15],         # one `for .. to'
+    "t32_forexit": [15, 4],      # one `for .. to' plus one `if .. exit'
+}
+BRANCH_WHY = {
+    "t09_if":     "three `if' statements",
+    "t10_while":  "two `while' loops",
+    "t11_for":    "a `for .. to' and a `for .. downto'",
+    "t12_repeat": "two `repeat .. until' loops",
+    "t15_label":  "a single `if .. then goto'",
+    "t22_case":   "a `case' with two arms, both falling through to `end'",
+    "t30_forloop": "a single `for .. to'",
+    "t32_forexit": "a `for .. to' and an `if .. exit'",
+}
+
+
+def source_operators(path):
+    """The relational operators of every `writeln (x OP y)' in source order."""
+    with open(path) as fh:
+        text = fh.read()
+    return [m.group(1) for m in RE_WRITELN_OP.finditer(text)]
+
+
+def check_runtime_region(verbose):
+    """A: the runtime's code region is pure code, so this is exhaustive."""
+    blob, code_end, size = probe_runtime()
+    if code_end > len(blob):
+        return ["the probe says code ends at %d but only %d bytes were dumped"
+                % (code_end, len(blob))]
+    instrs, stopped = sweep(blob[:code_end])
+    problems = []
+    if stopped is not None:
+        problems.append("the runtime's code region does not sweep clean: it "
+                        "stops at offset %04X, so this check cannot claim to "
+                        "have looked at everything" % stopped)
+    bad = [(o, b) for (o, b, _) in instrs if b == 0x0F]
+    for o, _ in bad:
+        problems.append("runtime offset %04X is a 0F-prefixed opcode, which "
+                        "does not exist on an 8086" % o)
+    if verbose:
+        print("runtime code region 0..%d: %d instructions swept%s"
+              % (code_end, len(instrs),
+                 "" if stopped is None else ", stopped at %04X" % stopped))
+    return problems, dict(size=size, code_end=code_end,
+                          ninstr=len(instrs), n0f=len(bad))
+
+
+def program_code_region(img, rt_size):
+    """(start, end) of the generated program's code region, both as IMAGE
+    offsets, derived from the image rather than from a restated constant: the
+    entry jump's displacement is the program's own answer for where the code
+    begins, and hdrCS is `pc + LoadBias' with pc the end of the generated code,
+    so hdrCS - LoadBias is where it stops."""
+    hdr = 3 + rt_size
+    if len(img) < hdr + 16:
+        return None
+    start = (3 + int.from_bytes(img[1:3], "little", signed=True)) % 0x10000
+    hdr_cs = int.from_bytes(img[hdr + 2:hdr + 4], "little")
+    end = hdr_cs - LOAD_BIAS
+    if not (start <= end <= len(img)):
+        return None
+    return start, end
+
+
+def main(argv):
+    verbose = "-v" in argv
+    if not os.path.exists(COMTEST):
+        print("FAIL: %s not built; run tests/run_com_tests.sh first" % COMTEST)
+        return 1
+
+    rt_problems, rt_info = check_runtime_region(verbose)
+    problems = list(rt_problems)
+
+    work = tempfile.mkdtemp(prefix="check8086.")
+    try:
+        fixtures = sorted(glob.glob(os.path.join(HERE, "fixtures", "*.pas")))
+        paths = "\n".join(fixtures) + "\n"
+        subprocess.run([COMTEST], input=paths.encode(), cwd=work,
+                       stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+
+        val_nibbles = collections.Counter()   # EmSetcc sites
+        br_nibbles = collections.Counter()   # EmJcc sites
+        ncmp = nval = 0
+        nlinked = 0
+        ordered = 0
+        ordered_cmps = 0
+        pinned = []
+        nbr_anchored = 0
+        swept_fixtures = 0
+        swept_bytes = 0
+        missing = []
+
+        for src in fixtures:
+            name = os.path.basename(src)[:-4]
+            com = os.path.join(work, name + ".COM")
+            if not os.path.exists(com):
+                continue                      # a fixture that errors by design
+            with open(com, "rb") as fh:
+                img = fh.read()
+            region = program_code_region(img, rt_info["size"])
+            if region is None:
+                problems.append("%s: could not locate the code region "
+                                "(entry jump and hdrCS disagree)" % name)
+                continue
+            start, end = region
+            code = img[start:end]
+            nlinked += 1
+
+            # A branch is found three ways -- anchored on the comparison
+            # before it (B, C) and by its own shape (D) -- and the three
+            # overlap, so they all go into one set of offsets and each site
+            # is counted and validated once, at the end.  Counting as we went
+            # reported 28 branch sites when there are 13: the anchored walk
+            # and the shape walk were both adding to the same histogram.
+            branch_offs = set()
+
+            # B: every CMP AX,CX -- the only shape EmCmpAxCx emits, and the
+            # only thing that can precede either lowering.  ParseCmp puts a
+            # comparison VALUE there; the FOR test puts a BRANCH there.
+            fixture_val_nibbles = []
+            for off in find_all(code, CMP_AX_CX):
+                ncmp += 1
+                nxt = code[off + 2:off + 8]
+                if is_value_shape(nxt):
+                    val_nibbles[nxt[3] & 0x0F] += 1
+                    fixture_val_nibbles.append(nxt[3] & 0x0F)
+                    nval += 1
+                elif is_branch_shape(nxt):
+                    branch_offs.add(off + 2)
+                    nbr_anchored += 1
+                else:
+                    problems.append(
+                        "%s+%04X: CMP AX,CX is followed by %s -- neither "
+                        "TP3 shape (MOV AX,1; Jcc +1; DEC AX, or Jcc +3; EJMP)"
+                        % (name, start + off,
+                           " ".join("%02X" % b for b in nxt) or "nothing"))
+
+            # C: every CMP AX,0000 -- what IF, WHILE and REPEAT emit to test
+            # a Boolean.  The three-byte anchor matters: a bare 3D also matches
+            # displacement and immediate bytes, and taking it as an opcode is
+            # what produced two false alarms here (a 3D inside a CALL
+            # displacement in t11_for, and one inside a string in t21_mixed).
+            for off in find_all(code, CMP_AX_ZERO):
+                nxt = code[off + 3:off + 6]
+                if is_branch_shape(nxt):
+                    branch_offs.add(off + 3)
+                    nbr_anchored += 1
+                else:
+                    problems.append(
+                        "%s+%04X: CMP AX,0000 is followed by %s -- the three "
+                        "EmCmpAxi (0) sites are IF/WHILE/REPEAT tests, so "
+                        "each must be Jcc +3; EJMP"
+                        % (name, start + off,
+                           " ".join("%02X" % b for b in nxt) or "nothing"))
+
+            # D: the same branches again, found by their own SHAPE rather than
+            # by the instruction before it.  This is what covers the CASE arm,
+            # whose EmCmpAxi carries a label rather than 0 and which no
+            # comparison anchor can therefore find.  Discovery only -- the
+            # counting and the checking happen once, over branch_offs.
+            for off in find_all(code, b"\xe9"):
+                if off >= 2 and is_branch_shape(code[off - 2:off + 1]):
+                    branch_offs.add(off - 2)
+
+            # E: each branch, counted under the nibble the COMPILER DECLARES
+            # (the emitted one put back through the negation) and required to
+            # be a condition Compiler.mod claims to use.
+            for off in sorted(branch_offs):
+                emitted = code[off] & 0x0F
+                declared = negated(emitted)
+                br_nibbles[declared] += 1
+                if declared not in JCC_NIBBLES:
+                    problems.append(
+                        "%s+%04X: branch emits %Xh, which declares the "
+                        "condition %Xh -- not one of the conditions "
+                        "Compiler.mod declares for a branch (%s)"
+                        % (name, start + off, code[off], declared,
+                           ", ".join("%Xh" % k
+                                     for k in sorted(JCC_NIBBLES))))
+
+            # F: where the region sweeps clean -- no inline strings, so the
+            # whole thing is code -- assert no 0F opcode over it as well.  This
+            # is extra coverage, not the backbone, and how much of the suite
+            # it reached is printed rather than implied.
+            instrs, stopped = sweep(code, LOAD_BIAS + start)
+            if stopped is None:
+                swept_fixtures += 1
+                swept_bytes += len(code)
+                for o, b, _ in instrs:
+                    if b == 0x0F:
+                        problems.append(
+                            "%s+%04X: 0F-prefixed opcode in swept code"
+                            % (name, start + o))
+
+            if verbose:
+                print("%-16s code %d..%d%s"
+                      % (name, start, end,
+                         "" if stopped is None
+                         else "  (sweep stops at +%04X: string data)"
+                              % stopped))
+
+            # H: WHICH branch condition each site means, per fixture.  The
+            # table is read off the .pas sources, not off the image -- that is
+            # the whole point, since a table measured from the image would
+            # agree with any behaviour including a wrong one.
+            #
+            # This closes a hole the mutations above MEASURED rather than
+            # assumed.  Clause E only rejects an emitted nibble that declares
+            # a condition Compiler.mod does not claim for a branch, and the
+            # inversion in EmJcc turns IF's declared 4 into an emitted 5 --
+            # which is CASE's declared nibble, and IS claimed.  So dropping
+            # the inversion for the IF and CASE sites alone left this check
+            # green (mutation M5) while every conditional in every program
+            # took the wrong path.  The FOR sites happen not to be blind that
+            # way, because FOR declares C and F, whose negations D and E are
+            # not declared for anything here -- so a whole-suite inversion is
+            # caught by luck, and a partial one is not.
+            #
+            # It catches M5 because `declared' is computed from the EMITTED
+            # byte by going back through the inversion: an IF that emitted
+            # JccShort instead of JccShortInv reads back as declaring 5.
+            got_br = sorted(negated(code[o] & 0x0F) for o in branch_offs)
+            want_br = sorted(BRANCH_SITES.get(name, got_br))
+            if got_br != want_br:
+                problems.append(
+                    "%s: its %d branch sites declare %s, but reading the "
+                    "source says they are %s (%s)"
+                    % (name, len(got_br),
+                       " ".join("%Xh" % n for n in got_br),
+                       " ".join("%Xh" % n for n in want_br),
+                       BRANCH_WHY.get(name, "not a fixture with branches")))
+            elif name in BRANCH_SITES:
+                pinned.append(name)
+
+            # G: for the one fixture whose operators are known from its
+            # SOURCE, the emitted nibbles must match them IN ORDER.  This is
+            # what catches a swap, which the shape clauses above cannot: a
+            # SETG where a SETGE belongs is still a perfectly good 8086 shape.
+            if name == ORDER_FIXTURE:
+                ops = source_operators(src)
+                want = [OP_NIBBLE[o] for o in ops]
+                if fixture_val_nibbles != want:
+                    problems.append(
+                        "%s: the %d comparisons emitted as %s, but the source "
+                        "asks in order for %s"
+                        % (name, len(fixture_val_nibbles),
+                           " ".join("%Xh" % n for n in fixture_val_nibbles),
+                           " ".join("%s=%Xh" % (o, n)
+                                    for o, n in zip(ops, want))))
+                else:
+                    ordered += 1
+                    ordered_cmps += len(want)
+    finally:
+        subprocess.run(["rm", "-rf", work])
+
+    # D: the declared conditions must all be exercised, or the check above is
+    # only as good as whatever the suite happened to use.
+    for nib, op in sorted(SETCC_NIBBLES.items()):
+        if val_nibbles[nib] == 0:
+            missing.append("`%s' (SETcc %02Xh) is declared by ParseCmp but no "
+                           "fixture uses it as a comparison" % (op, nib | 0x90))
+    for nib, where in sorted(JCC_NIBBLES.items()):
+        if br_nibbles[nib] == 0:
+            missing.append("the %s branch (Jcc nibble %Xh) is declared but no "
+                           "fixture emits it" % (where, nib))
+    problems += missing
+
+    print("8086 check: %d comparison sites, %d lowered to a Boolean value, "
+          "%d lowered to a branch" % (ncmp, nval, nbr_anchored))
+    print("            value conditions  : %s"
+          % "  ".join("%s x%d" % (SETCC_NIBBLES.get(n, "?%X?" % n), c)
+                      for n, c in sorted(val_nibbles.items())))
+    print("            branch conditions : %s"
+          % "  ".join("%s x%d" % (JCC_NIBBLES.get(n, "?%X?" % n), c)
+                      for n, c in sorted(br_nibbles.items())))
+    print("            runtime: %d bytes, %d swept, %d 0F-prefixed"
+          % (rt_info["size"], rt_info["ninstr"], rt_info["n0f"]))
+    print("            program code: %d of %d fixtures swept end to end, "
+          "%d bytes" % (swept_fixtures, nlinked, swept_bytes))
+    print("            %s: %d comparisons matched against their source "
+          "operators, in order" % (ORDER_FIXTURE, ordered_cmps))
+    # H must have been reached for EVERY fixture it names.  A table row for a
+    # fixture that no longer links, or whose region cannot be located, would
+    # otherwise sit there looking like coverage while testing nothing.
+    for name in sorted(set(BRANCH_SITES) - set(pinned)):
+        problems.append("%s: clause H expects %d branch sites, but the "
+                        "fixture contributed none -- the row is not being "
+                        "tested" % (name, len(BRANCH_SITES[name])))
+    print("            clause H: %d of %d fixtures matched the branch "
+          "conditions read off their source"
+          % (len(pinned), len(BRANCH_SITES)))
+
+    if ordered == 0:
+        problems.append("%s: no comparison was matched against its source "
+                        "operator, so a swapped condition would go unnoticed"
+                        % ORDER_FIXTURE)
+    if ncmp == 0:
+        problems.append("no comparison sites were found at all, so nothing "
+                        "above was checked")
+    if problems:
+        print("FAIL: %d problem(s)" % len(problems))
+        for p in problems:
+            print("  - %s" % p)
+        return 1
+    print("PASS: no 0F-prefixed opcode in the runtime or in swept program "
+          "code; every comparison is")
+    print("      lowered to TP3's shape; all %d declared comparison "
+          "conditions and all %d declared branch conditions are exercised"
+          % (len(SETCC_NIBBLES), len(JCC_NIBBLES)))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

shell/tests/fixtures/expected.tsv

@@ -34,6 +34,24 @@
 # TPSRC8 pwrinlin / TPSRC4 xwrtinl), so their rows changed from ERR to OK and
 # their code sizes went up by the literal's own bytes.
 #
+# RE-BASELINED for v-TP3-8086-LOWERING, when conditional branches and SETcc
+# stopped being 386-only.  Nine rows moved, every one of them UPWARD, and every
+# one of them by exactly the number of conditional sites in that fixture:
+#
+#   +1 per site.  A branch was 4 bytes (0F 8x rel16) and is now 5 (7x 03,
+#   then the 3-byte EJMP).  A comparison VALUE was 5 bytes (0F 9x rel8, then
+#   MOV AH,0) and is now 6 (B8 01 00, 7x 01, DEC AX).  So the deltas are
+#   t09 +6 = 3 values + 3 branches, t10 +4 = 2+2, t11 +2 = 0+2,
+#   t12 +4 = 2+2, t15 +2 = 1+1, t22 +2 = 0+2, t30 +1 = 0+1, t32 +3 = 1+2,
+#   t33 +13 = 13 values.
+#
+# That arithmetic was CHECKED, not assumed: the site counts were counted out of
+# the linked images (the same CMP AX,CX and CMP AX,0000 anchors and the same
+# 7x 03 E9 shape that check_8086.py uses) and each row was only written after
+# its delta matched its count.  Data sizes did not move at all.  The bytes
+# themselves are pinned by check_8086.py, and the BEHAVIOUR is pinned by
+# run_com_exec.py -- this row only says how big the code is.
+#
 # RE-BASELINED when the runtime was wired into the code buffer.  Every OK row
 # changed by exactly +3 code and -256 data, and both numbers are correct:
 #
@@ -144,6 +162,17 @@
 #                 while the loop's `done` label emitted it again, dropping four
 #                 bytes off a stack that had two to give.  Exits are now patched
 #                 at `done`, and the handler no longer touches the stack.
+#   t33_cmpops   all six relational operators as VALUES, each with a true case,
+#                 a false case and the equality boundary, plus one signed pair.
+#                 Added while making the 8086 branch/compare lowering checkable:
+#                 measuring which conditions the suite actually emitted showed
+#                 that `=', `<>' and `<=' were never used in a comparison at all,
+#                 because `writeln (a = b)` had no fixture.  Only `<', `>' and
+#                 `>=' had coverage - and `>' vs `>=' had ALREADY been swapped
+#                 once (bug 29), so a hole either side of the operators that
+#                 were covered is the last place to leave one.  m := -1, n := -2
+#                 makes the last line signed: unsigned, -1 > -2 is FALSE, so that
+#                 one line is what tells SETG from a byte compare.
 
 t01_minimal	OK	29	4
 t02_writeln	OK	38	4
@@ -153,20 +182,20 @@ t05_own_line_comment	OK	38	4
 t06_two_args	OK	52	4
 t07_big	OK	102	6
 t08_const	OK	76	7
-t09_if	OK	189	6
-t10_while	OK	147	6
-t11_for	OK	142	6
-t12_repeat	OK	133	6
+t09_if	OK	195	6
+t10_while	OK	151	6
+t11_for	OK	144	6
+t12_repeat	OK	137	6
 t13_proc	OK	126	6
 t14_types	ERR	102	83
-t15_label	OK	88	6
+t15_label	OK	90	6
 t16_str1	OK	42	4
 t17_two_str	OK	45	4
 t18_writeln_bare	OK	32	4
 t19_int1	OK	42	4
 t20_str3	OK	62	4
 t21_mixed	OK	62	4
-t22_case	OK	92	6
+t22_case	OK	94	6
 t23_str_empty	OK	36	4
 t24_str_quote	OK	41	4
 t25_str_as_value	ERR	102	48
@@ -174,7 +203,8 @@ t26_str_mixed_args	OK	58	4
 t27_localvar	OK	256	6
 t28_farparam	OK	153	8
 t29_readln	OK	82	7
-t30_forloop	OK	96	8
+t30_forloop	OK	97	8
 t31_procparam	OK	69	6
-t32_forexit	OK	135	8
+t32_forexit	OK	138	8
+t33_cmpops	OK	404	12
 uierror	ERR	41	331

shell/tests/fixtures/t33_cmpops.out

@@ -0,0 +1,13 @@
+FALSE
+TRUE
+TRUE
+FALSE
+FALSE
+FALSE
+FALSE
+TRUE
+TRUE
+FALSE
+TRUE
+TRUE
+TRUE

shell/tests/fixtures/t33_cmpops.pas

@@ -0,0 +1,25 @@
+program t33;
+var
+  a : integer;
+  b : integer;
+  m : integer;
+  n : integer;
+begin
+  a := 6;
+  b := 5;
+  writeln (a = b);
+  writeln (a = a);
+  writeln (a <> b);
+  writeln (a <> a);
+  writeln (a < b);
+  writeln (a < a);
+  writeln (a <= b);
+  writeln (a <= a);
+  writeln (a > b);
+  writeln (a > a);
+  writeln (a >= b);
+  writeln (a >= a);
+  m := -1;
+  n := -2;
+  writeln (m > n)
+end.

shell/tests/nonvacuity.sh

@@ -34,6 +34,14 @@
 #                      triple-faults on the second call - so nothing short of
 #                      running it, or of stating the register contract
 #                      explicitly, can see it.
+#   check_8086.py    the 8086 opcodes:    catches a conditional branch or a
+#                      SETcc emitted as `0F 8x'/`0F 9x', which are 386-only.
+#                      Nothing else here can: qemu-system-i386's lowest CPU
+#                      model is 486, where both are ordinary instructions, and
+#                      FCML's -m16 mode is a 386 too.  Clause H additionally
+#                      catches the branch polarity being inverted, which is
+#                      still a legal opcode and therefore invisible to every
+#                      shape-based check.
 #   run_com_exec.py   behaviour:            catches a*b that emits an ADD, `>'
 #                      and `>=' swapped, REPEAT..UNTIL that stops after one
 #                      pass, and two procedures whose parameters collide.
@@ -536,7 +544,7 @@ cp "$SAVED_C" Compiler.mod
 
 if rebuild_compiler; then
     if python3 tests/run_com_exec.py >/dev/null 2>&1; then
-        echo "  ok: all 30 executed fixtures pass on the restored compiler"
+        echo "  ok: all 31 executed fixtures pass on the restored compiler"
         pass=$((pass + 1))
     else
         echo "NOT RESTORED: run_com_exec.py is red after restoring Compiler.mod"
@@ -548,6 +556,180 @@ else
     fail=$((fail + 1))
 fi
 
+echo
+echo "== 8086 legality of the conditional lowering (check_8086.py)"
+# This whole section exists because of a fault that every other check in the
+# project was blind to, and being blunt about WHY is the point of listing it.
+#
+# EmJcc and EmSetcc emitted `0F 8x rel16' and `0F 9x rel8'.  `0F' is a
+# 386-and-later opcode prefix; the 8086 has none.  So EVERY conditional branch
+# and EVERY comparison in EVERY compiled program was an illegal instruction on
+# the machine this compiler targets.  And all of the following were green while
+# it was: the compile matrix, the .COM layout checker, the runtime golden, the
+# emitter audit, and 30 fixtures executing under qemu to the right answers.
+#
+# Two reasons, and the second is the one worth keeping:
+#   1. qemu-system-i386 has no 8086 model.  Its lowest is 486, where `0F 84' is
+#      a perfectly ordinary JZ.  So the execution oracle CANNOT see this class
+#      of fault, ever.
+#   2. FCML is this project's INDEPENDENT disassembler, and FCML's -m16 mode is
+#      a 386.  The one tool whose job is to say "this is not a real instruction"
+#      was architecturally guaranteed to agree with the bug.
+#
+# So a check was needed that asks the question nothing else was asking.  The
+# five mutations below are the mutations that check has to catch, and each was
+# run by hand and confirmed red BEFORE this section existed.
+#
+# M4 and M5 are the interesting pair.  M4 is a fault this project actually
+# introduced while fixing the above: EmJcc jumps TO its target, but the 8086
+# shape steps OVER a 3-byte EJMP, so the naive port inverts every conditional
+# in every program.  It was caught by execution, not by any byte check.  M5 is
+# the partial version -- the inversion dropped for the IF and CASE sites only,
+# kept for FOR -- and MEASURING that is what produced clause H, because the
+# check was blind to M5 as first written: IF declares nibble 4, CASE declares
+# 5, they negate into each other, and both are declared, so nothing complained
+# while every conditional in every program took the wrong path.  FOR declares
+# C and F, whose negations D and E are declared for nothing at all, so the
+# whole-suite version was caught by luck.
+#
+# Note these need `rebuild_compiler' only to refresh comtest; check_8086.py
+# relinks the fixtures itself.  Getting that wrong is how this section's first
+# draft reported a mutation as VACUOUS: the build was skipped, a stale comtest
+# ran the GOOD compiler, and the check passed.  A helper that exits non-zero
+# makes `if mutate_x; then' false and the case is silently skipped, which looks
+# exactly like a pass -- so rebuild_compiler is checked, not assumed.
+
+# M1: the original defect, restored verbatim.
+cp "$SAVED_C" Compiler.mod
+if python3 - <<'PYX'
+p = 'Compiler.mod'
+s = open(p).read()
+old = """   Ebyte (JccShortInv (cc)) ;        (* Jcc_s, taken when cc does NOT hold *)
+   Ebyte (03H) ;                     (* rel8: step over the 3-byte EJMP *)
+   RETURN EmJmpNear (target)         (* target = 0 => forward, see above *)"""
+new = """   Ebyte (0FH) ; Ebyte (cc) ; Eword (0) ;
+   IF target = 0 THEN
+      RETURN nPatch
+   END ;
+   RETURN 0"""
+assert s.count(old) == 1, 'EmJcc body found %d times -- update this mutation' % s.count(old)
+open(p, 'w').write(s.replace(old, new))
+PYX
+then
+    if rebuild_compiler; then
+        expect_red "check_8086 catches a 0F 8x branch (the original bug)" \
+            "0F 84" python3 tests/check_8086.py
+    else
+        echo "  FAIL: the compiler would not rebuild with the 0F branch"
+        fail=$((fail + 1))
+    fi
+else
+    echo "  BROKEN CASE: the 0F branch mutation did not apply"
+    fail=$((fail + 1))
+fi
+cp "$SAVED_C" Compiler.mod
+
+# M2: the other original defect, SETcc plus the MOV AH,0 it needed.
+cp "$SAVED_C" Compiler.mod
+if python3 - <<'PYX'
+p = 'Compiler.mod'
+s = open(p).read()
+old = """   Ebyte (0B8H) ; Eword (1) ;                           (* MOV AX,#0001 *)
+   Ebyte (JccShort (cc)) ;            (* taken when the comparison HOLDS *)
+   Ebyte (01H) ;                     (* rel8: step over the DEC AX *)
+   Ebyte (48H)                                          (* DEC AX *)"""
+new = """   Ebyte (0FH) ; Ebyte (cc) ; Ebyte (0C0H) ;
+   EmMovAh0 ()"""
+assert s.count(old) == 1, 'EmSetcc body found %d times -- update this mutation' % s.count(old)
+open(p, 'w').write(s.replace(old, new))
+PYX
+then
+    if rebuild_compiler; then
+        expect_red "check_8086 catches a 0F 9x SETcc (the original bug)" \
+            "0F 9F C0" python3 tests/check_8086.py
+    else
+        echo "  FAIL: the compiler would not rebuild with the 0F SETcc"
+        fail=$((fail + 1))
+    fi
+else
+    echo "  BROKEN CASE: the 0F SETcc mutation did not apply"
+    fail=$((fail + 1))
+fi
+cp "$SAVED_C" Compiler.mod
+
+# M4: the polarity inversion, everywhere.  Note the expected text is clause H's,
+# not a shape complaint: every byte here is a legal 8086 shape, which is the
+# entire reason a separate clause had to be written.
+cp "$SAVED_C" Compiler.mod
+if python3 - <<'PYX'
+p = 'Compiler.mod'
+s = open(p).read()
+old = '   Ebyte (JccShortInv (cc)) ;        (* Jcc_s, taken when cc does NOT hold *)'
+new = '   Ebyte (JccShort (cc)) ;           (* MUTATION: inversion dropped *)'
+assert s.count(old) == 1, 'EmJcc Ebyte found %d times -- update this mutation' % s.count(old)
+open(p, 'w').write(s.replace(old, new))
+PYX
+then
+    if rebuild_compiler; then
+        expect_red "clause H catches the branch polarity inverted everywhere" \
+            "but reading the source says" python3 tests/check_8086.py
+    else
+        echo "  FAIL: the compiler would not rebuild with the inversion dropped"
+        fail=$((fail + 1))
+    fi
+else
+    echo "  BROKEN CASE: the polarity mutation did not apply"
+    fail=$((fail + 1))
+fi
+cp "$SAVED_C" Compiler.mod
+
+# M5: the same inversion dropped for the IF and CASE sites ONLY.  This one is
+# the reason clause H exists: it was run by hand and the check stayed GREEN,
+# and FOR's C/F nibbles were the only reason the whole-suite version (M4)
+# happened to be caught.
+cp "$SAVED_C" Compiler.mod
+if python3 - <<'PYX'
+p = 'Compiler.mod'
+s = open(p).read()
+old = '   Ebyte (JccShortInv (cc)) ;        (* Jcc_s, taken when cc does NOT hold *)'
+new = """   IF (cc = 84H) OR (cc = 85H) THEN
+      Ebyte (JccShort (cc))            (* MUTATION: no inversion here *)
+   ELSE
+      Ebyte (JccShortInv (cc))
+   END ;"""
+assert s.count(old) == 1, 'EmJcc Ebyte found %d times -- update this mutation' % s.count(old)
+open(p, 'w').write(s.replace(old, new))
+PYX
+then
+    if rebuild_compiler; then
+        expect_red "clause H catches the inversion dropped for IF and CASE only" \
+            "t22_case" python3 tests/check_8086.py
+    else
+        echo "  FAIL: the compiler would not rebuild with the partial inversion"
+        fail=$((fail + 1))
+    fi
+else
+    echo "  BROKEN CASE: the partial-inversion mutation did not apply"
+    fail=$((fail + 1))
+fi
+cp "$SAVED_C" Compiler.mod
+
+# And the check on the RESTORED compiler, so a green above cannot come from a
+# mutation that failed to take and left the real defect in place.
+if rebuild_compiler; then
+    if python3 tests/check_8086.py >/dev/null 2>&1; then
+        echo "  ok: check_8086 green on the restored compiler"
+        pass=$((pass + 1))
+    else
+        echo "NOT RESTORED: check_8086.py is red after restoring Compiler.mod"
+        python3 tests/check_8086.py 2>&1 | grep -m3 -- '  - ' | sed 's/^/       /'
+        fail=$((fail + 1))
+    fi
+else
+    echo "NOT RESTORED: the compiler would not rebuild"
+    fail=$((fail + 1))
+fi
+
 echo
 echo "== the emitter-name audit of Compiler.mod (audit_helpers.py)"
 # These need no rebuild: the audit reads the SOURCE, not the built object, so

shell/tests/run_all.sh

@@ -152,6 +152,12 @@ run "EXECUTE under qemu" python3 tests/run_com_exec.py
 # depending on there being such a case.
 run "runtime entries"  python3 tests/rt_exec.py
 run "frame displ"     python3 tests/check_framedisp.py
+# The machine this compiler targets is an 8086, and the byte checks above all
+# ask "is this well formed" -- which `0F 84 lo hi' is.  qemu-system-i386's
+# lowest CPU model is 486, so the execution suite could not object either.
+# This one asks "does the target have this opcode at all", and asserts the
+# lowering that replaced the two illegal ones is the original compiler's shape.
+run "8086 opcodes"   python3 tests/check_8086.py
 run "UI error path"   python3 tests/uitest.py
 run "UI success path" python3 tests/comtest.py