Преглед изворни кода

Add the v-TP3-8086-LOWERING milestone summary, and move the tag onto it

The per-milestone `v-TAG.md` file this milestone was missing, written to the
same shape as `v-TP3-DEAD-FIXTURES.md`: what changed, the bugs, the tests, the
scope decisions, tag placement, next steps.

It records the two oracle facts that made this milestone necessary rather than
optional, because they are the parts that are easy to forget and expensive to
rediscover:

  - `qemu-system-i386` has no 8086 model.  Its lowest is 486, where `0F 84' is
    an ordinary JZ, so the execution oracle CANNOT see a 386-only opcode, ever.
  - FCML is this project's INDEPENDENT disassembler and its -m16 mode is a 386,
    so the one tool whose job is to say "this is not a real instruction" was
    architecturally guaranteed to agree with the bug.

All ground-truth line references were re-read from the original before writing
them down, rather than copied from the working notes, and two were corrected by
doing so: the IF/WHILE/REPEAT shape is at TPSRC8:247-252 / 271-276 / 292-297
(the three sites differ in how they save the jump position, not in the shape),
and `flgbool' is TPSRC9:412-424 with the emitted sequence at 418-422.  The
`brnchop' polarity is pinned at TPSRC9:35 (`MOV AL,#$75', JNZ, for a boolean
sitting in AX) against TPSRC9:90 (the comparison table's own opcode, for the
flags-set case), which is exactly the asymmetry that made the first port invert
every conditional.

The tag is MOVED from the work commit to this one, so its tree contains the
milestone's own account of it.  That is the convention the other tags follow and
the one `v-TP3-COM-IMAGE' departs from for reasons recorded in its tag message;
placing a tag on the work commit and writing the summary afterwards is how that
one ended up needing an explanatory tag.  Nothing is pushed and both commits
remain in the tree, so nothing is lost by moving it.

One claim was caught and corrected while writing this file rather than shipped:
it said a guard against a vacuous size cross-check "is now in the script".  That
cross-check was a throwaway command, not a committed script, so the sentence was
rewritten to say so and to note that re-baselining `expected.tsv' is still a
manual edit.  A milestone document asserting a safeguard that does not exist is
the same failure mode as the bug this milestone was about.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Eric Streit пре 6 дана
родитељ
комит
711a1219d6
1 измењених фајлова са 186 додато и 0 уклоњено
  1. 186 0
      v-TP3-8086-LOWERING.md

+ 186 - 0
v-TP3-8086-LOWERING.md

@@ -0,0 +1,186 @@
+v-TP3-8086-LOWERING — Summary
+=============================
+
+Tag: v-TP3-8086-LOWERING
+Commit: (the commit that adds this file)
+
+What changed
+------------
+The emitted code no longer contains an opcode the 8086 lacks.
+
+`EmJcc` emitted `0F 8x rel16` and `EmSetcc` emitted `0F 9x rel8` (SETcc).
+`0F` is a 386-and-later opcode-escape prefix; the 8086 — the machine TP3
+targets, and the machine this compiler exists to emit for — has none. Every
+relational operator and every conditional branch in the compiler went through
+these two emitters, so this was not two call sites but the whole idiom, and the
+output was not wrong code: it was not code.
+
+Replacement, read off the disassembly of the original rather than invented:
+
+    TPSRC8:247-252  if     CALL excond ; MOV AL,brnchop ; MOV AH,#$03 ;
+    TPSRC8:271-276  while  (same shape)
+    TPSRC8:292-297  repeat (same shape, jumping back to the loop head)
+        -> a SHORT Jcc of displacement 3, stepping over a 3-byte EJMP
+
+    TPSRC9:412-424  flgbool, emitting at 418-422
+        CALL ecode ; B $03,$B8,$01,$00   * MOV AX,#0001
+        MOV  AL,brnchop
+        CALL ebyte
+        CALL ecode                       * offset
+        B    $02,$01,$48                 * DEC AX
+        -> AX is 1 if the branch is taken, 0 if it ran
+
+Both are 8086 code and both are shorter than what they replace. The condition
+rides in the opcode's **low nibble** — `70H + cc` carries the identical
+condition `0F 8x` / `0F 9x` did — so all seven `EmJcc` sites and all six
+`EmSetcc` arms go on passing the byte they always passed, unchanged. The flags
+survive, which the `FOR` test needs: it emits `CMP` then `Jcc` with nothing in
+between.
+
+`JccShort` and `JccShortInv` are two named helpers rather than open-coded
+expressions, because an open-coded negation at two call sites is how they end up
+disagreeing. `JccShortInv` spells `n XOR 1` as `n + 1 - 2 * (n MOD 2)`: gm2
+under `-fiso` rejects `BITAND` and `BAND` as syntax errors and rejects arithmetic
+on a `BYTE` operand, so there is no XOR at all and every operand goes through
+`VAL`.
+
+Bugs found and fixed
+--------------------
+32. **`0F 8x` / `0F 9x` throughout — an illegal instruction on the target.**
+    Present since the code generator was written, with everything green.
+
+33. **The first attempt at that port inverted every conditional in every
+    program.** `EmJcc` jumps *to* its target, but the 8086 shape steps *over* an
+    `EJMP`, so writing `7x 03` and falling into the destination runs the two the
+    wrong way round. The polarity is genuinely opposite: TP3's `brnchop` is
+    taken when the condition is TRUE (`TPSRC9:35` sets it to `#$75`, JNZ, "AX is
+    non-zero"; `TPSRC9:90` loads the comparison table's own opcode for the
+    flags-set case), whereas the nibbles these call sites pass are taken when
+    the condition is **FALSE** — `IF`'s `EmJcc (84H)` is `JZ` patched to the
+    `ELSE`, so it must fire when the test failed. `EmSetcc` therefore keeps the
+    nibble as it stands and `EmJcc` negates it.
+
+    `t09_if` printed `pos / nonpos / lt` for a program whose hand-derived
+    output is `nonpos / pos / ge`, and `t12_repeat` looped forever. Only the
+    execution suite noticed — third instance in a row of a fault that passed
+    every byte-level check in this project.
+
+Tests / harness
+---------------
+- `run_all.sh`: 12 checks ALL PASS, up from 11; the new one is
+  `check_8086.py`.
+- `nonvacuity.sh`: 44 ok / 0 failed, up from 39.
+- Compile matrix: 34 passed / 0 failed.
+- Execution: 31 passed / 0 failed.
+- .COM linker: 31 checked / 0 failed.
+- Runtime entries: 36 passed / 0 failed.
+- `check_8086.py`: 26 comparison sites, 22 lowered to a Boolean value, 13 to a
+  branch; all 6 declared comparison conditions and all 4 declared branch
+  conditions exercised; 0 `0F`-prefixed opcodes in the runtime (219 bytes
+  swept) or in swept program code (28 of 31 fixtures, 2257 bytes — the other 3
+  have inline strings, which is printed rather than implied).
+
+`tests/check_8086.py` is new, and its design is mostly a list of what does
+**not** work:
+
+- A **linear sweep of the code region is not a sound oracle.** Inline string
+  literals are emitted into the code stream, so the sweep desynchronises and
+  then aborts on text: `t09_if` decodes 16 real instructions and dies on
+  `FE E9 0D 00`, which is ASCII.
+- A **bare `3D` anchor is unsound.** It matches displacement and immediate bytes
+  as readily as opcodes, and using one produced two false alarms — a `3D` inside
+  a `CALL` displacement in `t11_for`, and one inside a string in `t21_mixed`.
+  The anchor is the three-byte `3D 00 00` or nothing.
+
+So it anchors on **comparison sites** rather than instruction boundaries:
+`3B C1` (`EmCmpAxCx`) and `3D 00 00` (`EmCmpAxi (0)`) are the only sequences that
+can precede a lowering. Branches are *additionally* found by shape (`7x 03 E9`,
+searching for the `E9`), which is what covers the `CASE` arm whose `EmCmpAxi`
+carries a label rather than 0.
+
+"Is this an 8086 shape" is a much weaker question than it looks — a `SETG` where
+a `SETGE` belongs is still a fine shape — so two clauses compare against
+**source**:
+
+- **G** pins `t33_cmpops`'s 13 comparisons against the operators in source
+  order. It catches a `>` / `>=` swap, and the failure message shows the swap
+  (`… Dh Dh Fh Fh Dh` where the source asks `… Fh Fh Dh Dh Fh`).
+- **H** pins, per fixture, the multiset of conditions its branch sites declare,
+  read off the `.pas` sources with the reasoning written beside each row.
+
+**H's need was measured, not anticipated.** Mutation M5 — the polarity inversion
+applied to the IF and CASE sites only — left the check **green** while every
+conditional in every program took the wrong path, because `IF` declares nibble
+4, `CASE` declares 5, and they negate into each other. The whole-suite version
+(M4) was caught only by luck: `FOR` declares `C` and `F`, whose negations `D` and
+`E` are declared for nothing at all. M5's first run coming back green is the
+entire reason H exists.
+
+Five new non-vacuity cases, each run by hand and confirmed red before being
+written down:
+
+    M1  EmJcc back to `0F 8x rel16`                       -> red, 45 problems
+    M2  EmSetcc back to `0F 9x` + MOV AH,0                -> red, 49 problems
+    M3  the `>` / `>=` SETcc arms swapped (the old bug 29) -> red, 2 problems
+    M4  branch polarity inverted everywhere (bug 33)      -> red, 22 problems
+    M5  branch polarity inverted for IF and CASE only      -> red, 12 problems
+
+Two harness faults were found while building them, both of which had produced
+false results before:
+
+- The 8086 group's first draft rebuilt with `make`, which does **not** build
+  `comtest`, so `check_8086.py` linked the stale compiler and M1 came back
+  **VACUOUS**. A helper that exits non-zero makes `if mutate_x; then` false, so
+  the case is silently *skipped* — which looks exactly like a pass. The helpers
+  now check the rebuild and say so.
+- A first cross-check of the size re-baseline reported "no mismatch" because no
+  `.COM` files existed where it looked, so its loop never ran and a vacuous
+  "all clear" came back. It was re-run with an assertion on the image count,
+  and this time reported the real answer. Worth recording because the failure
+  looked exactly like success — but note that cross-check was a throwaway
+  command, **not** a committed script, so the guard does not guard anything
+  permanent. `run_compile_tests.sh` re-baselining is still a manual edit.
+
+Scope decisions and findings
+----------------------------
+- `t33_cmpops` is a new fixture, and it exists partly for clause G: `=`, `<>`
+  and `<=` had **never been used as a comparison anywhere in the suite**, and
+  `>` / `>=` had already been swapped once, so the coverage hole and the bug it
+  let through were the same hole. Its `.out` is hand-derived from the Pascal and
+  was written **before** the machine ran.
+- `expected.tsv` re-baselined for nine control-flow fixtures, every one upward
+  by exactly **+1 per lowered site** (a branch 4 -> 5 bytes, a value 5 -> 6).
+  The site counts were counted out of the linked images and each row written
+  only after its delta matched its own count; the arithmetic is spelled out in
+  the file. Data sizes did not move. Byte-level expectations for those sites now
+  live in `check_8086.py`, behaviour in `run_com_exec.py`; the matrix row only
+  says how big the code is.
+- `check_8086.py` cannot assert on the `CASE` arm's comparison: it is
+  `3D lo hi` with a non-zero label immediate, and no sound anchor can find it.
+  The arm is covered by shape and by `t22_case`'s execution, but the label
+  immediate itself is unchecked.
+- The check executes nothing. It proves the opcodes are 8086 and that
+  conditions are attached to the right constructs; the control flow those bytes
+  produce is still only checked by qemu **on a 486**.
+
+Tag placement
+-------------
+`v-TP3-8086-LOWERING` points at the commit that adds this file, so its tree
+contains this milestone's account of it — the convention every other tag
+follows, and the one `v-TP3-COM-IMAGE` departs from for reasons recorded in its
+own tag message. (The tag was first placed on the work commit and moved once
+this file was written; both commits are in the tree, and nothing is pushed.)
+
+What remains (next steps)
+-------------------------
+1. `CmdRun` (`R` key): in-process 8086 interpreter, cross-validated against
+   qemu on the same images. Also the **only** candidate for a real 8086
+   execution oracle, since qemu cannot be one.
+2. String variables (`s : string`, assignment, `writeln(s)`).
+3. Nested procedures/recursion, `var` parameters (`SEG:OFF`), range/index
+   checks, typed constants, array at use (`t14`), case subrange labels.
+4. `readln` of a `BYTE` (separate `xrdbyte` / `TU_RdByte`).
+5. Enforce the 4 KiB code window (error if `pc` reaches data start).
+6. Harmonise multi-name declarations (both `,` and `;` spellings).
+7. Harden program-header parameter loop against non-advancing input.
+8. FreeDOS as a third opinion (untried).