|
|
@@ -0,0 +1,186 @@
|
|
|
+v-TP3-8086-LOWERING — Summary
|
|
|
+=============================
|
|
|
+
|
|
|
+Tag: v-TP3-8086-LOWERING
|
|
|
+Commit: (the commit that adds this file)
|
|
|
+
|
|
|
+What changed
|
|
|
+------------
|
|
|
+The emitted code no longer contains an opcode the 8086 lacks.
|
|
|
+
|
|
|
+`EmJcc` emitted `0F 8x rel16` and `EmSetcc` emitted `0F 9x rel8` (SETcc).
|
|
|
+`0F` is a 386-and-later opcode-escape prefix; the 8086 — the machine TP3
|
|
|
+targets, and the machine this compiler exists to emit for — has none. Every
|
|
|
+relational operator and every conditional branch in the compiler went through
|
|
|
+these two emitters, so this was not two call sites but the whole idiom, and the
|
|
|
+output was not wrong code: it was not code.
|
|
|
+
|
|
|
+Replacement, read off the disassembly of the original rather than invented:
|
|
|
+
|
|
|
+ TPSRC8:247-252 if CALL excond ; MOV AL,brnchop ; MOV AH,#$03 ;
|
|
|
+ TPSRC8:271-276 while (same shape)
|
|
|
+ TPSRC8:292-297 repeat (same shape, jumping back to the loop head)
|
|
|
+ -> a SHORT Jcc of displacement 3, stepping over a 3-byte EJMP
|
|
|
+
|
|
|
+ TPSRC9:412-424 flgbool, emitting at 418-422
|
|
|
+ CALL ecode ; B $03,$B8,$01,$00 * MOV AX,#0001
|
|
|
+ MOV AL,brnchop
|
|
|
+ CALL ebyte
|
|
|
+ CALL ecode * offset
|
|
|
+ B $02,$01,$48 * DEC AX
|
|
|
+ -> AX is 1 if the branch is taken, 0 if it ran
|
|
|
+
|
|
|
+Both are 8086 code and both are shorter than what they replace. The condition
|
|
|
+rides in the opcode's **low nibble** — `70H + cc` carries the identical
|
|
|
+condition `0F 8x` / `0F 9x` did — so all seven `EmJcc` sites and all six
|
|
|
+`EmSetcc` arms go on passing the byte they always passed, unchanged. The flags
|
|
|
+survive, which the `FOR` test needs: it emits `CMP` then `Jcc` with nothing in
|
|
|
+between.
|
|
|
+
|
|
|
+`JccShort` and `JccShortInv` are two named helpers rather than open-coded
|
|
|
+expressions, because an open-coded negation at two call sites is how they end up
|
|
|
+disagreeing. `JccShortInv` spells `n XOR 1` as `n + 1 - 2 * (n MOD 2)`: gm2
|
|
|
+under `-fiso` rejects `BITAND` and `BAND` as syntax errors and rejects arithmetic
|
|
|
+on a `BYTE` operand, so there is no XOR at all and every operand goes through
|
|
|
+`VAL`.
|
|
|
+
|
|
|
+Bugs found and fixed
|
|
|
+--------------------
|
|
|
+32. **`0F 8x` / `0F 9x` throughout — an illegal instruction on the target.**
|
|
|
+ Present since the code generator was written, with everything green.
|
|
|
+
|
|
|
+33. **The first attempt at that port inverted every conditional in every
|
|
|
+ program.** `EmJcc` jumps *to* its target, but the 8086 shape steps *over* an
|
|
|
+ `EJMP`, so writing `7x 03` and falling into the destination runs the two the
|
|
|
+ wrong way round. The polarity is genuinely opposite: TP3's `brnchop` is
|
|
|
+ taken when the condition is TRUE (`TPSRC9:35` sets it to `#$75`, JNZ, "AX is
|
|
|
+ non-zero"; `TPSRC9:90` loads the comparison table's own opcode for the
|
|
|
+ flags-set case), whereas the nibbles these call sites pass are taken when
|
|
|
+ the condition is **FALSE** — `IF`'s `EmJcc (84H)` is `JZ` patched to the
|
|
|
+ `ELSE`, so it must fire when the test failed. `EmSetcc` therefore keeps the
|
|
|
+ nibble as it stands and `EmJcc` negates it.
|
|
|
+
|
|
|
+ `t09_if` printed `pos / nonpos / lt` for a program whose hand-derived
|
|
|
+ output is `nonpos / pos / ge`, and `t12_repeat` looped forever. Only the
|
|
|
+ execution suite noticed — third instance in a row of a fault that passed
|
|
|
+ every byte-level check in this project.
|
|
|
+
|
|
|
+Tests / harness
|
|
|
+---------------
|
|
|
+- `run_all.sh`: 12 checks ALL PASS, up from 11; the new one is
|
|
|
+ `check_8086.py`.
|
|
|
+- `nonvacuity.sh`: 44 ok / 0 failed, up from 39.
|
|
|
+- Compile matrix: 34 passed / 0 failed.
|
|
|
+- Execution: 31 passed / 0 failed.
|
|
|
+- .COM linker: 31 checked / 0 failed.
|
|
|
+- Runtime entries: 36 passed / 0 failed.
|
|
|
+- `check_8086.py`: 26 comparison sites, 22 lowered to a Boolean value, 13 to a
|
|
|
+ branch; all 6 declared comparison conditions and all 4 declared branch
|
|
|
+ conditions exercised; 0 `0F`-prefixed opcodes in the runtime (219 bytes
|
|
|
+ swept) or in swept program code (28 of 31 fixtures, 2257 bytes — the other 3
|
|
|
+ have inline strings, which is printed rather than implied).
|
|
|
+
|
|
|
+`tests/check_8086.py` is new, and its design is mostly a list of what does
|
|
|
+**not** work:
|
|
|
+
|
|
|
+- A **linear sweep of the code region is not a sound oracle.** Inline string
|
|
|
+ literals are emitted into the code stream, so the sweep desynchronises and
|
|
|
+ then aborts on text: `t09_if` decodes 16 real instructions and dies on
|
|
|
+ `FE E9 0D 00`, which is ASCII.
|
|
|
+- A **bare `3D` anchor is unsound.** It matches displacement and immediate bytes
|
|
|
+ as readily as opcodes, and using one produced two false alarms — a `3D` inside
|
|
|
+ a `CALL` displacement in `t11_for`, and one inside a string in `t21_mixed`.
|
|
|
+ The anchor is the three-byte `3D 00 00` or nothing.
|
|
|
+
|
|
|
+So it anchors on **comparison sites** rather than instruction boundaries:
|
|
|
+`3B C1` (`EmCmpAxCx`) and `3D 00 00` (`EmCmpAxi (0)`) are the only sequences that
|
|
|
+can precede a lowering. Branches are *additionally* found by shape (`7x 03 E9`,
|
|
|
+searching for the `E9`), which is what covers the `CASE` arm whose `EmCmpAxi`
|
|
|
+carries a label rather than 0.
|
|
|
+
|
|
|
+"Is this an 8086 shape" is a much weaker question than it looks — a `SETG` where
|
|
|
+a `SETGE` belongs is still a fine shape — so two clauses compare against
|
|
|
+**source**:
|
|
|
+
|
|
|
+- **G** pins `t33_cmpops`'s 13 comparisons against the operators in source
|
|
|
+ order. It catches a `>` / `>=` swap, and the failure message shows the swap
|
|
|
+ (`… Dh Dh Fh Fh Dh` where the source asks `… Fh Fh Dh Dh Fh`).
|
|
|
+- **H** pins, per fixture, the multiset of conditions its branch sites declare,
|
|
|
+ read off the `.pas` sources with the reasoning written beside each row.
|
|
|
+
|
|
|
+**H's need was measured, not anticipated.** Mutation M5 — the polarity inversion
|
|
|
+applied to the IF and CASE sites only — left the check **green** while every
|
|
|
+conditional in every program took the wrong path, because `IF` declares nibble
|
|
|
+4, `CASE` declares 5, and they negate into each other. The whole-suite version
|
|
|
+(M4) was caught only by luck: `FOR` declares `C` and `F`, whose negations `D` and
|
|
|
+`E` are declared for nothing at all. M5's first run coming back green is the
|
|
|
+entire reason H exists.
|
|
|
+
|
|
|
+Five new non-vacuity cases, each run by hand and confirmed red before being
|
|
|
+written down:
|
|
|
+
|
|
|
+ M1 EmJcc back to `0F 8x rel16` -> red, 45 problems
|
|
|
+ M2 EmSetcc back to `0F 9x` + MOV AH,0 -> red, 49 problems
|
|
|
+ M3 the `>` / `>=` SETcc arms swapped (the old bug 29) -> red, 2 problems
|
|
|
+ M4 branch polarity inverted everywhere (bug 33) -> red, 22 problems
|
|
|
+ M5 branch polarity inverted for IF and CASE only -> red, 12 problems
|
|
|
+
|
|
|
+Two harness faults were found while building them, both of which had produced
|
|
|
+false results before:
|
|
|
+
|
|
|
+- The 8086 group's first draft rebuilt with `make`, which does **not** build
|
|
|
+ `comtest`, so `check_8086.py` linked the stale compiler and M1 came back
|
|
|
+ **VACUOUS**. A helper that exits non-zero makes `if mutate_x; then` false, so
|
|
|
+ the case is silently *skipped* — which looks exactly like a pass. The helpers
|
|
|
+ now check the rebuild and say so.
|
|
|
+- A first cross-check of the size re-baseline reported "no mismatch" because no
|
|
|
+ `.COM` files existed where it looked, so its loop never ran and a vacuous
|
|
|
+ "all clear" came back. It was re-run with an assertion on the image count,
|
|
|
+ and this time reported the real answer. Worth recording because the failure
|
|
|
+ looked exactly like success — but note that cross-check was a throwaway
|
|
|
+ command, **not** a committed script, so the guard does not guard anything
|
|
|
+ permanent. `run_compile_tests.sh` re-baselining is still a manual edit.
|
|
|
+
|
|
|
+Scope decisions and findings
|
|
|
+----------------------------
|
|
|
+- `t33_cmpops` is a new fixture, and it exists partly for clause G: `=`, `<>`
|
|
|
+ and `<=` had **never been used as a comparison anywhere in the suite**, and
|
|
|
+ `>` / `>=` had already been swapped once, so the coverage hole and the bug it
|
|
|
+ let through were the same hole. Its `.out` is hand-derived from the Pascal and
|
|
|
+ was written **before** the machine ran.
|
|
|
+- `expected.tsv` re-baselined for nine control-flow fixtures, every one upward
|
|
|
+ by exactly **+1 per lowered site** (a branch 4 -> 5 bytes, a value 5 -> 6).
|
|
|
+ The site counts were counted out of the linked images and each row written
|
|
|
+ only after its delta matched its own count; the arithmetic is spelled out in
|
|
|
+ the file. Data sizes did not move. Byte-level expectations for those sites now
|
|
|
+ live in `check_8086.py`, behaviour in `run_com_exec.py`; the matrix row only
|
|
|
+ says how big the code is.
|
|
|
+- `check_8086.py` cannot assert on the `CASE` arm's comparison: it is
|
|
|
+ `3D lo hi` with a non-zero label immediate, and no sound anchor can find it.
|
|
|
+ The arm is covered by shape and by `t22_case`'s execution, but the label
|
|
|
+ immediate itself is unchecked.
|
|
|
+- The check executes nothing. It proves the opcodes are 8086 and that
|
|
|
+ conditions are attached to the right constructs; the control flow those bytes
|
|
|
+ produce is still only checked by qemu **on a 486**.
|
|
|
+
|
|
|
+Tag placement
|
|
|
+-------------
|
|
|
+`v-TP3-8086-LOWERING` points at the commit that adds this file, so its tree
|
|
|
+contains this milestone's account of it — the convention every other tag
|
|
|
+follows, and the one `v-TP3-COM-IMAGE` departs from for reasons recorded in its
|
|
|
+own tag message. (The tag was first placed on the work commit and moved once
|
|
|
+this file was written; both commits are in the tree, and nothing is pushed.)
|
|
|
+
|
|
|
+What remains (next steps)
|
|
|
+-------------------------
|
|
|
+1. `CmdRun` (`R` key): in-process 8086 interpreter, cross-validated against
|
|
|
+ qemu on the same images. Also the **only** candidate for a real 8086
|
|
|
+ execution oracle, since qemu cannot be one.
|
|
|
+2. String variables (`s : string`, assignment, `writeln(s)`).
|
|
|
+3. Nested procedures/recursion, `var` parameters (`SEG:OFF`), range/index
|
|
|
+ checks, typed constants, array at use (`t14`), case subrange labels.
|
|
|
+4. `readln` of a `BYTE` (separate `xrdbyte` / `TU_RdByte`).
|
|
|
+5. Enforce the 4 KiB code window (error if `pc` reaches data start).
|
|
|
+6. Harmonise multi-name declarations (both `,` and `;` spellings).
|
|
|
+7. Harden program-header parameter loop against non-advancing input.
|
|
|
+8. FreeDOS as a third opinion (untried).
|