# TP3-comp — state summary

Recreating **Turbo Pascal 3.0** (shell / editor / compiler / interpreter) in
**GNU Modula-2** (`gm2 -fiso -Wall`). Faithful to the original: screen
layout, keys and behaviour are reconstructed from the disassembled TP3.0
source in `Resources/turbopascal3source/TP3/` (TPSRC1-10) and the reference
manual, not guessed.

## Milestones

| Milestone | Tag | State |
|---|---|---|
| TP3.0 main-menu shell | `v0.1-shell` | done |
| WordStar-style editor | `v0.2-editor` | done |
| Shell/editor polish, Ctrl-K-D / Ctrl-K-X quit | `v-TP3-SHELL-EDITOR-QUIT` | done |
| Compiler skeleton + build recipe | `v-TP3-SHELL-COMPILES` | done |
| Parser `Skip` bug class (9 sites) | `v-TP3-PARSER-FIXES` | done |
| Standard procedures + `rel16` fix | `v-TP3-STDPROCS` | done |
| Runtime blob, linker, `CmdRun` interpreter | — | **not started** |

## Build

```sh
cd shell && make          # → shell/tpshell
```

Whole-program link **must** be two-phase; a single `gm2 -o` pass 3 silently
caps identifier/error counts on a compiler-sized program:

```sh
gm2 -fiso -c Compiler.mod                                     # phase A
gm2 -fiso -fgen-module-list=modules.lst -o /dev/null \
        Compiler.mod Term.o TextBuf.o Posix.o Editor.o        # phase B1 (rc=1 expected)
gm2 -fiso -fuse-module-list=modules.lst -o tpshell \
        Shell.mod Compiler.mod Term.o TextBuf.o Posix.o Editor.o   # phase B2
```

Current clean build: `make clean && make` → rc=0, `tpshell` **120792 bytes**.
The one diagnostic is `./Compiler.mod: ParseExpr: too many errors in pass 3`,
which is the expected phase-1 rollup that the recipe tolerates — not a real
error. `shell/build_tpshell.sh` and `shell/Makefile` are authoritative.

## What is verified, and how

Nothing here is "it compiles clean" — each claim below comes from a run.

### Compiler front end — `shell/tests/run_compile_tests.sh`

`tests/CompileTest.mod` links `Compiler` + `TextBuf` + `Posix`, reads fixture
paths from **stdin**, loads each exactly the way `LoadWorkFile` does
(LF→CR normalisation, `^Z` ends the text), calls `Compile`, and prints a
verdict plus a source excerpt with a caret at `errPos`. No pty, instant.

```
cd shell && tests/run_compile_tests.sh            # all fixtures
cd shell && tests/run_compile_tests.sh /some/dir  # another fixture set
printf '@dump\ntests/fixtures/t19_int1.pas\n' | ./compiletest   # hex-dump the image
```

**17 of 23 fixtures compile**, up from 1 (the empty program) when the
direct harness was first built.

Compiling: `t01` minimal · `t04` var+assign+`writeln` · `t06` two args ·
`t07` 10 assignments · `t08` const · `t09` if/then/else · `t10` while ·
`t11` for/to · `t12` repeat/until · `t13` procedure + value param ·
`t15` label + goto · `t16` `writeln('a')` · `t18` bare `writeln` ·
`t19` `writeln(1)` · `t20` 3 string args · `t21` mixed args ·
`t22` `case` with two labels.

Failing, all deliberately: `t02/t03/t05/t17` multi-char string literals and
`t14` `array [1..5] of integer` → `ENoLib` (102), the original's
"not implemented" path; `uierror` is a deliberate syntax error used by the
UI test.

### Shell + editor UI — `shell/tests/uitest.py`

Drives a real pty (`tests/ptyharness.py` supplies read-until-quiet, key
sending and a small VT100 emulator) and asserts the TP3 **compile-error
jump**: `W` load → `C` compile → `ESC` → editor opens with the cursor on the
error position → `Ctrl-K D` back to the menu → `Q` exit 0. **10/10 pass**,
and the reported error number is 41 (not 0 — this is what caught the
`errNo := errNo` self-assignment where `Compile`'s formal shadowed the
module variable).

The jump mirrors original TP3 `kcwait` + `editor2` (TPSRC5:333-336, :919):
`BX:=txerrpos; DEC BX; JMP editor2`, and `editor2` does `ADD BX,txbeg;
INC BX` — the `DEC`/`INC` cancel, so the net is `txbeg+txerrpos` = our 0-based
`errPos`. Armed as a sticky position (`Editor.GotoOffset`) rather than by
changing `Run`'s signature, so `Editor.def` stays additive.

`uitest.py` uses a fixture with a deliberate **syntax** error
(`x := 1 + ;` → error 41, line 9 col 12) rather than a missing library
feature: the latter move as the compiler grows, and a test whose
expectations drift with it stops being a test.

## Components

### Shell — `shell/Shell.mod`, `Term.mod`, `Posix.c`, `TextBuf.mod`

Exact TP3.0 screen (`Logged drive`, `Active directory`, `Work file`,
`Main file`, `Edit Compile Run Save` / `Dir Quit compiler Options`,
`Text: n bytes`, `Free: n bytes`, `>`), command letters drawn bold. Keys
`L A W M E C R S D O Q`; any other key redraws the menu, like TP3. `W`
auto-`.PAS` with `Loading`/`New File`; `S` writes `^Z` EOF and rotates the
old file to `.BAK` (unlink+rename, TP3's order); `D` is a DOS `*.*` glob
listing with `k bytes free`; `O` is the options submenu; `Q` confirms and
prompts to save when the text changed.

`E` and `C` are wired. **`R` is a stub** — it prints "Interpreter pending".

### Editor — `shell/Editor.mod`

WordStar-style full-screen editing over `TextBuf`. Status line
`Line n  Col n  Insert/Overwrite  Indent  X:FILENAME`; text on rows 2-24.

Movement (`^S/^D/^E/^X/^A/^F/^R/^C/^W/^Z`, `^Q S/D/E/X/R/C/B/K/P`, arrows,
PgUp/PgDn/Home/End), editing (`^V` insert/overtype, `^G` delete char,
backspace joins lines, `^T`/`^Y` word/line, `^Q-Y` to EOL, `^N`/CR break,
TAB auto-indent to the word start above), block (`^K B/T/H` mark, word,
show, `^K C/V/Y` copy/move/delete, `^K R/W` file read/write), search
(`^Q-F`, `^Q-A`, `^L` repeat; options B/G/n/U/W, N = no-confirm; `^A` any
char, CR LF matches a line break), `^P` literal control char,
`^U`/ESC aborts a prompt. `^K-D` returns to the shell with the text still in
memory and `changed` reported. Disk format matches TP3: CRLF plus trailing
`^Z`, load normalises, save re-expands.

Detail: `TP3-EDITOR.md`, `TP3-EDITOR-PSEUDOCODE.md`.

### Compiler — `shell/Compiler.mod`

Single-pass Pascal → 8086, following TPSRC6 `turbo` / TPSRC7-10: one pass
over the shared `TextBuf` emitting machine code into `cbuf`, a patch list
for forward references, TP3-style error reporting (number + relative
position), and code/data size accounting. Emitted image is a byte array
(mode word, CS/DS, size words, `CALL initmem`, `MOV BP,SP`, generated code).

Working subset (v0.4): integer/char/boolean/byte scalars, constants with
folding, globals, locals, value parameters, procedures and scalar-result
functions, `ARRAY[const..const]` with constant indexing, control flow,
`GOTO`/`EXIT`, and the standard procedures `WRITE`, `WRITELN`, `READ`,
`READLN`, `HALT`.

**Standard procedures** are `KBuiltin`, not `KProc`, because they are not
called generically. TP3 (TPSRC8 `pwriteln`/`pwrloop`/`prdtyped`) does *not*
pass a descriptor to the runtime: it inspects each argument's class and emits
a *different call per type*, so formatting is fixed at compile time and the
runtime only ever sees a value. `IoCall` mirrors that — one call per
argument, then a final call for the line break:

```
writeln(1)    MOV AX,1     ; PUSH AX ; CALL 20H ; ADD SP,2 ; CALL 40H
writeln('a')  MOV AX,'a'   ; PUSH AX ; CALL 28H ; ADD SP,2 ; CALL 40H
readln(x)     LEA AX,[0104]; PUSH AX ; CALL 48H ; ADD SP,2 ; CALL 60H
```

`READ`/`READLN` push the *address* so the runtime can store
(`EmPushVarAddr`: `8D 46 disp` / `8D 06 off`); a non-variable argument is
`ETypeErr` (56), as in TP3. `TU_WrInt/Char/Bool/Real`, `TU_WrLn`,
`TU_RdInt/Char/Bool`, `TU_RdLn`, `TU_Halt` continue the existing `TU_*`
image-base space (`TU_InitMem=8H`, `TU_ProgEnd=10H`, `TU_StackChk=18H`).

Detail: `TP3-COMPILER.md`.

## Honest limitations

- **A compiled image still cannot be executed.** `CmdRun` is a stub and the
  runtime blob does not exist. The emitted code is verified *byte by byte*
  against the offsets the compiler intends, but nothing has ever run it.
- **The `TU_*` entry offsets are placeholders under an assumed model** — a
  runtime blob prepended to the image, with every `pc` rebased by its size.
  That model is assumed, not implemented; the linker is unwritten. Note the
  prologue's `CALL TU_InitMem` targets offset 8, which is currently the
  `hdrMax` header word, so the scheme is only coherent once the runtime is
  actually prepended.
- **No string runtime.** `RdConst` gives a 1-character literal as `TScalar`
  (its char code) and only longer literals as `TString`, so multi-char
  literals raise `ENoLib`. `writeln('a')` works via a `chr` flag on `ERes`;
  without it the compiler emitted the *integer* writer and would have printed
  97 while the test still said OK.
- **Not implemented** (all `ENoLib`): real, set, record, file, string, and
  any type wider than 2 bytes. `with` is `ENoLib`. `case` *is* implemented
  (cascade `CMP`/`JNZ` per label, per RESUME-TP3.md §3.6) but only over
  scalar labels — subrange labels and label lists are untested.
- gm2 string-literal → `ARRAY OF CHAR` assignment copies the literal *plus a
  NUL* and leaves the tail untouched, so NUL-terminated tables are safe —
  this was checked, not assumed.

## Three bugs only a byte-level dump could find

None of these produced a compile error, and two produced *passing* tests.
All three were found by hex-dumping the emitted image and decoding it by
hand — code sizes looked perfectly plausible throughout.

1. **`rel16` off by 2 in every direct CALL/JMP.** `EmCall`/`EmJmpNear`/
   `EmJcc` computed the displacement from `pc` at a point where `pc` already
   pointed past the opcode and *at* the displacement field; x86 measures
   from the end of the instruction (`pc + 2`). `ResolvePatches`, on the
   forward-patched path, was already right — which is why forward gotos
   looked fine and backward ones did not.

2. **`EmMovAxSp` emitted `8B 04` with no SIB byte.** ModRM `04` means "a SIB
   byte follows", so the `CMP AX,imm16` of the *next* instruction was eaten
   as that SIB byte, and the intended `MOV AX,[SP]` became
   `MOV AX,[BP+DI+disp]`. Every `case` label comparison therefore vanished
   and `case` compiled to a chain of loads from garbage addresses — while
   the fixture reported OK. Correct encoding is `8B 44 24 00` (`[SP]` cannot
   use mod=00, that computes `BP+SP`).

3. **The dump tool's own offset column was wrong.** It printed 4 *nibbles*
   through a helper that formats a **byte**, so every row was labelled 16×
   too large (`0010` shown as `00000100`). A misleading tool is worse than
   none — it corrupts any offset arithmetic done from its output.

## gm2 / ISO Modula-2 pitfalls hit along the way

- **Two-phase link** (above) — a single whole-program pass 3 caps
  identifiers/errors silently.
- `EXIT` inside the program-header parameter `WHILE` **ICEs gm2** in pass 3
  (`ExitStatement → PopExit → M2StackWord_PopWord → invalidloc`).
  Extracting the loop into its own procedure did *not* help. Use a `BOOLEAN`
  "advanced" flag, never `EXIT`. (`LOOP`+`EXIT` is fine, and is used widely.)
- A bare `HALT` aborts under `-fiso` (SIGABRT, exit 134); `HALT (0)` is
  correct.
- `CHAR` is not the ZType: `ch = 09H` must be `ORD (ch) = 09H`.
- ISO forbids dropping a function result in a statement → the `DropCh`/
  `DropB`/`DropC` discard helpers wrap ~39 call sites.
- `AND`/`OR`/`NOT` on 16-bit `CARDINAL` → `BitAnd`/`BitOr`/`BitNot`.
- `PROCEDURE f : T` is invalid; the file's convention is
  `PROCEDURE f () : T`.
- ISO will not index a plain string constant as an array (needs an array
  constructor) — compute hex digits with `CHR` instead.
- Foreign modules (`FOR "C"`) must use `ADDRESS`, not Modula-2 pointer
  types; cast at the call site. `Posix` has no `Posix.mod` (do not try to
  rebuild it) and no `argv` binding.
- termios raw mode: every newline is an explicit `CR LF`.
- Foreign modules and `Posix` cannot pass `argv`; the test harness reads
  fixture paths from **stdin** instead.

## Reference material

- `TP3-COMPILER.md` — the compiler: what was changed, the `Skip` bug class,
  the `rel16` off-by-2, standard procedures, current matrix.
- `TP3-EDITOR.md` / `TP3-EDITOR-PSEUDOCODE.md` — the original TP3 editor,
  from TPSRC5/TPSRC6.
- `RESUME-TP3.md` — book-derived reference for the 8086 code TP3 generates
  per Pascal construct (types, skeletons, arithmetic, IF/CASE/REPEAT/WHILE/
  FOR, procedures, parameters, functions, I/O, typed constants, absolutes),
  plus the `TU_*` runtime entry list.
- `Resources/turbopascal3source/TP3/` — TPSRC1-10, the disassembled original.
  **This is the ground truth**; when our behaviour and the book disagree, the
  disassembly wins.
- `Resources/coeur-tp-ocr/` — OCR of "Au coeur de Turbo Pascal".

## Next steps

1. **Runtime blob + linker.** Everything downstream is gated on it: the
   `TU_*` entries have to land somewhere real and every `pc` needs rebasing.
2. **`CmdRun`** — the interpreter. Without it no compiled program has ever
   run, so the codegen is unproven in the only way that counts.
3. **String runtime** — unlocks the last 4 real fixture failures.
4. Nested procedures / recursion, `var` parameters (the `SEG:OFF` push from
   RESUME-TP3.md §3.11), range/index checks (`TU_RANGE_CHECK`,
   `TU_INDEX_CHECK`), and typed constants (RESUME-TP3.md §3.14).