docs: summary step 1.12 — complete TopSpeed V3 grammar (662-file corpus parse-clean)

docs/summary_step1.12.md

@@ -0,0 +1,87 @@
+# V3 step 1.12 — complete TopSpeed V3 grammar: full 662-file corpus parse-clean (done 2026-09-20)
+
+Deliverable: `compiler/src/TopSpeed-V3-M2.atg` — the **complete historical
+TopSpeed/Clarion Modula-2 V3 dialect** as a Coco/R grammar, reconstructed
+from the real V3 distribution and validated to parse the whole thing.
+
+The modern-subset frontend (`compiler/src/M2.atg`) is untouched; the two
+grammars coexist (see `docs/grammar.md`). The scratch compiler used for
+validation is `/tmp/opencode/buildv3/M2` (corpus area
+`/home/eric/bin/Dos/M2/TS-V3`).
+
+## Validation numbers
+
+- CR: **25 benign LL(1) advisories** (all "start & successor of a
+  deletable structure" + a couple "start of several alternatives"),
+  **0 attribute errors, 0 undefined/unreachable/circular nonterminals**.
+- **Full 662-file corpus: 0 syntax errors.** Every remaining sweep row is a
+  semantic `not a type` (parse-clean), e.g. GWINC2M/GRAPH/WINDOW DEFs,
+  COMPDEMO, LIB.MOD, WINDOW.MOD. LIB.MOD:1224 (was `'FOR' expected`),
+  WINDOW.MOD, GRAPH.MOD all clean.
+- Scratch regression suite: **35 passed, 0 failed**.
+
+## Dialect coverage added vs. the modern subset
+
+Identifiers `_`/`@`; `&`=AND, `~`=NOT, `<>`=`#`; literal suffixes `H`/`C`/`B`
++ `0x` hex; `::=` constant aliases; anchored subranges (SHORTINT[-1..1]);
+qualified type/designator idents; ARRAY (fixed + open), RECORD with
+variants, SET OF, POINTER TO, PROCEDURE types; WITH, IN, IS, set
+constructors `{}`, shifts `<<`/`>>`, calls; Clarion-class `CLASS ...`
+(decl+single inheritance+IMPLEMENTATION blocks, CLAUSE extra parent = 230);
+`LABEL`/`GOTO`; `INLINE`; enum TYPEs; mapped-array constructors (VarDecl
+nil slots + factors).
+
+Constructs the scalar backend cannot lower yet parse fully and end with
+standard `SemError(230)` — the parse-now/lower-later discipline.
+
+## Fence policy (scanner, `M2S.mod` manual patch)
+
+`(*%F flag*)` regions are **skipped** (level-matched, nested `%T`/`%E`
+balance); `(*%T tag*)` is **whitespace but its region is compiled**;
+standalone `(*%E*)` is an empty comment. All flags off in default build.
+
+- Required for the shipped corpus: DEMO/OMAIN/COMPDEMO doc banners,
+  WINDOW.MOD's OS/2 alternate body, LIB.MOD variant blocks.
+- The alternative "%T also skip" policy was tested and **rejected**: it
+  strands LIB's `IsOfClass` common loop body (the `%T`/`%F` markers each
+  supply a WHILE header; skipping both leaves the shared `IF…;
+  ThisObject:=…; END;` dangling → `LIB.MOD:126:9 ident expected`).
+- **Re-apply after every CR regeneration** (regeneration garbles `M2S.def`
+  too — restore from repo; never copy an older patched `M2S.mod` over a
+  fresh scan — token tables renumber when literals change).
+
+## The LIB.MOD:1224 bug — root cause and fix
+
+Not a fence issue. `WHILE [seg:ofs bp]^ # 0 DO` failed `'FOR' expected` at
+the `^`: the mapped-array constructor `[Expr : Expr [ TypeIdent ]]` was a
+*Factor* with no designator-suffix chain, so `^`/`.id`/subscript/call after
+it was unparsable. Confirmed by minimal repro (`tt.mod` failed identically
+at 8:21); ST@ trace showed WHILE dispatched then died inside its
+expression. Earlier line-bisection results were marker-pairing artifacts.
+
+Fix: new `MapTail` production (suffix chain `{ "." ident | "^" |
+"[" Expr "]" | "(" [ params ] ")" }`, each `SemError(230)`), called after
+the mapped branch of `Fact`.
+
+## Other grammar bugs fixed along the way
+
+- Empty-ELSE: `[ "ELSE" [ StatSeq ] ]` in IF and CASE (was mandatory
+  StatSeq) — closed the MODBASE/REP16/ELSE cascade errors.
+- `lqn` not `lxq` in Statement's GOTO attribute (compile error).
+- This session's earlier 4/35 regression came from a stale token table
+  (patched `M2S.mod` copied over a fresh regen after LABEL/GOTO renumbered
+  literals) — re-patching the fresh file restored 35/35.
+
+## SymTab / QbeGen (in repo)
+
+`SHORTCARD` predefined (SymTab); `QbeGen.NormLit` handling `0x`/hex/
+octal-B/C-character literal forms. Committed with the ATG.
+
+## Next
+
+Composites in the modern subset (step 3: ARRAY/RECORD/SET/POINTER, WITH,
+IN) can cross-check against `TopSpeed-V3-M2.atg`. Decide how the recovered
+dialect rules relate to the R10-clean `M2.atg` (the corpus grammar is the
+deliverable; merging is a separate decision). Scratch tooling/tests live
+under `/tmp/opencode` (not committed); the canonical patched scanner is
+`/tmp/opencode/M2S.mod.patched`.