plan.md 6.9 KB

m2compiler-V3 — Actual Plan (locked 2026-09-19)

Modern Modula-2 syntax, Coco/R frontend, fresh QBE backend, Blaise-style phased process, self-hosting with byte-identical .ssa fixpoint.

1. Toolchain and dialect

  • Frontend: Coco/R (M2.atg → scanner/parser/driver), gm2 -fiso, CR -m -C regeneration, two-phase gm2 -fgen-module-list link (proven in V1/V2/Test2).
  • Backend: fresh QbeGen emitting QBE SSA text (.ssa), then qbe -o .s + cc link + execute. System qbe pinned in vendor/qbe-pin.txt.
  • Modern syntax base: PIM4/ISO core + Kowarsch/Redux cleanups — # only (no <>), NOT/AND/OR only (no ~/&), no B/C octal literals, H hex kept, unary minus takes a Factor. Evolved from V2 M2comp.atg (1290 lines), not pasted verbatim.
  • Test convention (from Test2): global VAR ExitCode : INTEGER selects process exit code; otherwise return 0.

2. Directory tree (Blaise-adapted)

Blaise (graemeg/blaise) uses compiler/, runtime/, stdlib/, tools/migration-analyser/, vendor/qbe/, docs/, per-module target/. V3 port:

m2compiler-V3/
  README.md  build.sh  run_tests.sh
  compiler/src/      M2.atg, compiler.frm, parser.frm, scanner.frm,
                     FileIO, SymTab, AST, QbeGen, generated M2S/M2P/M2 (gitignored)
  compiler/tests/    accept / reject (.LST) / run (exit-code) tests
  runtime/syslib/    SYSTEM, Storage, SysIO, SysClock, Trap, Utf8
                     always-linked, self-hostable, NO gm2-only imports
  stdlib/            Strings, TextIO (+U*), WholeIO, Math, Files
                     opt-in via IMPORT, classic DEFINITION+OPAQUE,
                     Benjamin-R10 semantics, classic syntax only
  tools/redux-check/ modern-syntax enforcement helper
  vendor/qbe-pin.txt
  bootstrap/         stage1/ (gm2-built), stage2/, stage3/ (V3-built),
                     fixpoint.sh (rebuild 2x, diff -r *.ssa)
  docs/              plan.md (this file), spec.md, blaise-phases-map.md,
                     summary_stepN.md

Build outputs (.o, .ssa, .s, binaries) are never committed.

3. Library spec (Benjamin, classic form)

  • Semantics follow Benjamin/Actual/m2bsk-master/r10/stdlib/*.if (+ blueprints/templates as semantic reference) and PDFs-master/.
  • Syntax is classic DEFINITION/IMPLEMENTATION + OPAQUE only. Explicitly OUT: INTERFACE MODULE, [ProtoX]/[TMIN] attributes, PROCEDURE [+] operator overloads, .bp/.gen.if templates.
  • runtime/syslib vs stdlib/ split: syslib is always linked and trivially compilable; stdlib is opt-in via IMPORT.

4. Phased roadmap

Step Goal Acceptance
1 pipeline minimal M2.atg + fresh QbeGen hello → .ssa → qbe → cc → run 3 run tests
2 scalars scalar type system, SymTab errors 200–224 + 230/232/233 (+234 unicode) ~20 tests
3 composites length-prefixed arrays/records/sets/pointers (§5) layout tests green
4 modules/procs DEFINITION/IMPLEMENTATION, opaque TYPE T;, nested/display, separate compilation ADT tests (V2 goal_step8 scope)
5 stdlib Strings/TextIO/WholeIO/Math/Files per Benjamin I/O + file tests
6 harden CASE/WITH/LOOP/EXIT/FOR-BY, conversions, traps full-suite green
7 syslib port rewrite FileIO on SysIO+Storage only; ban Environment, FileSysOp, TextIO, RawIO, WholeIO, IOChan, SysClock, ProgramArgs via grep gate compiler sources syslib-clean; Stage1 still gm2-built
8 fixpoint bootstrap/fixpoint.sh: Stage1→Stage2→Stage3, diff -r all .ssa byte-identical, suite green on all stages SELF-HOSTED

Full-language-first: the compiler itself may use the full language; self-hosting is the last phase. Hygiene from step 1: deterministic %tN/@LN counters, sorted symbol emission, no timestamps/paths in .ssa; restricted SYSTEM use (ADR/TSIZE/BYTE only).

5. Memory rule — length-prefixed arrays & strings (LOCKED)

  • Every ARRAY value: bytes 0..7 = LONGCARD element count (HIGH-LOW+1), QBE l; bytes 8.. = packed elements.
  • Applies to ALL arrays including fixed ARRAY [lo..hi] (bounds still checked statically AND stored at runtime). 8 bytes overhead/array.
  • Multi-dim: each level has its own header (recursive).
  • Records: field offsets include headers (8 + count*elemSize).
  • SYSTEM.TSIZE on arrays includes the header.
  • Open formal ARRAY OF T: single l descriptor pointer; callee reads count via loadl. No hidden length parameter.
  • Whole-array := requires equal counts, else Trap.RangeFault (no silent truncation — safety-perimeter principle).
  • Index a[i]: base + 8 + (i-lo)*elemSize + runtime i < lo+count check.
  • HIGH(a)/LEN(a): static for fixed, header read for open.
  • String = ARRAY OF CHAR with this header; literals emitted as data $n = { l count, b ... }. No trailing NUL; C boundary via SysIO.ExportCStr copy.
  • SYSTEM.BYTE buffers (FileIO, SymTab.Name) conform automatically.

6. Unicode — UCHAR / UString (LOCKED)

  • UCHAR: 32-bit codepoint 0..1114111 (QBE w). CHAR unchanged (8-bit, 0..255). Distinct types, no implicit mixing.
  • UString = LONGCARD count (codepoints) + count × l codepoints (prior header rule, element = UCHAR, 4 bytes each).
  • Source files are UTF-8. Identifiers stay ASCII. Coco/R captures literal bytes; LexUString action strict-RFC3629-decodes (no overlongs/surrogates/>10FFFF/truncation).
  • Literals: U'..' / U".."; single codepoint U'a' → UCHAR, longer/empty → UString. 'a' ≠ U'a'.
  • Invalid UTF-8 in literal → hard lex error 234, in .LST, no .ssa.
  • Conversions: UCHR(c:CHAR):UCHAR safe; CHR8(u:UCHAR):CHAR traps unless u ≤ 255; ORD 8-bit; new UORD(u):LONGCARD.
  • runtime/syslib/Utf8 (decode/Encode, MaxUTF8Length=4, MaxCodePoint=1114111, cf. Benjamin Utf8.def): shared by the lexer (decode) and output libs (encode).
  • TextIO.UWriteChar/UWriteString/UWriteLn: encode to UTF-8, emit via SysIO byte descriptors. Raw WriteChar stays byte-oriented.

7. Interactive ATG order (next)

  1. COMPILER M2 header + CHARACTERS/IGNORE/COMMENTS/TOKENS (incl. ustring, grown LexString buffer).
  2. Module / Definition / Implementation / Import (+ opaque hook).
  3. CONST/TYPE/VAR/ProcHeading + error codes (200–234).
  4. Types (scalars → UCHAR → composites → proc types).
  5. Statements (IF/CASE/WHILE/REPEAT/LOOP/EXIT/FOR/WITH/RETURN).
  6. Expressions (precedence, IN, set/pointer/UCHAR ops).
  7. Action wiring (SymTab/AST/QbeGen hooks) + compiler.frm driver.

Each section: proposed snippet → user approve/tweak → append to compiler/src/M2.atg → CR -m -C syntax check before moving on. Per-section self-host question: "can this cover the compiler's own source idioms?".

8. Open points for build time

  • Exact U'..' vs U".." spelling confirmation during §1.
  • stdlib Phase-5 subset order: SYSTEM+Strings+TextIO first?
  • Trap exit-code conventions for run tests.