M2compiler-V3/docs/STATUS.md

m2compiler-V3 — status (2026-10-22)

This answers "where are we now?".

Numbers

• Suite 252/252 (compiler/run_tests.sh).
• Self-hosting FIXPOINT OK — stage2 and stage3 images are byte-identical (3,906,044 bytes).
• Corpus 485 / 612 sources compile with 0 errors, 0 crashes (tools/v3-corpus/corpus.sh over the gm2 testsuite; the harness now resolves V3's own shipped libraries); the rest are genuine language gaps, gm2 leniencies V3 deliberately rejects, or unresolved imports of gm2-only modules.
• Branch master; tree clean.

Architecture (current)

per unit:
  Parse   -> AST + declarations; each procedure body is skipped and
             re-parsed at unit end, so forward references bind normally
  Lower   -> walk the AST, emit QBE (the only emitter)
  Emit    -> buffered image, EndModule

• Two-phase parse. Procedure bodies are deferred (module level) and re-parsed after the unit's declarations. The re-parse is order-aware: a body sees only module-level variables, constants and types declared before its procedure (forward procedure calls remain allowed). -notwophase restores the legacy inline parse for debugging.
• No resolve/check pass. The Check module was deleted once it was proven vestigial (gutting it left the emitted .ssa byte-identical): binding is late by construction.
• Lower is the only emitter; the AST is the interface.

Language coverage

Program modules; CONST/VAR/TYPE (scalars, subranges, enums, arrays, records incl. variants, sets, pointers); WITH/IN; whole array/record copy; NEW/DISPOSE; full expressions and control flow; GOTO/labels; procedures (value/VAR params, functions, recursion, FORWARD, nested procedures, procedure values); Clarion-form CLASS (methods, single inheritance, virtual dispatch); PIM coroutines; Unicode (UCHAR/UString, UTF-8 edges); ISO type conversions T(x); separate compilation (FROM/IMPORT, qualified L.x, opaque types); generics (ISO 10514-2 refinement): type/value/procedure parameters, constraints, nested generics, generic classes.

Multi-dimensional arrays use the PIM/ISO repeated-bracket form (ARRAY [lo..hi], [lo..hi] OF T) as well as V3's earlier one-bracket comma list. Subranges keep their ordinal base: ['A'..'Z'] is a character type and [red..blue] belongs to its enumeration, so base-typed values assign and compare; FOR accepts any ordinal control variable (FOR c := red TO blue). The ISO empty string literal (""/'') is compatible with CHAR (value 0) in assignment, comparison, CASE labels, FOR bounds, actuals and RETURN, while remaining a string for ARRAY OF CHAR (whole-array assignment and content comparison).

Landed recently (this thread)

• AST + Lower (L0–L7) — Lower emits the whole session from the AST.
• Two-phase parse; forward machinery deleted; Check deleted.
• Generics increment 3: value/procedure parameters, constraint model, nested generics, generic classes via generic modules.
• ISO type conversions T(x) (incl. CONST folding), the builtins ODD/TRUNC/FLOAT, and BOOLEAN/ENUM array bounds + indices: corpus 365 → 381.
• SET CONST (static masks): a set constant (CharSet{"0".."9"}, a union of set constants) is folded to a mask at parse and re-emitted as a pooled data $setN descriptor. Builtin CONST folding (CAP/ABS/ODD over immediates), BOOLEAN CONSTs, representation-changing CONST conversions (VAL(REAL, int), VAL(LONGINT, int), …), procedure values (ADR(proc), VAL↔PROC conversions, p := foo into a PROC), SYSTEM sized reals (REAL32/REAL64), FROM SYSTEM IMPORT SHIFT/ROTATE/CAST/… (a builtin import is accepted), and ISO foreign modules (DEFINITION/IMPLEMENTATION MODULE FOR "C", EXPORT UNQUALIFIED), and ISO CONVERT(T, x) (a VAL sibling), set↔word VAL/CAST (BITSET), and enumeration literals with their type (export/import/ qualified; imp*) and ORDL de-reserved (a gm2/FileIO helper, not a built-in; fileio/baseimport): corpus 382 → 405. Fixed a SHIFT-by-a-negative fault and the qualified-const value (LoadDesignator).
• The corpus harness now resolves transitive imports (it used to under-provide .defs); this more accurate harness is the method from 2026-10-18.
• Clean M2 exit codes (0 ok / 1 errors / 2 usage-IO).
• Bug fixes: PopPos wiping repass diagnostics; global record data size (w 0 → l 0 for pointer/LONGINT/proc fields); template arithmetic over a type formal; proc-typed formals in DEF/IMPL; imported procedures used as values.
• Showcase showcase23 (generic doubly linked list).

Conformance policy

V3 follows Modula-2 (PIM/ISO), not gm2's degradations. In particular the language requires declaration before use (the only forward reference is a POINTER's base type), so the use-before- declaration programs that gm2 accepts are rejected by design. The corpus is gcc's gm2 testsuite, so its number mixes conforming programs with gm2 leniencies; each known case is catalogued (with a reproducer) in docs/gaps/.

Known non-conforming behaviour still retained (pre-existing, not yet removed): forward procedure calls from a body (the two-phase parse defers procedure bodies, and the generated parser's mutually recursive productions rely on it). Everything else is strict:

• the declaration part: module-level forward type/VAR references, forward types inside a TYPE block, and forward SET OF <type> are rejected (Track A, 2026-10-18);
• inside a deferred procedure body, a reference to a module-level variable, constant or type declared after the enclosing procedure is rejected as undeclared (Track P3, 2026-10-19) — the body re-parse is order-aware (SymTab declaration sequence + Lookup). FORWARD remains the way to call a later procedure. Corpus 427 → 421 (the drop is 6 gm2-leniency files now correctly rejected, e.g. block, blockindirect, nestedfor).

Module-level forward procedure references and FORWARD declarations still work (t_fwdproc, t_fwdok).

What remains

1. Corpus — the 2026-10-22 triage fixed four conforming clusters (+18 sources: PIM/ISO ARRAY [..], [..] OF, typed CHAR/enum subranges, ordinal FOR, ISO empty string ""/'' as CHAR). The remaining LANG entries are mostly: 48 files importing libc that need real prototypes (V3's FROM libc IMPORT declares a permissive no-result varargs external, so using printf/malloc in an expression fails); declaration order cases V3 rejects by policy (docs/gaps/); ISO exceptions/PACKEDSET/SYSTEM sized types (WORD32, BITSET32, qualified SYSTEM.CARDINAL8); and gm2-only modules (M2RTS/Builtins/…). See docs/analysis_gaps.md.
2. The size-sensitive V3 miscompile — root-caused and fixed (2026-10-19; see docs/summary_l3_selfhost_fix.md). It was never size-sensitive: the fixpoint gate matched the literal string "Parsed correctly" that the ASTDUMP debug dump printed unconditionally, so the gate passed while the compiler session was actually reporting Incorrect source (Lower's forward procedure references → invalid call). Removing ASTDUMP merely removed the masking line, which looked like a size effect. Fixed by a3ef897 (remove ASTDUMP; gate on Parsed correctly; declare Lower's walkers in Lower.def). The gate now fails on errors (verified with t_fwdvar), and self-hosting stays byte-identical.
3. Robustness — table-limit overflows are now loud (2026-10-19): the two-phase span table (which silently dropped a body's codegen, t_manyproc), the label/GOTO tables, the 32-actual AST buffer and SymTab's MaxTypes all report 230 (implementation limit) instead of truncating silently. Still weak: error recovery and message polish.
4. Library holes — the channel framework grew ChanConsts and StreamFile (ISO 10514-1) on top of IOChan, and the PIM/gm2 I/O family gained NumberIO, RealInOut, FpuIO (StrIO/InOut already existed). Also fixed the corpus harness fidelity: it now searches V3's own shipped libraries (stdlib, runtime/syslib) via a default $V3_LIBS, so a file importing e.g. Strings resolves against the real .def instead of being mislabelled LIB. Building StreamFile also fixed a real bug: a set CONST exported from a definition module is now emitted under its global name (<unit>_<name>), so a client's qualified reference binds. Still missing: gm2 runtime modules (M2RTS, Builtins, Processes, …).
5. Housekeeping — docs/INDEX.md is the entry point; the unreferenced historical records are under docs/archive/ (2026-10-19). resolve-check was merged to master.

How to check for yourself

cd compiler
./build.sh          # M2.atg -> scanner/parser/driver -> ./M2
./run_tests.sh      # 249/249
cd ..
./bootstrap/fixpoint.sh   # FIXPOINT OK (byte-identical stage2/stage3)