M2compiler-V3/docs/STATUS.md

m2compiler-V3 — status (2026-10-23)

This answers "where are we now?".

Numbers

• Suite 272/272 (compiler/run_tests.sh).
• Self-hosting FIXPOINT OK — stage2 and stage3 images are byte-identical (4,108,488 bytes).
• Corpus 512 / 612 sources compile with 0 errors, 0 crashes (tools/v3-corpus/corpus.sh over the gm2 testsuite; the harness now resolves V3's own shipped libraries); the rest are genuine language gaps, gm2 leniencies V3 deliberately rejects, or unresolved imports of gm2-only modules. Compile-only caveat: lazyunique now parses with EXCEPT but its generated QBE still hits a pre-existing pointer-chain (p^.next^.value) codegen gap.
• Branch master; tree clean.

Architecture (current)

per unit:
  Parse   -> AST + declarations; each procedure body is skipped and
             re-parsed at unit end, so forward references bind normally
  Lower   -> walk the AST, emit QBE (the only emitter)
  Emit    -> buffered image, EndModule

• Two-phase parse. Procedure bodies are deferred (module level) and re-parsed after the unit's declarations. The re-parse is order-aware: a body sees only module-level variables, constants and types declared before its procedure (forward procedure calls remain allowed). -notwophase restores the legacy inline parse for debugging.
• No resolve/check pass. The Check module was deleted once it was proven vestigial (gutting it left the emitted .ssa byte-identical): binding is late by construction.
• Lower is the only emitter; the AST is the interface.

Language coverage

Program modules; CONST/VAR/TYPE (scalars, subranges, enums, arrays, records incl. variants, sets, pointers); WITH/IN; whole array/record copy; NEW/DISPOSE; full expressions and control flow; GOTO/labels; procedures (value/VAR params, functions, recursion, FORWARD, nested procedures, procedure values); Clarion-form CLASS (methods, single inheritance, virtual dispatch); PIM coroutines; Unicode (UCHAR/UString, UTF-8 edges); ISO type conversions T(x); separate compilation (FROM/IMPORT, qualified L.x, opaque types); generics (ISO 10514-2 refinement): type/value/procedure parameters, constraints, nested generics, generic classes.

Multi-dimensional arrays use the PIM/ISO repeated-bracket form (ARRAY [lo..hi], [lo..hi] OF T) as well as V3's earlier one-bracket comma list. Subranges keep their ordinal base: ['A'..'Z'] is a character type and [red..blue] belongs to its enumeration, so base-typed values assign and compare; FOR accepts any ordinal control variable (FOR c := red TO blue). The ISO empty string literal (""/'') is compatible with CHAR (value 0) in assignment, comparison, CASE labels, FOR bounds, actuals and RETURN, while remaining a string for ARRAY OF CHAR (whole-array assignment and content comparison). PACKEDSET OF <ordinal> is supported with 1/2/4-byte storage (1/2/4/8/16/32-bit spans); SIZE and ORD follow the packed width, and INCL/EXCL/IN/set operations/assignment work on packed values. Wider packed sets are a loud implementation limit (230). Runtime subrange range checks now trap on assignment, value actuals, RETURN and INC/DEC (QbeGen.CheckRange); the post-increment of a FOR control variable is not checked (the final successor may legitimately exceed a full subrange). Adjacent one-character string literals concatenate in CONST ("W" + "o"), relations of ordinal constants fold (so ORD(a > b) works in a CONST), HIGH/LEN of a string CONST use the pooled literal's length, INC/DEC accept any ordinal, a leading | before the first CASE alternative is tolerated, and __FUNCTION__ folds to the current procedure (module name at unit level).

Procedure types accept comma-separated unnamed parameter types (PROCEDURE(ARRAY OF CHAR, ARRAY OF CHAR), PROCEDURE (CHAR, VAR T, ...)) as well as named groups and ; sections, and PROC is accepted as a result type (PROCEDURE (): PROC).

The hard-coded SYSTEM module is registered as a real module, so qualified access resolves (SYSTEM.CARDINAL8, SYSTEM.BITSET32) as well as FROM SYSTEM IMPORT. ISO/gm2 sized names are predefined: WORD8/16/32/64, BITSET8/16/32, INTEGER/CARDINAL8..64, REAL32/64/128, COMPLEX32/64/128, CSIZE_T/CSSIZE_T/COFF_T, plus BITSPERLOC/LOCSPERWORD/LOCSPERBYTE. BYTE, SBYTE, WORD8, CARDINAL8 and INTEGER8 are true 8-bit types (SIZE 1, loadub/loadsb/storeb); WORD16, CARDINAL16 and INTEGER16 are true 16-bit types (SIZE 2, loaduh/loadsh/storeh). Scalars, parameters, record fields and array elements use the sized storage; ARRAY OF BYTE is a real byte buffer. WORD/WORD32 remain 32-bit, and COMPLEX* is a declaration-compatible REAL placeholder.

Landed recently (this thread)

• AST + Lower (L0–L7) — Lower emits the whole session from the AST.
• Two-phase parse; forward machinery deleted; Check deleted.
• Generics increment 3: value/procedure parameters, constraint model, nested generics, generic classes via generic modules.
• ISO type conversions T(x) (incl. CONST folding), the builtins ODD/TRUNC/FLOAT, and BOOLEAN/ENUM array bounds + indices: corpus 365 → 381.
• SET CONST (static masks): a set constant (CharSet{"0".."9"}, a union of set constants) is folded to a mask at parse and re-emitted as a pooled data $setN descriptor. Builtin CONST folding (CAP/ABS/ODD over immediates), BOOLEAN CONSTs, representation-changing CONST conversions (VAL(REAL, int), VAL(LONGINT, int), …), procedure values (ADR(proc), VAL↔PROC conversions, p := foo into a PROC), SYSTEM sized reals (REAL32/REAL64), FROM SYSTEM IMPORT SHIFT/ROTATE/CAST/… (a builtin import is accepted), and ISO foreign modules (DEFINITION/IMPLEMENTATION MODULE FOR "C", EXPORT UNQUALIFIED), and ISO CONVERT(T, x) (a VAL sibling), set↔word VAL/CAST (BITSET), and enumeration literals with their type (export/import/ qualified; imp*) and ORDL de-reserved (a gm2/FileIO helper, not a built-in; fileio/baseimport): corpus 382 → 405. Fixed a SHIFT-by-a-negative fault and the qualified-const value (LoadDesignator).
• The corpus harness now resolves transitive imports (it used to under-provide .defs); this more accurate harness is the method from 2026-10-18.
• Clean M2 exit codes (0 ok / 1 errors / 2 usage-IO).
• Bug fixes: PopPos wiping repass diagnostics; global record data size (w 0 → l 0 for pointer/LONGINT/proc fields); template arithmetic over a type formal; proc-typed formals in DEF/IMPL; imported procedures used as values.
• Showcase showcase23 (generic doubly linked list).

Conformance policy

V3 follows Modula-2 (PIM/ISO), not gm2's degradations. In particular the language requires declaration before use (the only forward reference is a POINTER's base type), so the use-before- declaration programs that gm2 accepts are rejected by design. The corpus is gcc's gm2 testsuite, so its number mixes conforming programs with gm2 leniencies; each known case is catalogued (with a reproducer) in docs/gaps/.

Known non-conforming behaviour still retained (pre-existing, not yet removed): forward procedure calls from a body (the two-phase parse defers procedure bodies, and the generated parser's mutually recursive productions rely on it); and discarding the result of a foreign C varargs function used as a statement (printf/snprintf) — a documented compatibility carve-out. An ordinary Modula-2 function used as a statement is still 233 (t_bad_funstmt). Everything else is strict:

• the declaration part: module-level forward type/VAR references, forward types inside a TYPE block, and forward SET OF <type> are rejected (Track A, 2026-10-18);
• inside a deferred procedure body, a reference to a module-level variable, constant or type declared after the enclosing procedure is rejected as undeclared (Track P3, 2026-10-19) — the body re-parse is order-aware (SymTab declaration sequence + Lookup). FORWARD remains the way to call a later procedure. Corpus 427 → 421 (the drop is 6 gm2-leniency files now correctly rejected, e.g. block, blockindirect, nestedfor).

Module-level forward procedure references and FORWARD declarations still work (t_fwdproc, t_fwdok).

What remains

1. Corpus — the 2026-10-22 triage fixed four conforming clusters (+18 sources: PIM/ISO ARRAY [..], [..] OF, typed CHAR/enum subranges, ordinal FOR, ISO empty string ""/'' as CHAR), added real libc prototypes (+6), ISO SYSTEM sized types + qualified SYSTEM (+8), ISO procedure-type syntax (+4), ISO SysClock (+2), and small fixes (+6: string-constant folding, ORD(bool) in CONST, HIGH/LEN of string CONSTs, ordinal INC/DEC, leading CASE |, __FUNCTION__). Corpus is 511/612 at the stop point. The remaining LANG entries are mostly: declaration order cases V3 rejects by policy (docs/gaps/); ISO exceptions (M3 libraries/runtime; M4 compiler traps pending — see docs/design-exceptions.md); multi-dimensional open arrays (representation, see docs/handoff-2026-10-23-next.md); and gm2-only extensions. See docs/analysis_gaps.md.
2. The size-sensitive V3 miscompile — root-caused and fixed (2026-10-19; see docs/summary_l3_selfhost_fix.md). It was never size-sensitive: the fixpoint gate matched the literal string "Parsed correctly" that the ASTDUMP debug dump printed unconditionally, so the gate passed while the compiler session was actually reporting Incorrect source (Lower's forward procedure references → invalid call). Removing ASTDUMP merely removed the masking line, which looked like a size effect. Fixed by a3ef897 (remove ASTDUMP; gate on Parsed correctly; declare Lower's walkers in Lower.def). The gate now fails on errors (verified with t_fwdvar), and self-hosting stays byte-identical.
3. Robustness — table-limit overflows are now loud (2026-10-19): the two-phase span table (which silently dropped a body's codegen, t_manyproc), the label/GOTO tables, the 32-actual AST buffer and SymTab's MaxTypes all report 230 (implementation limit) instead of truncating silently. Still weak: error recovery and message polish.
4. Library holes — the channel framework grew ChanConsts and StreamFile (ISO 10514-1) on top of IOChan, and the PIM/gm2 I/O family gained NumberIO, RealInOut, FpuIO (StrIO/InOut already existed). Also fixed the corpus harness fidelity: it now searches V3's own shipped libraries (stdlib, runtime/syslib) via a default $V3_LIBS, so a file importing e.g. Strings resolves against the real .def instead of being mislabelled LIB. Building StreamFile also fixed a real bug: a set CONST exported from a definition module is now emitted under its global name (<unit>_<name>), so a client's qualified reference binds. SysClock now implements the ISO 10514-1 API (DateTime, CanGetClock, GetClock, IsValidDateTime, CanSetClock/ SetClock as an unsupported no-op) on top of the shim's time calls, with Time/Clock kept as PIM-style extras. Still missing: gm2 runtime modules (M2RTS, Builtins, Processes, …).
5. Housekeeping — docs/INDEX.md is the entry point; the unreferenced historical records are under docs/archive/ (2026-10-19). resolve-check was merged to master.

How to check for yourself

cd compiler
./build.sh          # M2.atg -> scanner/parser/driver -> ./M2
./run_tests.sh      # 272/272
cd ..
./bootstrap/fixpoint.sh   # FIXPOINT OK (byte-identical stage2/stage3)