M2compiler-V3/docs/STATUS.md

m2compiler-V3 — status (2026-10-07)

This answers "where are we after all the refactoring?".

MILESTONE (2026-10-07): the switch is flipped

Lower is now the only active emitter. A plain ./M2 run emits the image through Lower from the AST; the grammar's inline emit is still present but runs suppressed (SetNoEmit) and is discarded, so it no longer reaches the buffer. Verified:

• run_tests.sh is 204/204 (the expect_run tests now exercise Lower end to end through qbe/cc/execution).
• bootstrap/fixpoint.sh is FIXPOINT OK (3,553,701 bytes).

The -lower flag is now test-only: it does not change what is emitted, only names the image <mod>L.ssa for the byte-compare harness. The byte-identity milestone (Lower == legacy inline on every single-file test and every session) is what made this a no-op flip.

The remaining work is purely mechanical: delete the grammar's inline emit (533 QbeGen.* call sites), retire the -lower flag and the now-degenerate byte-compare tests, then remove the dead LowerSession/MaterializeGlobals scaffolding.

TL;DR

• The compiler works and self-hosts today. run_tests.sh is 204/204 and bootstrap/fixpoint.sh is FIXPOINT OK (3,553,701 bytes, byte-identical stage2/stage3).
• The two-phase refactor (L0–L7) is done: Lower reproduces the whole compiler image byte-for-byte from the AST, so flipping the emitter was a no-op.
• Lower is the only active emitter. The grammar's inline emit is retained but suppressed; deleting it (and the -lower test flag) is the remaining cleanup.

What the refactor actually is

There are two ways the compiler can turn Modula-2 into QBE IR:

1. Legacy "inline" emit — the grammar emits QBE IR while it parses. This used to be the default; it now only runs suppressed (its output is discarded) and is being deleted.
2. Lower (two-phase) — the grammar builds an AST, and a separate module Lower walks the AST and emits QBE IR. This is now the only active emitter ("the flip").

The refactor is being done in slices:

phase	what	state
L0–L2	scalars, control flow (IF/WHILE/FOR/…)	done
L3	procedures, calls, the self-host root-cause fix	done
L4	builtins, CASE, WITH	done
L5	records/arrays/pointers/sets/strings, literals	done
L6	nested modules, classes, qualified names	done
L7	the flip: Lower emits the whole session	done (flip landed 2026-10-07)

Everything is committed in small steps; the many commits are incremental fixes to the new Lower path, not changes to the working legacy compiler.

Where exactly we are (L7)

The flip is driven per unit: the driver suppresses the grammar's inline emit (SetNoEmit), snapshots the QbeGen state before each unit's parse, restores it, and lets Lower emit the unit from the AST. This is now the only path — plain ./M2 runs it, and the self-hosting fixpoint runs it too. EndSession writes the single session image.

The -lower flag survives only for the test harness: it names the image <mod>L.ssa instead of <mod>.ssa so run_tests.sh can still byte-compare two runs. Both runs now use Lower, so the comparison is degenerate; it will be retired together with the inline emit.

Because Lower was verified byte-identical to the legacy inline emit on every single-file test and every session, flipping the default did not change any emitted image.

What L7 fixed so far

• Lower is V3-legal (forward references declared in Lower.def; V3 has no whole-module pass) and the SymTab.FormalOk open-array def/impl bug.
• Whole-session tables + an open-addressing hash index for FindSym/FindProc.
• Symbol scoping: a procedure's locals no longer collide with another procedure's (HIGH(s) picked the wrong s).
• Chunked sequences where a list can exceed AST.MaxChild: CASE arms, multi-name VAR declarations, CASE bodies.
• External procedures, indirect calls (proc-typed variables), VAL, LEN/HIGH, qualified calls, qualified proc-typed variables, const values, 1-char strings passed as ARRAY OF CHAR.
• Per-unit global materialisation (MaterializeReplace).
• Duplicate nested procedure names resolve by scope and uid.

What remains

Cleanup with no codegen gaps left. Not purely mechanical: many QbeGen.* calls are semantic queries or parser state, not emit, and must stay — TopLoop/PushLoop/PopLoop (validates EXIT), TopWith/PushWith/PopWith (validates a bare field name in WITH), IsImm/Fold2/NormInt/NormReal/StrFold/… (constant values that Lower consumes via NoteConstVal), LocFull, Mangled, ConstInt. Only the true emission calls are removed, production group by group, with suite + fixpoint after each.

Done:

1. Builtin statements (EXIT, INCL/EXCL, INC/DEC, NEW, DISPOSE).
2. Control flow (IF/WHILE/REPEAT/LOOP/FOR/CASE), RETURN, HALT.
3. Declarations/lifecycle: DeclVar/DeclConst, the module/impl/ nested/class BeginInit/EndInit/BeginBody/EndModule, FlushPend, FwdPatch. (This exposed and fixed a real bug — see below.)

Still to do:

1. Expressions/designators/calls: DONE (2026-10-08). The runtime emit is gone from SimExpr, Term, Expr, Fact, Design, ResultComp, ArgList, ActParam and assignment; a runtime result is the non-constant sentinel "@" (which IsImm/ConstInt reject), while constant folding (NegFold/Fold2/StrFold) and every check stay. The coupled change that had blocked this landed together: Fact drops the NoteAddr chain, ActParam checks VAR-actual addressability by AST kind (NkDesignator, or NkStrLit) instead of the NoteAddr/AddrOfVal chain and drops CallArg, and ArgList drops CallBegin/CallEnd. (NoteAddr/AddrOfVal remain, now used only by Lower for its own emit.)
2. Proc headings, set/ctor literals, string pooling: remaining. ProcHeading still calls BeginFunc/FuncParam/EndFuncHeader/ EndFunc/Mangled/SetFuncRes/AbortFunc; note FuncParam is a check that relies on BeginFunc resetting nPar, so these must be split rather than dropped. SetLit/TypedBraceLit (NewSetTemp/SetZero/SetRange/SetBit/Ctor*) and the DeclStr/DeclUStr string pool are value-ish; CopyOp/IsImm/ IntStr/Fold2/Norm*/Conv* are the value layer and stay.
3. Retire the -lower test flag and the now-degenerate byte-compare tests. Done 2026-10-08: -lower/IsLowerParam/lowerTest and EndSession's testSuffix are gone, and the 23 lower_ok cases dropped (suite now 181; the programs are still covered by the expect_run/session cases).
4. Remove the dead scaffolding. Done 2026-10-08: LowerSession, LowerUnit, the whole Can* capability-walker family, CanLower/CanExpr/CanDecls/CanProc, the dead SetLower/lowerRequested, plus every compiler proc a tree-wide scan (tests included) finds unreferenced — QbeGen's CloseModule/FwdPatch/FwdElemPatch/Remark/StrLen, SymTab's CurModule/CurUnit/ProcDepth/ParamName/ ModDefined/ModImplemented/Qual*/NestArray/NewSub/ SetElemCheck/SetFor/UnaryCheck/FwdVarKind, and AST's TxtLen/SetKind (AST.AddChild stays; tests/t_ast.mod uses it). Still needed until the inline emit is gone: SaveSession/RestoreSession/SetNoEmit (the grammar still emits suppressed) and MaterializeGlobals/MaterializeReplace (called from LowerOneUnit).

Note: full deletion to zero is gated on moving constant folding into Lower (item 4). Until then the expression/value layer stays.

Fixed (2026-10-08): the note ring leaked the grammar's notes into Lower

Removing QbeGen.DeclVar from VarDecl had made Lower resolve one VAR actual to the wrong symbol (Showcase11: Strings.Replace's src became $Strings_nl). Root cause: QbeGen.AddrOfVal scans the whole 256-entry nVal/nAddr VAR-actual note ring, and RestoreSession restored nn but never cleared the ring — so the grammar's suppressed parse-time NoteAddr entries survived into Lower. Their keys are temps that restart per function, so a stale entry could false-match Lower's own key. Removing DeclVar (fewer AllocLocal/NewTemp calls) shifted those keys and exposed it. RestoreSession now clears the ring; Lower re-notes its own VAR actuals, so an empty ring is correct. The fix is output-neutral on Showcase11 and corrects some latent resolutions in the compiler's own image (fixpoint size moved).

Are we near completion?

• Of the working compiler: yes — it is done, green, and self-hosting, and it now emits through Lower.
• Of the L7 flip: the flip is done; only the mechanical deletion of the unused inline emitter remains. It is not required for the compiler to function — it is a quality/architecture goal.

How to check for yourself

cd compiler
./build.sh          # rebuild
./run_tests.sh      # 204/204
cd ..
bootstrap/fixpoint.sh   # FIXPOINT OK