M2compiler-V3/docs/STATUS.md

m2compiler-V3 — status (2026-10-07)

This answers "where are we after all the refactoring?".

MILESTONE (2026-10-07): the switch is flipped

Lower is now the only active emitter. A plain ./M2 run emits the image through Lower from the AST; the grammar's inline emit is still present but runs suppressed (SetNoEmit) and is discarded, so it no longer reaches the buffer. Verified:

• run_tests.sh is 204/204 (the expect_run tests now exercise Lower end to end through qbe/cc/execution).
• bootstrap/fixpoint.sh is FIXPOINT OK (3,553,701 bytes).

The -lower flag is now test-only: it does not change what is emitted, only names the image <mod>L.ssa for the byte-compare harness. The byte-identity milestone (Lower == legacy inline on every single-file test and every session) is what made this a no-op flip.

The remaining work is purely mechanical: delete the grammar's inline emit (533 QbeGen.* call sites), retire the -lower flag and the now-degenerate byte-compare tests, then remove the dead LowerSession/MaterializeGlobals scaffolding.

TL;DR

• The compiler works and self-hosts today. run_tests.sh is 204/204 and bootstrap/fixpoint.sh is FIXPOINT OK (3,553,701 bytes, byte-identical stage2/stage3).
• The two-phase refactor (L0–L7) is done: Lower reproduces the whole compiler image byte-for-byte from the AST, so flipping the emitter was a no-op.
• Lower is the only active emitter. The grammar's inline emit is retained but suppressed; deleting it (and the -lower test flag) is the remaining cleanup.

What the refactor actually is

There are two ways the compiler can turn Modula-2 into QBE IR:

1. Legacy "inline" emit — the grammar emits QBE IR while it parses. This used to be the default; it now only runs suppressed (its output is discarded) and is being deleted.
2. Lower (two-phase) — the grammar builds an AST, and a separate module Lower walks the AST and emits QBE IR. This is now the only active emitter ("the flip").

The refactor is being done in slices:

phase	what	state
L0–L2	scalars, control flow (IF/WHILE/FOR/…)	done
L3	procedures, calls, the self-host root-cause fix	done
L4	builtins, CASE, WITH	done
L5	records/arrays/pointers/sets/strings, literals	done
L6	nested modules, classes, qualified names	done
L7	the flip: Lower emits the whole session	done (flip landed 2026-10-07)

Everything is committed in small steps; the many commits are incremental fixes to the new Lower path, not changes to the working legacy compiler.

Where exactly we are (L7)

The flip is driven per unit: the driver suppresses the grammar's inline emit (SetNoEmit), snapshots the QbeGen state before each unit's parse, restores it, and lets Lower emit the unit from the AST. This is now the only path — plain ./M2 runs it, and the self-hosting fixpoint runs it too. EndSession writes the single session image.

The -lower flag survives only for the test harness: it names the image <mod>L.ssa instead of <mod>.ssa so run_tests.sh can still byte-compare two runs. Both runs now use Lower, so the comparison is degenerate; it will be retired together with the inline emit.

Because Lower was verified byte-identical to the legacy inline emit on every single-file test and every session, flipping the default did not change any emitted image.

What L7 fixed so far

• Lower is V3-legal (forward references declared in Lower.def; V3 has no whole-module pass) and the SymTab.FormalOk open-array def/impl bug.
• Whole-session tables + an open-addressing hash index for FindSym/FindProc.
• Symbol scoping: a procedure's locals no longer collide with another procedure's (HIGH(s) picked the wrong s).
• Chunked sequences where a list can exceed AST.MaxChild: CASE arms, multi-name VAR declarations, CASE bodies.
• External procedures, indirect calls (proc-typed variables), VAL, LEN/HIGH, qualified calls, qualified proc-typed variables, const values, 1-char strings passed as ARRAY OF CHAR.
• Per-unit global materialisation (MaterializeReplace).
• Duplicate nested procedure names resolve by scope and uid.

What remains

Cleanup with no codegen gaps left. Not purely mechanical: many QbeGen.* calls are semantic queries or parser state, not emit, and must stay — TopLoop/PushLoop/PopLoop (validates EXIT), TopWith/PushWith/PopWith (validates a bare field name in WITH), IsImm/Fold2/NormInt/NormReal/StrFold/… (constant values that Lower consumes via NoteConstVal), LocFull, Mangled, ConstInt. Only the true emission calls are removed, production group by group, with suite + fixpoint after each.

Done:

1. Builtin statements (EXIT, INCL/EXCL, INC/DEC, NEW, DISPOSE).
2. Control flow (IF/WHILE/REPEAT/LOOP/FOR/CASE), RETURN, HALT.

Still to do:

1. Expressions / Fact / Term (the big one: CopyOp, IntStr, Op3*, WidenLong, Cmp*, AddrOf*, Set*, folding), and designators.
2. Calls (ArgList/ActParam), declarations/proc headings, imports, classes.
3. Retire the -lower test flag and the now-degenerate byte-compare tests in run_tests.sh.
4. Remove the dead scaffolding: LowerSession, MaterializeGlobals, SaveSession/RestoreSession and the SetNoEmit machinery (once nothing emits inline), plus the dead SetLower/lowerRequested.

Are we near completion?

• Of the working compiler: yes — it is done, green, and self-hosting, and it now emits through Lower.
• Of the L7 flip: the flip is done; only the mechanical deletion of the unused inline emitter remains. It is not required for the compiler to function — it is a quality/architecture goal.

How to check for yourself

cd compiler
./build.sh          # rebuild
./run_tests.sh      # 204/204
cd ..
bootstrap/fixpoint.sh   # FIXPOINT OK