M2compiler-V3/docs/summary_lower_l7.md

Lower phase — L7 (the flip): status and plan (2026-10-05)

Branch ast-stage-c. L7 = "flip: lower from the AST instead of inline; delete the inline emit + Fwd* machinery".

Why L7 is not incremental

The flip makes Lower the sole emitter. The obstacle is that QbeGen holds one session state, used by the inline emit during parse; a per-unit Lower emit would be clobbered by the next unit's inline emit. So the flip is a big-bang: either remove/suppress the grammar's 533 QbeGen.* call sites, or give QbeGen two states — neither is a byte-verifiable step.

The pragmatic path is therefore a session re-emit: collect every unit's AST and re-emit the whole session with Lower after parsing. That is byte-comparable against the legacy session image and becomes the flip once the inline emit is removed.

Started (d30abfd, tag v3-lower7-scaffold)

• Lower.LowerSession/LowerOneUnit/MaterializeGlobals.
• Driver (compiler.frm) collects GetUnit() per file and calls Lower.LowerSession under -lower.
• SymTab.Materialize makes a unit's own VAR/CONST visible in globScope, so GlobalRef resolves "<unit>_<name>" at session end (the unit scopes are closed by then).
• Removed Lower.Reset from the M2 root so the owned tables persist across a session.

Still scaffolding (inline emit untouched); suite 200/200, FIXPOINT OK (3,401,129 bytes).

Progress (ce2fc67, tag v3-lower7-session)

• DefUnit now builds an AST (proc headings recorded as NkProcDecl); LowerDecls replays a bodyless proc as a heading.
• Session re-emit works for small sessions: lower_ok_files added, and DBasicProg, DQualProg, TQual (def-CONST), TQualP (def-PROC) byte-match. Fixes: image named after the program module; MaterializeGlobals also materializes TYPE/CLASS names; SymTab.ReenterModGlobal keeps nested-module globals' "<outer>_<name>" symbols; AST.Init once per session under -lower.

Suite 204/204; FIXPOINT OK (3,405,813 bytes).

Remaining gaps

1. AST-arena size (fundamental). LowerSession needs every unit's AST alive at once, but AST.MaxNodes = 65536 is sized for one unit; the compiler's session overflows it (MakeNode → NoNode, later units dump (nil)). Growing the arena past the 4 MiB sessBuf is not viable. So the session re-emit cannot handle the compiler — the flip must instead emit per unit at each unit's END (AST valid, scope open), which is the two-state / inline-emit-removal problem.
2. External procs. storage_prog differs: Lower emits $malloc_6(l link, ...) where the legacy emits $malloc(...) — the ext flag in Lower.NoteProc is hardcoded FALSE and Mangled's ProcLink fails after scopes close.
3. Definition-module proc replay (done), nested globals (done).

Per-unit emission attempted — blocked by parse-time state

Tried plan item 3 (Lower emits each unit at its END, with the legacy output suppressed via SetNoEmit). Small sessions and every lower_ok single-file test byte-match, but the compiler's session hangs/differs:

• The grammar's inline QbeGen calls still run during parse; noEmit only suppresses the output, not the state. So nTemp (stuck at 211), inBody (stuck TRUE, so main's header is dropped), nLoc, funcDepth, … are all corrupted before Lower's per-unit emit.
• EndModule had to respect noEmit too, or it closed the session.

Conclusion: per-unit Lower needs the grammar's parse-time QbeGen state neutralised — which is exactly the flip. Options:

1. Remove/replace the grammar's 533 QbeGen.* calls. The blockers are the ~20 semantic checks that read QbeGen values (IsImm, ConstInt, Fold); they must be reworked to use SymTab/the AST.
2. Two QbeGen states — but a second 4 MiB sessBuf pushes the compiler image past its own buffer, so this does not self-host.

So the flip is a grammar rewrite of the emit actions, gated by the fixpoint. Reverted to the green session-re-emit state (ce2fc67).