M2compiler-V3/docs/plan-lower.md

Lower phase — design and plan (docs/plan-lower.md)

Branch ast-stage-c. The inert AST frontend is complete (slices 1–13): the parser builds a full tree alongside the legacy emit, which is unchanged. This document is the plan of record for consuming that tree.

Goal

Move code generation out of the grammar actions into a tree walk (Lower) that runs after a unit's declarations are complete in SymTab, then delete the inline emit and the forward-patching machinery (QbeGen.FwdPatchAll/FwdDesignator/FixLoadClass/FixStoreClass, the grammar's FwdVarNote/FwdVarFlush, the Design forward branch).

Why: lowering-during-parse is what forces forward references to be patched by hand; with the whole tree available, forward references resolve naturally and the patches disappear.

Mechanism (test-only first)

Lower re-emits a unit through the existing QbeGen API into a fresh backend state and writes a second image, so the old and new paths can be byte-compared:

1. QbeGen.OpenModule(name) fully resets the backend (counters, buffers) and writes the header — confirmed clean for a second emit.
2. Lower.LowerUnit(unit, modName): OpenModule(modName ++ "L"), SetModule(modName), emit declarations, BeginBody, walk the body, EndModule(modName ++ "L"). The module symbol name stays the original, so mangled symbols match; only the output file name differs.
3. The driver's -lower flag runs step 2 after a normal parse; a shell test compares gen_ssa/<Prog>L.ssa with gen_ssa/<Prog>.ssa.

Because the legacy parse still runs, the suite and fixpoint are unaffected (-lower is opt-in).

Why byte-compare is achievable

The emit is deterministic: fresh temps %tN and labels @LN come from counters reset by OpenModule. If the walk issues the same QbeGen calls in the same order as the grammar did, the image is byte-identical. Lower therefore skips the checks (already run during parse) and only replays the emit — driven by the AST structure plus SymTab (for types/kinds) and the node ty field.

Subset order (each a gate: suite + fixpoint + cmp on chosen tests)

Step	Subset	Notes
L0	program unit; scalar CONST/VAR; body of x := <int expr>; int literals/idents, + - * DIV MOD, unary -	mechanism proof; t_exit, t_arith
L1	relations, booleans, short-circuit AND/OR (Delay), NOT	t_lower1; docs/summary_two-phase-lower1.md
L2	control flow: IF/ELSIF/ELSE, WHILE, REPEAT, LOOP/EXIT, FOR (no BY), HALT	t_lower2; docs/summary_two-phase-lower2.md
L3	procedures/functions: BeginFunc/params/locals/CallBegin…
L4	calls, builtins, WITH, CASE
L5	arrays/sets/records/pointers/strings
L6	classes/vtables, imports, nested modules
L7	flip: lower from the AST instead of inline; delete the inline emit + Fwd* machinery	fixpoint is the hard gate

Everything is gated by run_tests.sh and fixpoint.sh; a subset that does not yet cmp stays out of CanLower so it is skipped.

Design constraints

• Lower is a new module (compiler/src/Lower.def/.mod), linked like AST; it imports AST, SymTab, QbeGen.
• It must not use local arrays until the local-array miscompile is fixed (docs/wip/local-array-bug.mod).
• CanLower(unit) gates the comparison: only fully-supported units are lowered, so the harness never reports a false mismatch.
• Sequences are chunked (AstAppend): a walk iterates children 0..MaxChild-2 then follows child[MaxChild-1].

Entry point / driver

• Grammar exports GetUnit(): AST.Node (the last parsed unit's root).
• compiler.frm accepts -lower, and after a normal parse calls Lower.LowerUnit(M2P.GetUnit(), progName).
• run_tests.sh gains a lower_ok <prog> helper: run ./M2 -lower, then cmp gen_ssa/<Prog>L.ssa gen_ssa/<Prog>.ssa.

Risks

• Order divergence (temps/labels) → cmp fails; fix the walk order.
• Symbol-state drift — Lower reads SymTab/node ty; keep it read-only.
• Scope creep — one subset per increment; keep CanLower strict.