# m2compiler-V3 — status (2026-10-07)

This answers "where are we after all the refactoring?".

## MILESTONE (2026-10-07): the switch is flipped

**Lower is now the only active emitter.**  A plain `./M2` run emits
the image through `Lower` from the AST; the grammar's inline emit is
still present but runs suppressed (`SetNoEmit`) and is discarded, so
it no longer reaches the buffer.  Verified:

- `run_tests.sh` is **204/204** (the `expect_run` tests now exercise
  Lower end to end through qbe/cc/execution).
- `bootstrap/fixpoint.sh` is **FIXPOINT OK** (3,553,701 bytes).

The `-lower` flag is now test-only: it does not change *what* is
emitted, only names the image `<mod>L.ssa` for the byte-compare
harness.  The byte-identity milestone (Lower == legacy inline on every
single-file test and every session) is what made this a no-op flip.

The remaining work is purely mechanical: delete the grammar's inline
emit (533 `QbeGen.*` call sites), retire the `-lower` flag and the
now-degenerate byte-compare tests, then remove the dead
`LowerSession`/`MaterializeGlobals` scaffolding.

## TL;DR

- **The compiler works and self-hosts today.**  `run_tests.sh` is
  **204/204** and `bootstrap/fixpoint.sh` is **FIXPOINT OK**
  (3,553,701 bytes, byte-identical stage2/stage3).
- The **two-phase refactor (L0–L7) is done**: `Lower` reproduces the
  whole compiler image byte-for-byte from the AST, so flipping the
  emitter was a no-op.
- **Lower is the only active emitter.**  The grammar's inline emit is
  retained but suppressed; deleting it (and the `-lower` test flag) is
  the remaining cleanup.

## What the refactor actually is

There are **two ways** the compiler can turn Modula-2 into QBE IR:

1. **Legacy "inline" emit** — the grammar emits QBE IR *while* it
   parses.  This used to be the default; it now only runs suppressed
   (its output is discarded) and is being deleted.
2. **Lower (two-phase)** — the grammar builds an **AST**, and a
   separate module `Lower` walks the AST and emits QBE IR.  This is
   now the **only active emitter** ("the flip").

The refactor is being done in slices:

| phase | what | state |
|---|---|---|
| L0–L2 | scalars, control flow (`IF`/`WHILE`/`FOR`/…) | done |
| L3 | procedures, calls, the self-host root-cause fix | done |
| L4 | builtins, `CASE`, `WITH` | done |
| L5 | records/arrays/pointers/sets/strings, literals | done |
| L6 | nested modules, classes, qualified names | done |
| **L7** | **the flip: Lower emits the whole session** | **done** (flip landed 2026-10-07) |

Everything is committed in small steps; the many commits are
**incremental fixes to the new `Lower` path**, not changes to the
working legacy compiler.

## Where exactly we are (L7)

The flip is driven per unit: the driver suppresses the grammar's
inline emit (`SetNoEmit`), snapshots the QbeGen state before each
unit's parse, restores it, and lets `Lower` emit the unit from the
AST.  This is now the *only* path — plain `./M2` runs it, and the
self-hosting fixpoint runs it too.  `EndSession` writes the single
session image.

The `-lower` flag survives only for the test harness: it names the
image `<mod>L.ssa` instead of `<mod>.ssa` so `run_tests.sh` can still
byte-compare two runs.  Both runs now use Lower, so the comparison is
degenerate; it will be retired together with the inline emit.

Because `Lower` was verified byte-identical to the legacy inline emit
on every single-file test and every session, flipping the default did
not change any emitted image.

## What L7 fixed so far

- `Lower` is **V3-legal** (forward references declared in `Lower.def`;
  V3 has no whole-module pass) and the `SymTab.FormalOk` open-array
  def/impl bug.
- Whole-session tables + an open-addressing **hash index** for
  `FindSym`/`FindProc`.
- **Symbol scoping**: a procedure's locals no longer collide with
  another procedure's (`HIGH(s)` picked the wrong `s`).
- **Chunked sequences** where a list can exceed `AST.MaxChild`:
  `CASE` arms, multi-name `VAR` declarations, `CASE` bodies.
- External procedures, indirect calls (proc-typed variables), `VAL`,
  `LEN`/`HIGH`, qualified calls, qualified proc-typed variables,
  const values, 1-char strings passed as `ARRAY OF CHAR`.
- Per-unit global materialisation (`MaterializeReplace`).
- Duplicate nested procedure names resolve by scope and uid.

## What remains

Cleanup with no codegen gaps left.  **Not** purely mechanical: many
`QbeGen.*` calls are semantic queries or parser state, not emit, and
must stay — `TopLoop`/`PushLoop`/`PopLoop` (validates `EXIT`),
`TopWith`/`PushWith`/`PopWith` (validates a bare field name in `WITH`),
`IsImm`/`Fold2`/`NormInt`/`NormReal`/`StrFold`/… (constant values that
`Lower` consumes via `NoteConstVal`), `LocFull`, `Mangled`, `ConstInt`.
Only the true emission calls are removed, production group by group,
with suite + fixpoint after each.

Done:
1. Builtin statements (`EXIT`, `INCL/EXCL`, `INC/DEC`, `NEW`, `DISPOSE`).
2. Control flow (`IF`/`WHILE`/`REPEAT`/`LOOP`/`FOR`/`CASE`), `RETURN`,
   `HALT`.
3. Declarations/lifecycle: `DeclVar`/`DeclConst`, the module/impl/
   nested/class `BeginInit`/`EndInit`/`BeginBody`/`EndModule`,
   `FlushPend`, `FwdPatch`.  (This exposed and fixed a real bug — see
   below.)

Still to do:
4. **Expressions: PARTIALLY DONE (2026-10-08).**  The runtime emit
   has been removed from `SimExpr`, `Term`, `Expr` and `Fact`'s
   always-runtime branches; a runtime result is the non-constant
   sentinel `"@"` (which `IsImm`/`ConstInt` reject), while constant
   folding (`NegFold`/`Fold2`/`StrFold`) and every check stay.  This
   works because those productions' values only feed further value ops
   and constant checks.
   **The rest is one coupled change:** `Design` → `Fact` → `ArgList` →
   `ActParam` plus the `NoteAddr`/`AddrOfVal` chain.  `Fact` records
   `NoteAddr(q, qd)`/`AddrOf(qn, qa)` and `ActParam` recovers the
   VAR-actual address with `AddrOfVal(qe, qa)` — and the note key is the
   actual's *emitted* value, so dropping `ArgList`'s `CallBegin` alone
   (fewer `NewTemp`s) shifts the keys and breaks calls (verified: 75
   suite failures).  To finish: `Design` emits nothing (literal for a
   constant, `"@"` otherwise), `Fact` drops the notes, `ActParam`
   replaces `AddrOfVal` with an AST-kind addressability check and drops
   `CallArg`, then `ArgList` drops `CallBegin`/`CallEnd`; afterwards
   `NoteAddr`/`AddrOfVal`/`AddrOf` in `QbeGen` are dead.
5. **Calls, proc headings, imports, classes: BLOCKED too, by fused
   validation.**  The `QbeGen` emit functions double as semantic
   checks and the grammar depends on their result: `QbeGen.CallArg`
   (`IF NOT … THEN SemError(233)`), `AddrOfVal` (VAR-actual address +
   233), `FuncParam` (`IF NOT …` on a heading).  Removing the calls
   would remove error detection, so this needs the checks
   reimplemented in `SymTab`/the grammar first — again a real change,
   not cleanup.
6. Retire the `-lower` test flag and the now-degenerate byte-compare
   tests in `run_tests.sh`.
7. Remove the dead scaffolding.  Done 2026-10-08: `LowerSession`,
   `LowerUnit`, the whole `Can*` capability-walker family,
   `CanLower`/`CanExpr`/`CanDecls`/`CanProc`, the dead
   `SetLower`/`lowerRequested`, plus every compiler proc a tree-wide
   scan (tests included) finds unreferenced — `QbeGen`'s
   `CloseModule`/`FwdPatch`/`FwdElemPatch`/`Remark`/`StrLen`,
   `SymTab`'s `CurModule`/`CurUnit`/`ProcDepth`/`ParamName`/
   `ModDefined`/`ModImplemented`/`Qual*`/`NestArray`/`NewSub`/
   `SetElemCheck`/`SetFor`/`UnaryCheck`/`FwdVarKind`, and `AST`'s
   `TxtLen`/`SetKind` (`AST.AddChild` stays; `tests/t_ast.mod` uses it).
   Still needed until the inline emit is gone:
   `SaveSession`/`RestoreSession`/`SetNoEmit` (the grammar still emits
   suppressed) and `MaterializeGlobals`/`MaterializeReplace` (called
   from `LowerOneUnit`).

Note: full deletion to zero is gated on moving constant folding into
`Lower` (item 4).  Until then the expression/value layer stays.

### Fixed (2026-10-08): the note ring leaked the grammar's notes into Lower

Removing `QbeGen.DeclVar` from `VarDecl` had made `Lower` resolve one
VAR actual to the wrong symbol (`Showcase11`: `Strings.Replace`'s `src`
became `$Strings_nl`).  Root cause: `QbeGen.AddrOfVal` scans the whole
256-entry `nVal`/`nAddr` VAR-actual note ring, and `RestoreSession`
restored `nn` but never cleared the ring — so the grammar's suppressed
parse-time `NoteAddr` entries survived into `Lower`.  Their keys are
temps that restart per function, so a stale entry could false-match
`Lower`'s own key.  Removing `DeclVar` (fewer `AllocLocal`/`NewTemp`
calls) shifted those keys and exposed it.  `RestoreSession` now clears
the ring; `Lower` re-notes its own VAR actuals, so an empty ring is
correct.  The fix is output-neutral on `Showcase11` and corrects some
latent resolutions in the compiler's own image (fixpoint size moved).


## Are we near completion?

- **Of the working compiler:** yes — it is done, green, and
  self-hosting, and it now emits through `Lower`.
- **Of the L7 flip:** the flip is done; only the mechanical deletion of
  the unused inline emitter remains.  It is not required for the
  compiler to function — it is a quality/architecture goal.

## How to check for yourself

```sh
cd compiler
./build.sh          # rebuild
./run_tests.sh      # 204/204
cd ..
bootstrap/fixpoint.sh   # FIXPOINT OK
```