# m2compiler-V3 — status (2026-10-07)

This answers "where are we after all the refactoring?".

## MILESTONE (2026-10-07): the switch is flipped

**Lower is now the only active emitter.**  A plain `./M2` run emits
the image through `Lower` from the AST; the grammar's inline emit is
still present but runs suppressed (`SetNoEmit`) and is discarded, so
it no longer reaches the buffer.  Verified:

- `run_tests.sh` is **204/204** (the `expect_run` tests now exercise
  Lower end to end through qbe/cc/execution).
- `bootstrap/fixpoint.sh` is **FIXPOINT OK** (3,553,701 bytes).

The `-lower` flag is now test-only: it does not change *what* is
emitted, only names the image `<mod>L.ssa` for the byte-compare
harness.  The byte-identity milestone (Lower == legacy inline on every
single-file test and every session) is what made this a no-op flip.

The remaining work is purely mechanical: delete the grammar's inline
emit (533 `QbeGen.*` call sites), retire the `-lower` flag and the
now-degenerate byte-compare tests, then remove the dead
`LowerSession`/`MaterializeGlobals` scaffolding.

## TL;DR

- **The compiler works and self-hosts today.**  `run_tests.sh` is
  **204/204** and `bootstrap/fixpoint.sh` is **FIXPOINT OK**
  (3,553,701 bytes, byte-identical stage2/stage3).
- The **two-phase refactor (L0–L7) is done**: `Lower` reproduces the
  whole compiler image byte-for-byte from the AST, so flipping the
  emitter was a no-op.
- **Lower is the only active emitter.**  The grammar's inline emit is
  retained but suppressed; deleting it (and the `-lower` test flag) is
  the remaining cleanup.

## What the refactor actually is

There are **two ways** the compiler can turn Modula-2 into QBE IR:

1. **Legacy "inline" emit** — the grammar emits QBE IR *while* it
   parses.  This used to be the default; it now only runs suppressed
   (its output is discarded) and is being deleted.
2. **Lower (two-phase)** — the grammar builds an **AST**, and a
   separate module `Lower` walks the AST and emits QBE IR.  This is
   now the **only active emitter** ("the flip").

The refactor is being done in slices:

| phase | what | state |
|---|---|---|
| L0–L2 | scalars, control flow (`IF`/`WHILE`/`FOR`/…) | done |
| L3 | procedures, calls, the self-host root-cause fix | done |
| L4 | builtins, `CASE`, `WITH` | done |
| L5 | records/arrays/pointers/sets/strings, literals | done |
| L6 | nested modules, classes, qualified names | done |
| **L7** | **the flip: Lower emits the whole session** | **done** (flip landed 2026-10-07) |

Everything is committed in small steps; the many commits are
**incremental fixes to the new `Lower` path**, not changes to the
working legacy compiler.

## Where exactly we are (L7)

The flip is driven per unit: the driver suppresses the grammar's
inline emit (`SetNoEmit`), snapshots the QbeGen state before each
unit's parse, restores it, and lets `Lower` emit the unit from the
AST.  This is now the *only* path — plain `./M2` runs it, and the
self-hosting fixpoint runs it too.  `EndSession` writes the single
session image.

The `-lower` flag survives only for the test harness: it names the
image `<mod>L.ssa` instead of `<mod>.ssa` so `run_tests.sh` can still
byte-compare two runs.  Both runs now use Lower, so the comparison is
degenerate; it will be retired together with the inline emit.

Because `Lower` was verified byte-identical to the legacy inline emit
on every single-file test and every session, flipping the default did
not change any emitted image.

## What L7 fixed so far

- `Lower` is **V3-legal** (forward references declared in `Lower.def`;
  V3 has no whole-module pass) and the `SymTab.FormalOk` open-array
  def/impl bug.
- Whole-session tables + an open-addressing **hash index** for
  `FindSym`/`FindProc`.
- **Symbol scoping**: a procedure's locals no longer collide with
  another procedure's (`HIGH(s)` picked the wrong `s`).
- **Chunked sequences** where a list can exceed `AST.MaxChild`:
  `CASE` arms, multi-name `VAR` declarations, `CASE` bodies.
- External procedures, indirect calls (proc-typed variables), `VAL`,
  `LEN`/`HIGH`, qualified calls, qualified proc-typed variables,
  const values, 1-char strings passed as `ARRAY OF CHAR`.
- Per-unit global materialisation (`MaterializeReplace`).
- Duplicate nested procedure names resolve by scope and uid.

## What remains

Cleanup with no codegen gaps left.  **Not** purely mechanical: many
`QbeGen.*` calls are semantic queries or parser state, not emit, and
must stay — `TopLoop`/`PushLoop`/`PopLoop` (validates `EXIT`),
`TopWith`/`PushWith`/`PopWith` (validates a bare field name in `WITH`),
`IsImm`/`Fold2`/`NormInt`/`NormReal`/`StrFold`/… (constant values that
`Lower` consumes via `NoteConstVal`), `LocFull`, `Mangled`, `ConstInt`.
Only the true emission calls are removed, production group by group,
with suite + fixpoint after each.

Done:
1. Builtin statements (`EXIT`, `INCL/EXCL`, `INC/DEC`, `NEW`, `DISPOSE`).
2. Control flow (`IF`/`WHILE`/`REPEAT`/`LOOP`/`FOR`/`CASE`), `RETURN`,
   `HALT`.
3. Declarations/lifecycle: `DeclVar`/`DeclConst`, the module/impl/
   nested/class `BeginInit`/`EndInit`/`BeginBody`/`EndModule`,
   `FlushPend`, `FwdPatch`.  (This exposed and fixed a real bug — see
   below.)

Still to do:
4. **Expressions/designators/calls: DONE (2026-10-08).**  The runtime
   emit is gone from `SimExpr`, `Term`, `Expr`, `Fact`, `Design`,
   `ResultComp`, `ArgList`, `ActParam` and assignment; a runtime result
   is the non-constant sentinel `"@"` (which `IsImm`/`ConstInt`
   reject), while constant folding (`NegFold`/`Fold2`/`StrFold`) and
   every check stay.
   The coupled change that had blocked this landed together: `Fact`
   drops the `NoteAddr` chain, `ActParam` checks VAR-actual
   addressability by AST kind (`NkDesignator`, or `NkStrLit`) instead
   of the `NoteAddr`/`AddrOfVal` chain and drops `CallArg`, and
   `ArgList` drops `CallBegin`/`CallEnd`.  (`NoteAddr`/`AddrOfVal`
   remain, now used only by `Lower` for its own emit.)
5. **Proc headings, set/ctor literals, string pooling: remaining.**
   `ProcHeading` still calls `BeginFunc`/`FuncParam`/`EndFuncHeader`/
   `EndFunc`/`Mangled`/`SetFuncRes`/`AbortFunc`; note `FuncParam` is a
   check that relies on `BeginFunc` resetting `nPar`, so these must be
   split rather than dropped.  `SetLit`/`TypedBraceLit`
   (`NewSetTemp`/`SetZero`/`SetRange`/`SetBit`/`Ctor*`) and the
   `DeclStr`/`DeclUStr` string pool are value-ish; `CopyOp`/`IsImm`/
   `IntStr`/`Fold2`/`Norm*`/`Conv*` are the value layer and stay.
6. Retire the `-lower` test flag and the now-degenerate byte-compare
   tests.  **Done 2026-10-08:** `-lower`/`IsLowerParam`/`lowerTest` and
   `EndSession`'s `testSuffix` are gone, and the 23 `lower_ok` cases
   dropped (suite now **181**; the programs are still covered by the
   `expect_run`/session cases).
7. Remove the dead scaffolding.  Done 2026-10-08: `LowerSession`,
   `LowerUnit`, the whole `Can*` capability-walker family,
   `CanLower`/`CanExpr`/`CanDecls`/`CanProc`, the dead
   `SetLower`/`lowerRequested`, plus every compiler proc a tree-wide
   scan (tests included) finds unreferenced — `QbeGen`'s
   `CloseModule`/`FwdPatch`/`FwdElemPatch`/`Remark`/`StrLen`,
   `SymTab`'s `CurModule`/`CurUnit`/`ProcDepth`/`ParamName`/
   `ModDefined`/`ModImplemented`/`Qual*`/`NestArray`/`NewSub`/
   `SetElemCheck`/`SetFor`/`UnaryCheck`/`FwdVarKind`, and `AST`'s
   `TxtLen`/`SetKind` (`AST.AddChild` stays; `tests/t_ast.mod` uses it).
   Still needed until the inline emit is gone:
   `SaveSession`/`RestoreSession`/`SetNoEmit` (the grammar still emits
   suppressed) and `MaterializeGlobals`/`MaterializeReplace` (called
   from `LowerOneUnit`).

Note: full deletion to zero is gated on moving constant folding into
`Lower` (item 4).  Until then the expression/value layer stays.

### Fixed (2026-10-08): the note ring leaked the grammar's notes into Lower

Removing `QbeGen.DeclVar` from `VarDecl` had made `Lower` resolve one
VAR actual to the wrong symbol (`Showcase11`: `Strings.Replace`'s `src`
became `$Strings_nl`).  Root cause: `QbeGen.AddrOfVal` scans the whole
256-entry `nVal`/`nAddr` VAR-actual note ring, and `RestoreSession`
restored `nn` but never cleared the ring — so the grammar's suppressed
parse-time `NoteAddr` entries survived into `Lower`.  Their keys are
temps that restart per function, so a stale entry could false-match
`Lower`'s own key.  Removing `DeclVar` (fewer `AllocLocal`/`NewTemp`
calls) shifted those keys and exposed it.  `RestoreSession` now clears
the ring; `Lower` re-notes its own VAR actuals, so an empty ring is
correct.  The fix is output-neutral on `Showcase11` and corrects some
latent resolutions in the compiler's own image (fixpoint size moved).


## Are we near completion?

- **Of the working compiler:** yes — it is done, green, and
  self-hosting, and it now emits through `Lower`.
- **Of the L7 flip:** the flip is done; only the mechanical deletion of
  the unused inline emitter remains.  It is not required for the
  compiler to function — it is a quality/architecture goal.

## How to check for yourself

```sh
cd compiler
./build.sh          # rebuild
./run_tests.sh      # 204/204
cd ..
bootstrap/fixpoint.sh   # FIXPOINT OK
```