# m2compiler-V3 — status (2026-10-06)

This answers "where are we after all the refactoring?".

## MILESTONE (2026-10-07)

**The L7 flip is byte-identical.**  Running the compiler on its own
sources under `-lower` now produces exactly the same image as the
legacy inline emit: **`cmp` reports MATCH**, both 3,510,051 bytes
(736/736 functions).  `run_tests.sh` is **204/204** and
`bootstrap/fixpoint.sh` is **FIXPOINT OK** (3,510,051 bytes).

The remaining work is the mechanical part of the flip: delete the
grammar's inline emit and the `-lower` flag, then remove the dead
`LowerSession`/`MaterializeGlobals` scaffolding.

## TL;DR

- **The compiler works and self-hosts today.**  `run_tests.sh` is
  **204/204** and `bootstrap/fixpoint.sh` is **FIXPOINT OK**
  (3,510,051 bytes, byte-identical stage2/stage3).
- The **two-phase refactor (L0–L7) now works end to end**: `Lower`
  reproduces the whole compiler image byte-for-byte from the AST.
- The flip is still **opt-in** (`-lower`) and does **not** affect the
  normal build; the final step is to make it the only emitter.

## What the refactor actually is

There are **two ways** the compiler can turn Modula-2 into QBE IR:

1. **Legacy "inline" emit** — the grammar emits QBE IR *while* it
   parses.  This is the default and is what the working compiler and
   the self-hosting fixpoint use.
2. **Lower (two-phase)** — the grammar builds an **AST**, and a
   separate module `Lower` walks the AST and emits QBE IR.  This is
   selected with the **`-lower`** flag and is the goal of the
   refactor ("the flip" = make Lower the only emitter and delete the
   inline emit).

The refactor is being done in slices:

| phase | what | state |
|---|---|---|
| L0–L2 | scalars, control flow (`IF`/`WHILE`/`FOR`/…) | done |
| L3 | procedures, calls, the self-host root-cause fix | done |
| L4 | builtins, `CASE`, `WITH` | done |
| L5 | records/arrays/pointers/sets/strings, literals | done |
| L6 | nested modules, classes, qualified names | done |
| **L7** | **the flip: Lower emits the whole session** | **in progress** |

Everything is committed in small steps; the many commits are
**incremental fixes to the new `Lower` path**, not changes to the
working legacy compiler.

## Where exactly we are (L7)

The flip is driven per unit: the driver suppresses the grammar's
inline emit (`SetNoEmit`), snapshots the QbeGen state before each
unit's parse, restores it, and lets `Lower` re-emit the unit.  On the
compiler's own sources:

- `-lower` emits the **full** image: **736 functions** (same as
  legacy), ~1.5 s.
- It is **~74 % byte-identical** before the first divergence:
  `first diff at byte 2,935,735, line 89,349` of `120,266`.
- Progress of the first divergence across the session:
  `line 9 → 327 → 1124 → 3437 → 6961 → 56294 → 70934 → 82194 → 86942 → 89349`.

The first divergence is now the generic grammar `ArgList` call, which
has **9 arguments** but `AST.MaxChild = 8` caps a call at 7, so `Lower`
drops the tail.

## What L7 fixed so far

- `Lower` is **V3-legal** (forward references declared in `Lower.def`;
  V3 has no whole-module pass) and the `SymTab.FormalOk` open-array
  def/impl bug.
- Whole-session tables + an open-addressing **hash index** for
  `FindSym`/`FindProc`.
- **Symbol scoping**: a procedure's locals no longer collide with
  another procedure's (`HIGH(s)` picked the wrong `s`).
- **Chunked sequences** where a list can exceed `AST.MaxChild`:
  `CASE` arms, multi-name `VAR` declarations, `CASE` bodies.
- External procedures, indirect calls (proc-typed variables), `VAL`,
  `LEN`/`HIGH`, qualified calls, qualified proc-typed variables,
  const values, 1-char strings passed as `ARRAY OF CHAR`.
- Per-unit global materialisation (`MaterializeReplace`).
- Duplicate nested procedure names resolve by scope and uid.

## What remains

1. **Calls with more than 7 arguments** (the `ArgList` case):
   chunk the call actuals and make the argument loops follow the
   `NkBlock` chunks.  (A prototype regressed `AstCallNode`'s legacy
   emit and was reverted.)
2. Whatever the next first-divergence reveals.  Historically each fix
   has exposed one more small gap; there may be a few more.
3. Only then: delete the grammar's inline emit (the literal flip) and
   remove the `-lower` flag.

## Are we near completion?

- **Of the working compiler:** yes — it is done, green, and
  self-hosting.  The refactor has not broken it.
- **Of the L7 flip:** the *hard architectural part is finished*
  (per-unit emit, scoping, tables, the AST chunking mechanism).  The
  remainder is a **tail of codegen gaps** that only matter because the
  verification is exact byte-identity over a 120k-line session.
  Realistically a handful more small fixes; not a rewrite.  It is not
  required for the compiler to function — it is a quality/architecture
  goal.

## How to check for yourself

```sh
cd compiler
./build.sh          # rebuild
./run_tests.sh      # 204/204
cd ..
bootstrap/fixpoint.sh   # FIXPOINT OK
```