|
|
@@ -0,0 +1,115 @@
|
|
|
+# m2compiler-V3 — status (2026-10-06)
|
|
|
+
|
|
|
+This answers "where are we after all the refactoring?".
|
|
|
+
|
|
|
+## TL;DR
|
|
|
+
|
|
|
+- **The compiler works and self-hosts today.** `run_tests.sh` is
|
|
|
+ **204/204** and `bootstrap/fixpoint.sh` is **FIXPOINT OK**
|
|
|
+ (3,758,751 bytes, byte-identical stage2/stage3).
|
|
|
+- The **two-phase refactor (L0–L6) is done and committed**. The last
|
|
|
+ step, **L7 "the flip"**, is **in progress** and is **opt-in**
|
|
|
+ (`-lower`); it does **not** affect the normal build.
|
|
|
+- The flip already produces the full compiler image (736/736
|
|
|
+ functions), but it is only **~74 % byte-identical** to the legacy
|
|
|
+ image: the first difference is at line 89,349 of 120,266.
|
|
|
+- **We are not near completion of the flip.** The mechanism is done;
|
|
|
+ what remains is closing the last code-generation gaps until the
|
|
|
+ images are byte-identical. Each gap found so far has been a small,
|
|
|
+ local fix, but the session is huge and byte-identity is exacting.
|
|
|
+
|
|
|
+## What the refactor actually is
|
|
|
+
|
|
|
+There are **two ways** the compiler can turn Modula-2 into QBE IR:
|
|
|
+
|
|
|
+1. **Legacy "inline" emit** — the grammar emits QBE IR *while* it
|
|
|
+ parses. This is the default and is what the working compiler and
|
|
|
+ the self-hosting fixpoint use.
|
|
|
+2. **Lower (two-phase)** — the grammar builds an **AST**, and a
|
|
|
+ separate module `Lower` walks the AST and emits QBE IR. This is
|
|
|
+ selected with the **`-lower`** flag and is the goal of the
|
|
|
+ refactor ("the flip" = make Lower the only emitter and delete the
|
|
|
+ inline emit).
|
|
|
+
|
|
|
+The refactor is being done in slices:
|
|
|
+
|
|
|
+| phase | what | state |
|
|
|
+|---|---|---|
|
|
|
+| L0–L2 | scalars, control flow (`IF`/`WHILE`/`FOR`/…) | done |
|
|
|
+| L3 | procedures, calls, the self-host root-cause fix | done |
|
|
|
+| L4 | builtins, `CASE`, `WITH` | done |
|
|
|
+| L5 | records/arrays/pointers/sets/strings, literals | done |
|
|
|
+| L6 | nested modules, classes, qualified names | done |
|
|
|
+| **L7** | **the flip: Lower emits the whole session** | **in progress** |
|
|
|
+
|
|
|
+Everything is committed in small steps; the many commits are
|
|
|
+**incremental fixes to the new `Lower` path**, not changes to the
|
|
|
+working legacy compiler.
|
|
|
+
|
|
|
+## Where exactly we are (L7)
|
|
|
+
|
|
|
+The flip is driven per unit: the driver suppresses the grammar's
|
|
|
+inline emit (`SetNoEmit`), snapshots the QbeGen state before each
|
|
|
+unit's parse, restores it, and lets `Lower` re-emit the unit. On the
|
|
|
+compiler's own sources:
|
|
|
+
|
|
|
+- `-lower` emits the **full** image: **736 functions** (same as
|
|
|
+ legacy), ~1.5 s.
|
|
|
+- It is **~74 % byte-identical** before the first divergence:
|
|
|
+ `first diff at byte 2,935,735, line 89,349` of `120,266`.
|
|
|
+- Progress of the first divergence across the session:
|
|
|
+ `line 9 → 327 → 1124 → 3437 → 6961 → 56294 → 70934 → 82194 → 86942 → 89349`.
|
|
|
+
|
|
|
+The first divergence is now the generic grammar `ArgList` call, which
|
|
|
+has **9 arguments** but `AST.MaxChild = 8` caps a call at 7, so `Lower`
|
|
|
+drops the tail.
|
|
|
+
|
|
|
+## What L7 fixed so far
|
|
|
+
|
|
|
+- `Lower` is **V3-legal** (forward references declared in `Lower.def`;
|
|
|
+ V3 has no whole-module pass) and the `SymTab.FormalOk` open-array
|
|
|
+ def/impl bug.
|
|
|
+- Whole-session tables + an open-addressing **hash index** for
|
|
|
+ `FindSym`/`FindProc`.
|
|
|
+- **Symbol scoping**: a procedure's locals no longer collide with
|
|
|
+ another procedure's (`HIGH(s)` picked the wrong `s`).
|
|
|
+- **Chunked sequences** where a list can exceed `AST.MaxChild`:
|
|
|
+ `CASE` arms, multi-name `VAR` declarations, `CASE` bodies.
|
|
|
+- External procedures, indirect calls (proc-typed variables), `VAL`,
|
|
|
+ `LEN`/`HIGH`, qualified calls, qualified proc-typed variables,
|
|
|
+ const values, 1-char strings passed as `ARRAY OF CHAR`.
|
|
|
+- Per-unit global materialisation (`MaterializeReplace`).
|
|
|
+- Duplicate nested procedure names resolve by scope and uid.
|
|
|
+
|
|
|
+## What remains
|
|
|
+
|
|
|
+1. **Calls with more than 7 arguments** (the `ArgList` case):
|
|
|
+ chunk the call actuals and make the argument loops follow the
|
|
|
+ `NkBlock` chunks. (A prototype regressed `AstCallNode`'s legacy
|
|
|
+ emit and was reverted.)
|
|
|
+2. Whatever the next first-divergence reveals. Historically each fix
|
|
|
+ has exposed one more small gap; there may be a few more.
|
|
|
+3. Only then: delete the grammar's inline emit (the literal flip) and
|
|
|
+ remove the `-lower` flag.
|
|
|
+
|
|
|
+## Are we near completion?
|
|
|
+
|
|
|
+- **Of the working compiler:** yes — it is done, green, and
|
|
|
+ self-hosting. The refactor has not broken it.
|
|
|
+- **Of the L7 flip:** the *hard architectural part is finished*
|
|
|
+ (per-unit emit, scoping, tables, the AST chunking mechanism). The
|
|
|
+ remainder is a **tail of codegen gaps** that only matter because the
|
|
|
+ verification is exact byte-identity over a 120k-line session.
|
|
|
+ Realistically a handful more small fixes; not a rewrite. It is not
|
|
|
+ required for the compiler to function — it is a quality/architecture
|
|
|
+ goal.
|
|
|
+
|
|
|
+## How to check for yourself
|
|
|
+
|
|
|
+```sh
|
|
|
+cd compiler
|
|
|
+./build.sh # rebuild
|
|
|
+./run_tests.sh # 204/204
|
|
|
+cd ..
|
|
|
+bootstrap/fixpoint.sh # FIXPOINT OK
|
|
|
+```
|