|
|
@@ -1,84 +1,96 @@
|
|
|
-# Handoff — L3 via option 2: functionally proven, blocked by no-FORWARD
|
|
|
+# Handoff — L3 via option 2: 183/183 but self-host blocked by V3 codegen
|
|
|
|
|
|
-Branch **`ast-stage-c`**, HEAD **`90b1d76`**. Suite **182/182**;
|
|
|
+Branch **`ast-stage-c`**, HEAD **`edf24d9`**. Suite **182/182**;
|
|
|
fixpoint **OK** (**3,147,849 bytes**). Clean except `toto.mod`.
|
|
|
|
|
|
## Headline
|
|
|
|
|
|
-Option 2 (zero `SymTab` surface) **works functionally**: with it,
|
|
|
-`t_lower3.mod` re-emits **byte-identical** to the legacy image and the
|
|
|
-suite reaches **183/183**. It does **not** self-host yet, and the cause
|
|
|
-is now precisely known and is **not** the table design.
|
|
|
+Option 2 (zero `SymTab` surface) is **functionally complete** and
|
|
|
+**byte-verified**: `t_lower3` matches the legacy image and the suite
|
|
|
+reaches **183/183** with `lower_ok t_lower3`. It **still does not
|
|
|
+self-host**, and the cause is now a *different* V3 codegen bug, not the
|
|
|
+table design.
|
|
|
|
|
|
-## What was built and proven (this session, then reverted to keep green)
|
|
|
+## What was built and verified this session
|
|
|
|
|
|
`compiler/src/Lower.def` / `Lower.mod`:
|
|
|
|
|
|
-- Owned tables: `syms[]` (`name/kind/ty`) and `procs[]`
|
|
|
- (`uid/res/depth/ext/params`), with `Reset`, `NoteVar`, `NoteParam`,
|
|
|
- `NoteProc`, `SetProcRes`, `ScopeEnter`/`ScopeLeave`, and resolvers
|
|
|
- `FindSym`/`FindProc`/`Resolve`/`ProcInfo`/`ParamInfo`. **No `SymTab`
|
|
|
- edits at all.**
|
|
|
+- Owned, **flat** tables (no arrays of records — see the bug below):
|
|
|
+ `symName : ARRAY OF CHAR` (stride 64), `symKind`, `symTy`, and
|
|
|
+ `procs : ARRAY OF ProcEnt` with a flat `pname` (stride 64). Helpers
|
|
|
+ `Reset`, `NoteVar`, `NoteParam`, `NoteProc`, `SetProcRes`,
|
|
|
+ no-op `ScopeEnter`/`ScopeLeave`, `CopyN`/`CopySpan`, `FindSym`/
|
|
|
+ `FindProc`/`Resolve`/`ProcInfo`/`ParamInfo`. **No `SymTab` edits.**
|
|
|
- All `SymTab.ProcUid/ProcNPar/ParamType/...` call-sites switched to
|
|
|
`ProcInfo`/`ParamInfo`; `LowerDesign`/`IsSimpleScalar`/`IsCall` to
|
|
|
`Resolve`/`FindProc`.
|
|
|
-- Grammar hooks (inert; fill tables only): `M2` root `Lower.Reset`;
|
|
|
- `VarDecl`/`ConstDecl` `Lower.NoteVar`; `ProcHeading` `Lower.ScopeEnter`
|
|
|
- + `Lower.NoteProc` + `Lower.SetProcRes`; `ParamSection`
|
|
|
- `Lower.NoteParam`/`NoteVar`; `ProcDecl` `Lower.ScopeLeave` and
|
|
|
- `AST.SetOp(astNode, 0)`.
|
|
|
-- **Key bug fixed:** `ScopeLeave` must be a **no-op during parse** — a
|
|
|
- real pop removed locals/params *before* the end-of-unit walk (same
|
|
|
- closed-scope trap as `SymTab`). With it no-op'd, procedure bodies emit
|
|
|
- and `t_lower3` matches.
|
|
|
-
|
|
|
-## The blocker (why it can't self-host yet)
|
|
|
-
|
|
|
-`Lower` now has **three mutually-recursive procedure pairs**:
|
|
|
-
|
|
|
-- `CanExpr` ⇄ `CanCallArgs`
|
|
|
-- `CanProc` ⇄ `CanDecls` (via `CanDecl`)
|
|
|
-- `LowerExpr` ⇄ `LowerCall`
|
|
|
-
|
|
|
-The V3 self-hosted compiler rejects mutual recursion **without `FORWARD`**
|
|
|
-(`Lower.mod:NNN invalid call`), and `FORWARD` is not usable either (it
|
|
|
-breaks gm2's whole-program pass — see `docs/summary_two-phase-lower2.md`).
|
|
|
-The committed `Lower.mod` avoids this by **merging** mutually recursive
|
|
|
-bodies into one dispatching procedure (`CanStmt`, `LowerStmt`).
|
|
|
-
|
|
|
-## The fix (next session)
|
|
|
-
|
|
|
-Break all three cycles the same way the committed code already does —
|
|
|
-**merge, don't split**:
|
|
|
-
|
|
|
-1. **Inline `CanCallArgs` into `CanExpr`**'s `NkCall` branch (it is a
|
|
|
- few lines: arity + recurse `CanExpr` on actuals).
|
|
|
-2. **Inline `CanProc` into `CanDecl`** (call `CanStmt` on `NkProcDecl`
|
|
|
- body/children directly).
|
|
|
-3. **Inline `LowerCall` into `LowerExpr`**'s `NkCall` branch.
|
|
|
-
|
|
|
-After inlining, no procedure calls a later or mutually-recursive one, so
|
|
|
-the V3 compiler accepts it and the fixpoint should return. All three are
|
|
|
-mechanical (their bodies are small and already written).
|
|
|
+- Grammar hooks (inert): `M2` root `Lower.Reset`; `VarDecl`/`ConstDecl`
|
|
|
+ `Lower.NoteVar`; `ProcHeading` `Lower.ScopeEnter`+`NoteProc`+
|
|
|
+ `SetProcRes`; `ParamSection` `Lower.NoteParam`/`NoteVar`; `ProcDecl`
|
|
|
+ `Lower.ScopeLeave` + `AST.SetOp(astNode, 0)`.
|
|
|
+- `ScopeLeave` is a **no-op**: parse-time notes must survive until the
|
|
|
+ end-of-unit walk.
|
|
|
+
|
|
|
+Result: `t_lower3` (function + VAR-param procedure + calls + RETURN)
|
|
|
+**byte-matches**; suite **183/183**.
|
|
|
+
|
|
|
+## New finding — nested-array record field miscompiles
|
|
|
+
|
|
|
+Repro `docs/wip/nested-array-field-bug.mod`:
|
|
|
+
|
|
|
+```modula2
|
|
|
+TYPE PE = RECORD
|
|
|
+ name : ARRAY [0..63] OF CHAR;
|
|
|
+ pname : ARRAY [0..3] OF ARRAY [0..63] OF CHAR; (* nested *)
|
|
|
+END;
|
|
|
+VAR procs : ARRAY [0..3] OF PE;
|
|
|
+...
|
|
|
+procs[nProcs-1].pname[j][i] := name[i];
|
|
|
+```
|
|
|
|
|
|
-Then: `run_tests.sh` already needs `lower_ok t_lower3.mod`; expect
|
|
|
-**183/183** and **FIXPOINT OK**.
|
|
|
+`./M2` compiles it, but the **generated program segfaults** (rc 139;
|
|
|
+want 42). A **flat** `pname : ARRAY [0..2047] OF CHAR` (manual stride)
|
|
|
+works. This is a real user-facing bug — likely the element stride /
|
|
|
+descriptor-pointer setup for a nested-array field (same family as the
|
|
|
+Stage-A array-of-record stride fix in `QbeGen.ElemSize`, and
|
|
|
+`docs/wip/local-array-bug.mod`).
|
|
|
+
|
|
|
+It was hit directly: the V3-built compiler (`m2s2`) crashed on **every**
|
|
|
+input when `Lower`'s formal-name table used a nested-array field.
|
|
|
+Flattening fixed that crash.
|
|
|
+
|
|
|
+## Why self-host still fails
|
|
|
+
|
|
|
+After flattening, `m2s2` works on small programs (`t_exit`, a `CONST`
|
|
|
+program) but **still crashes (SIGABRT) on the full compiler input**
|
|
|
+(empty output). Shrinking the tables (`MaxSym` 2048→512, `MaxProc`
|
|
|
+512→64) did **not** help, so it is not table size — it is a
|
|
|
+**size/complexity-triggered latent V3 codegen bug** in the compiled
|
|
|
+`Lower`/`M2P` code. HEAD self-hosts; the ~250 added lines tip it over.
|
|
|
+
|
|
|
+This is the third instance of the same compiler fragility (local arrays,
|
|
|
+nested-array fields, and now this). Finding the exact construct needs a
|
|
|
+bisect of the added code with a V3-built compiler — a dedicated task.
|
|
|
+
|
|
|
+## Next steps
|
|
|
+
|
|
|
+1. **Fix the nested-array-field bug in `QbeGen`** (like the Stage-A
|
|
|
+ stride fix): make `arr[i].field[j][k]` correct. Add the repro to the
|
|
|
+ suite. This is a real, self-contained bug and may also matter for the
|
|
|
+ size-triggered failure.
|
|
|
+2. **Bisect the size-triggered self-host crash**: build `m2s2`, then
|
|
|
+ compile subsets of the added `Lower`/`M2P` code with it to find the
|
|
|
+ construct that crashes. Candidates: the flat-stride indexing, the
|
|
|
+ `CopySpan` offsets, or a specific grammar hook.
|
|
|
+3. Once self-host is green: keep `lower_ok t_lower3`, expect 183/183 +
|
|
|
+ FIXPOINT OK.
|
|
|
|
|
|
## Do NOT
|
|
|
|
|
|
-- Do not add `FORWARD` (breaks gm2 whole-program pass).
|
|
|
-- Do not edit `SymTab`.
|
|
|
-- Do not run legacy + `L` emitters concurrently (shared `QbeGen` state).
|
|
|
-- Do not leave an `ASTDUMP` in `M2.atg` (it was committed once by
|
|
|
- accident and cost time — `git show v3-lower3-wip:compiler/src/M2.atg`
|
|
|
- had it).
|
|
|
-
|
|
|
-## Also re-apply (two real bugs, from earlier this session)
|
|
|
-
|
|
|
-- `NkProcDecl` `op` must be `0` for a normal body (`CanProc` checks it;
|
|
|
- default is `-1`).
|
|
|
-- `NkCall` child 0 is an `NkDesignator`, not a bare `NkIdent` — unwrap it
|
|
|
- (`CalleeName`), else calls mangle as `$_0`.
|
|
|
+- No `FORWARD` (breaks gm2 whole-program pass).
|
|
|
+- No `SymTab` edits.
|
|
|
+- No arrays of records with aggregate (nested-array) fields in `Lower`.
|
|
|
+- No committed `ASTDUMP`.
|
|
|
|
|
|
## Resume
|
|
|
|