|
|
@@ -0,0 +1,124 @@
|
|
|
+# Two-phase refactor — analysis and plan (2026-10-02)
|
|
|
+
|
|
|
+Decision: **no restart.** The AST/two-phase change is a **middle-of-the-
|
|
|
+compiler** refactor, not a rewrite. We reuse everything and convert the
|
|
|
+frontend's *pass structure* on a dedicated branch.
|
|
|
+
|
|
|
+This document is the plan of record; `docs/session-handoff-2026-10-01.md`
|
|
|
+has the short version.
|
|
|
+
|
|
|
+## 1. Why not restart from scratch
|
|
|
+
|
|
|
+Everything painful in the last two sessions traced to one cause:
|
|
|
+**lowering happens during parsing**, so nothing can be looked at twice.
|
|
|
+An AST-first compiler fixes that — but a *from-scratch* restart would
|
|
|
+throw away the parts that are expensive and already correct:
|
|
|
+
|
|
|
+| Component | Reuse on restart? | Reuse on refactor? |
|
|
|
+| --- | --- | --- |
|
|
|
+| Coco/R frame, scanner/parser driver (`M2S`/`M2P`/`compiler.frm`) | mostly | yes |
|
|
|
+| **`SymTab`** — descriptors, scoping, checks, classes, vtables | large, hard-won | yes |
|
|
|
+| **`QbeGen`** — emission, layouts, calls, sets, records, classes | the whole backend | yes |
|
|
|
+| runtime shim, `stdlib/`, `FileIO`, fixpoint + corpus harnesses, tests | yes | yes |
|
|
|
+| `docs/` (language report, gaps, grammar) | yes | yes |
|
|
|
+| **`AST.def/.mod`** (Stage A, tag `v3-ast-stageA`) | n/a | already started |
|
|
|
+| `M2.atg` production **grammar** | yes (recognition is fine) | yes |
|
|
|
+| `M2.atg` production **actions** | rewritten | rewritten (same grammar) |
|
|
|
+
|
|
|
+The one thing a restart legitimately replaces is the **dialect/grammar**
|
|
|
+— and the grammar is not the bottleneck. The **pass structure** is.
|
|
|
+
|
|
|
+So: change what the grammar *actions do* (build nodes instead of emit),
|
|
|
+not what the grammar *recognizes*.
|
|
|
+
|
|
|
+## 2. Pass structure (target)
|
|
|
+
|
|
|
+```
|
|
|
+per unit:
|
|
|
+ 1. Parse -> AST (declarations + bodies), SymTab entries for
|
|
|
+ names/kinds as declared (order-independent later)
|
|
|
+ 2. Resolve/Check -> bind identifiers, complete types, 2xx diagnostics
|
|
|
+ 3. Lower -> walk AST, call the existing QbeGen API
|
|
|
+ 4. Emit -> unchanged (buffered image, EndModule)
|
|
|
+```
|
|
|
+
|
|
|
+Key property: **step 3 runs after the whole unit's declarations are in
|
|
|
+SymTab**, so forward references resolve naturally. The text-patching
|
|
|
+machinery and the `Design` placeholder path are deleted.
|
|
|
+
|
|
|
+## 3. Flag strategy (keep the suite green while converting)
|
|
|
+
|
|
|
+- Add `M2.atg` flag `TWO_PHASE` (a grammar-level BOOLEAN or a
|
|
|
+ `QbeGen.SetLower` switch).
|
|
|
+- Phase 1: parser **both** builds AST nodes **and** emits (today's
|
|
|
+ behaviour). AST building is inert; nothing reads it. Suite/fixpoint
|
|
|
+ stay green. (This is basically Stage A applied to the productions.)
|
|
|
+- Phase 2: implement `Lower(n)`; run it in a **test-only** mode
|
|
|
+ (compile a unit, lower from the AST, compare to the old `.ssa`).
|
|
|
+- Phase 3: flip a unit's lowering to the AST walk; keep the old path for
|
|
|
+ units not yet converted. Convert production-by-production.
|
|
|
+- Phase 4: when everything lowers from the AST, delete the old actions
|
|
|
+ and the patching code.
|
|
|
+
|
|
|
+## 4. Increment order (branch `ast-stage-c`)
|
|
|
+
|
|
|
+Suggested slices, each ending in `build.sh` + `run_tests.sh` +
|
|
|
+`fixpoint.sh` (revert the slice if either fails):
|
|
|
+
|
|
|
+1. **Flag + scaffolding**: `TWO_PHASE` flag; parser builds AST nodes for
|
|
|
+ one trivial production (e.g. `Ident`/`NkIdent`) alongside emission.
|
|
|
+ Nothing consumes them. Green.
|
|
|
+2. **Expressions → AST**: `Expr`/`SimExpr`/`Term`/`Fact` build
|
|
|
+ `NkBinExpr`/`NkUnary`/`NkIdent`/literals, *in addition* to emitting.
|
|
|
+ Add `Lower` for expressions, exercised in a test-only compare.
|
|
|
+3. **Statements → AST**: `Block`/`Stat` build `NkAssign`/`NkIf`/`NkWhile`/
|
|
|
+ …; `Lower` them.
|
|
|
+4. **Declarations → AST**: `ConstBlock`/`TypeBlock`/`VarBlock`/
|
|
|
+ `ProcDecl`; the unit's decls complete SymTab before any body is
|
|
|
+ lowered.
|
|
|
+5. **Flip**: bodies lower from the AST; delete `FwdPatchAll` /
|
|
|
+ `FixLoadClass` / `FixStoreClass`, the `Design` forward-variable
|
|
|
+ placeholder, and `pendVar*` if superseded.
|
|
|
+6. **Cleanup + docs**: update `docs/features.md`, `language-report.md`,
|
|
|
+ `analysis_gaps.md`; tag.
|
|
|
+
|
|
|
+## 5. Verification gates
|
|
|
+
|
|
|
+- **Fixpoint is the hard gate**: `./bootstrap/fixpoint.sh` must print
|
|
|
+ `FIXPOINT OK` (byte-identical stage2/stage3). The compiler compiles
|
|
|
+ itself, so it exercises the new path on real, large input.
|
|
|
+- **Suite** `run_tests.sh` (178/178 today) after every slice.
|
|
|
+- **Corpus** (`tools/v3-corpus/corpus.sh`) — expect the `undeclared
|
|
|
+ identifier` / forward-reference buckets to shrink once the AST lands;
|
|
|
+ track the compile-OK counts.
|
|
|
+- **Byte-compare** in Phase 2/3: lower-from-AST `.ssa` vs old `.ssa` for
|
|
|
+ the test suite; they should match until a feature deliberately
|
|
|
+ changes.
|
|
|
+
|
|
|
+## 6. What this unlocks (delete list / new capability)
|
|
|
+
|
|
|
+- Delete: `QbeGen.FwdPatchAll`, `FwdDesignator`, `FwdMangle`,
|
|
|
+ `FwdAddrOper`, `FwdPatch`, `FwdElemPatch`, `FixLoadClass`,
|
|
|
+ `FixStoreClass`, `BufReplaceAll`; the grammar's `FwdVarNote`/
|
|
|
+ `FwdVarFlush`/`nFvarRefs`; the `Design` forward branch and
|
|
|
+ `SymTab.FwdVar*`; `EXPR`'s `IsFwdVar` leniency.
|
|
|
+- New capability, now natural: forward **procedures** without a
|
|
|
+ DEFINITION; **set/aggregate values in `CONST`** (unblocks
|
|
|
+ `ChanConsts`/`StreamFile`, parked in `docs/wip/`); statement-context
|
|
|
+ result suffixes (`F()[i] := x`); `GOTO`/labels; typed `CONST`.
|
|
|
+
|
|
|
+## 7. Risks and mitigations
|
|
|
+
|
|
|
+- **Self-hosting order constraints** — the AST removes them, but during
|
|
|
+ the transition `M2.atg`/`QbeGen.mod` must still compile under the
|
|
|
+ *current* compiler; keep declaration-before-use until the flip.
|
|
|
+- **Two paths diverging** — mitigate with byte-compare in Phases 2–3.
|
|
|
+- **Scope creep into Stage C in one go** — do not. Slices above are
|
|
|
+ deliberately one concern each.
|
|
|
+- **Fixpoint red for a long stretch** — expected in Phase 4; keep the
|
|
|
+ old path available until every unit lowers from the AST.
|
|
|
+
|
|
|
+## 8. State / entry point
|
|
|
+
|
|
|
+Branch off `v3-session-2026-10-02` (`03f250e`). Baseline: suite 178/178,
|
|
|
+fixpoint OK (2,900,782 bytes). Start with §4 slice 1.
|