ソースを参照

docs: two-phase analysis and plan (no restart; middle-of-compiler)

Records the decision: do not restart the compiler. The AST/two-phase
change reuses SymTab, QbeGen, the runtime, harnesses and gramar
recognition; only grammar actions change. plan-two-phase.md gives the
target pass structure, a flag strategy (build AST alongside emission
until a lowering walk byte-matches), the increment order, the hard
gate (self-hosting fixpoint), the delete list, and the risks.
Eric Streit 1 週間 前
親
コミット
1c93f19dbf
2 ファイル変更、130 行追加、0 行削除
  1. 124 0
      docs/plan-two-phase.md
  2. 6 0
      docs/session-handoff-2026-10-01.md

+ 124 - 0
docs/plan-two-phase.md

@@ -0,0 +1,124 @@
+# Two-phase refactor — analysis and plan (2026-10-02)
+
+Decision: **no restart.**  The AST/two-phase change is a **middle-of-the-
+compiler** refactor, not a rewrite.  We reuse everything and convert the
+frontend's *pass structure* on a dedicated branch.
+
+This document is the plan of record; `docs/session-handoff-2026-10-01.md`
+has the short version.
+
+## 1. Why not restart from scratch
+
+Everything painful in the last two sessions traced to one cause:
+**lowering happens during parsing**, so nothing can be looked at twice.
+An AST-first compiler fixes that — but a *from-scratch* restart would
+throw away the parts that are expensive and already correct:
+
+| Component | Reuse on restart? | Reuse on refactor? |
+| --- | --- | --- |
+| Coco/R frame, scanner/parser driver (`M2S`/`M2P`/`compiler.frm`) | mostly | yes |
+| **`SymTab`** — descriptors, scoping, checks, classes, vtables | large, hard-won | yes |
+| **`QbeGen`** — emission, layouts, calls, sets, records, classes | the whole backend | yes |
+| runtime shim, `stdlib/`, `FileIO`, fixpoint + corpus harnesses, tests | yes | yes |
+| `docs/` (language report, gaps, grammar) | yes | yes |
+| **`AST.def/.mod`** (Stage A, tag `v3-ast-stageA`) | n/a | already started |
+| `M2.atg` production **grammar** | yes (recognition is fine) | yes |
+| `M2.atg` production **actions** | rewritten | rewritten (same grammar) |
+
+The one thing a restart legitimately replaces is the **dialect/grammar**
+— and the grammar is not the bottleneck.  The **pass structure** is.
+
+So: change what the grammar *actions do* (build nodes instead of emit),
+not what the grammar *recognizes*.
+
+## 2. Pass structure (target)
+
+```
+per unit:
+  1. Parse            -> AST (declarations + bodies), SymTab entries for
+                         names/kinds as declared (order-independent later)
+  2. Resolve/Check    -> bind identifiers, complete types, 2xx diagnostics
+  3. Lower            -> walk AST, call the existing QbeGen API
+  4. Emit             -> unchanged (buffered image, EndModule)
+```
+
+Key property: **step 3 runs after the whole unit's declarations are in
+SymTab**, so forward references resolve naturally.  The text-patching
+machinery and the `Design` placeholder path are deleted.
+
+## 3. Flag strategy (keep the suite green while converting)
+
+- Add `M2.atg` flag `TWO_PHASE` (a grammar-level BOOLEAN or a
+  `QbeGen.SetLower` switch).
+- Phase 1: parser **both** builds AST nodes **and** emits (today's
+  behaviour).  AST building is inert; nothing reads it.  Suite/fixpoint
+  stay green.  (This is basically Stage A applied to the productions.)
+- Phase 2: implement `Lower(n)`; run it in a **test-only** mode
+  (compile a unit, lower from the AST, compare to the old `.ssa`).
+- Phase 3: flip a unit's lowering to the AST walk; keep the old path for
+  units not yet converted.  Convert production-by-production.
+- Phase 4: when everything lowers from the AST, delete the old actions
+  and the patching code.
+
+## 4. Increment order (branch `ast-stage-c`)
+
+Suggested slices, each ending in `build.sh` + `run_tests.sh` +
+`fixpoint.sh` (revert the slice if either fails):
+
+1. **Flag + scaffolding**: `TWO_PHASE` flag; parser builds AST nodes for
+   one trivial production (e.g. `Ident`/`NkIdent`) alongside emission.
+   Nothing consumes them.  Green.
+2. **Expressions → AST**: `Expr`/`SimExpr`/`Term`/`Fact` build
+   `NkBinExpr`/`NkUnary`/`NkIdent`/literals, *in addition* to emitting.
+   Add `Lower` for expressions, exercised in a test-only compare.
+3. **Statements → AST**: `Block`/`Stat` build `NkAssign`/`NkIf`/`NkWhile`/
+   …; `Lower` them.
+4. **Declarations → AST**: `ConstBlock`/`TypeBlock`/`VarBlock`/
+   `ProcDecl`; the unit's decls complete SymTab before any body is
+   lowered.
+5. **Flip**: bodies lower from the AST; delete `FwdPatchAll` /
+   `FixLoadClass` / `FixStoreClass`, the `Design` forward-variable
+   placeholder, and `pendVar*` if superseded.
+6. **Cleanup + docs**: update `docs/features.md`, `language-report.md`,
+   `analysis_gaps.md`; tag.
+
+## 5. Verification gates
+
+- **Fixpoint is the hard gate**: `./bootstrap/fixpoint.sh` must print
+  `FIXPOINT OK` (byte-identical stage2/stage3).  The compiler compiles
+  itself, so it exercises the new path on real, large input.
+- **Suite** `run_tests.sh` (178/178 today) after every slice.
+- **Corpus** (`tools/v3-corpus/corpus.sh`) — expect the `undeclared
+  identifier` / forward-reference buckets to shrink once the AST lands;
+  track the compile-OK counts.
+- **Byte-compare** in Phase 2/3: lower-from-AST `.ssa` vs old `.ssa` for
+  the test suite; they should match until a feature deliberately
+  changes.
+
+## 6. What this unlocks (delete list / new capability)
+
+- Delete: `QbeGen.FwdPatchAll`, `FwdDesignator`, `FwdMangle`,
+  `FwdAddrOper`, `FwdPatch`, `FwdElemPatch`, `FixLoadClass`,
+  `FixStoreClass`, `BufReplaceAll`; the grammar's `FwdVarNote`/
+  `FwdVarFlush`/`nFvarRefs`; the `Design` forward branch and
+  `SymTab.FwdVar*`; `EXPR`'s `IsFwdVar` leniency.
+- New capability, now natural: forward **procedures** without a
+  DEFINITION; **set/aggregate values in `CONST`** (unblocks
+  `ChanConsts`/`StreamFile`, parked in `docs/wip/`); statement-context
+  result suffixes (`F()[i] := x`); `GOTO`/labels; typed `CONST`.
+
+## 7. Risks and mitigations
+
+- **Self-hosting order constraints** — the AST removes them, but during
+  the transition `M2.atg`/`QbeGen.mod` must still compile under the
+  *current* compiler; keep declaration-before-use until the flip.
+- **Two paths diverging** — mitigate with byte-compare in Phases 2–3.
+- **Scope creep into Stage C in one go** — do not.  Slices above are
+  deliberately one concern each.
+- **Fixpoint red for a long stretch** — expected in Phase 4; keep the
+  old path available until every unit lowers from the AST.
+
+## 8. State / entry point
+
+Branch off `v3-session-2026-10-02` (`03f250e`).  Baseline: suite 178/178,
+fixpoint OK (2,900,782 bytes).  Start with §4 slice 1.

+ 6 - 0
docs/session-handoff-2026-10-01.md

@@ -6,6 +6,12 @@ except the user's `compiler/toto.mod`.
 
 ## >>> NEXT SESSION: architectural work — FINDING (2026-10-02) <<<
 
+**Plan of record: `docs/plan-two-phase.md`.**  No restart — it is a
+middle-of-the-compiler refactor reusing SymTab, QbeGen, the runtime and
+the harnesses; only the grammar *actions* change (build nodes instead of
+emit).  Branch `ast-stage-c` off `v3-session-2026-10-02`; work §4 slice
+by slice with the fixpoint as the hard gate.
+
 **Stage A is DONE** (tag `v3-ast-stageA`, commit `c91695c`): the AST
 node module `compiler/src/AST.def/.mod` exists and is tested
 (`tests/t_ast.mod`), and a real bug was fixed along the way (array-of-