M2compiler-V3/docs/summary_two-phase-m2-2026-10-16.md

Session summary — 2026-10-16: two-phase parse, milestones 2a–2c

Branch resolve-check, HEAD f500a12, tags v3-two-phase-m2a, v3-two-phase-m2b, v3-two-phase-m2c. Suite 199/199; fixpoint OK (3,620,461 bytes); tree clean. Design/record: docs/plan-two-phase-parse.md.

Goal

Delete SymTab.FwdVarRef/FwdVarName/FwdVarResolve, Check.NoteForward/FlushForward, the Design forward branch and the Op=1 marker — by parsing a unit's declarations before its procedure bodies, so forward references bind normally.

What landed (all behind -twophase, default off)

Step	Commit	What
2a	dce67fc	scanner.SeekTo(p): replay NextCh over the in-memory buffer (O(p), no I/O); body-span table + a SeekTo round-trip fidelity check in the probe.
2b	cdb6436	SymTab.DeferReset/DeferBody/ReenterDeferred/ParamName; Lower.ReenterScopeProc/CurScopeProc; span table carries the deferral index.
2c	f500a12	Real deferral: phase-1 DeferSkip (leave scanner at the matching END, close like FORWARD), phase-2 re-parse loop at module ends.

Mechanism (2c)

• Phase 1 — DeferSkip (reached from Block, only for plain ProcDecl bodies; methods stay inline) consumes the body tokens for real, verifies END <name>, leaves the scanner at that END so Block's trailing END ident consumes it, records the span; then ProcDecl closes the proc like FORWARD (AbortFunc, no body node).
• Phase 2 — an inlined loop at both module ends (ProgModule/ImplUnit) does, per span: SeekTo+Adv, SymTab.ReenterDeferred, Lower.ReenterScopeProc, QbeGen.BeginFunc, Block(nm), attach body + block decls to the proc node, ScopeLeave/CloseProc/EndFunc. Scanner/parser state and the unit's own astDecl/astStmt are saved and restored around the loop.
• Opens/closes pair within each pass; registrations run once (skip pass); the re-parse uses the resume paths.

Two real bugs found and fixed while landing it

1. The re-parse loop clobbered the unit's astDecl/astStmt before ProgModule built the unit (main got a proc's statements). Fixed by saving/restoring them around the loop.
2. PopPos restores the scanner's errors counter (deliberate for the shadow probe), so it wiped every diagnostic recorded during the re-parse — 210/232 etc. inside deferred bodies were silently dropped (t_bad_fwd was wrongly accepted). Fixed by re-instating the post-loop count after PopPos.

Findings worth keeping

• No forward procedure tolerance exists. A call to a proc declared later in the same module is 233 (+ a knock-on 201 at EOF); a bare P; two lines early is 233. The compiler's own sources self-compile only because each .mod is paired with its .def, parsed first (so e.g. SemError is declared before NoteLabel's body compiles), plus strict declaration-before-use inside modules. Consequence for the grammar: atg helpers must be declared before use (NoteSpan had to move above ProbeBody); the FwdVar tolerance covers module-level variables only.
• Suppressed inline emit = emission only. Under SetNoEmit every QbeGen counter still advances and is discarded per unit by RestoreSession; Lower's tables are permanent session state.
• Byte-identity vs the old default is the wrong gate for forward code. Deferral makes a forward variable ordinary, so it compiles like a normally-ordered one; the legacy FwdVar path is the divergent one (it drops a loadw). Deferred output matches the normal codegen and runs correctly.

Measured (2c)

• Suite 199/199 — 193 core + gates: twophase_same (t_nestedproc, t_classmethod) identity; twophase_run (t_fwdvar/t_fwdsel/t_fwdprop) -> 42; twophase_fail (t_bad_fwd) incompatible assignment.
• All 142 single-file tests under -twophase: 0 signals, 138 byte-identical. The 4 differing files are exactly the forward-reference tests.
• Sessions DBasicProg/TString/Showcase12 byte-identical.
• Fixpoint OK (flag off).

Known gap (why it is not flipped)

On some multi-module implementation sessions (repro: convtypes.def/.mod conversions.def/.mod, i.e. Conversions.mod inside the Library1 session) -twophase recorded a spurious ident expected positioned inside a repassed body — while the emitted .ssa was identical to the default.

FIXED and flipped (same session). Root cause: DeferSkip pushed a scanner snapshot for its failure-restore but, on success, never popped it — leaking one snapshot per deferred body until PushPos began failing, after which the module-end PopPos restored a stale slot and the parser's SynError read a stale nextPos. A new scanner.DropPos (discard the top snapshot without restoring) is called on the success path. With that, deferral is byte-clean on Library1, the whole suite passes with deferral on, and the fixpoint is byte-identical with deferral on: the two-phase parse is now the default (-notwophase restores the old inline parse for debugging).

Still to delete: FwdVar*/NoteForward/FlushForward and the Design forward branch — still exercised by nested-local forwards (a nested proc naming a parent local declared later), which module-level deferral does not cover and which was already broken (fails to link) before the flip.

COMPLETED (same session) — machinery deleted

Nested deferral was tried (defer during the repass) and reverted: it regressed nested-proc codegen (t_nestdeep/showcase10). So nested forwards are instead made a clean 201, and the whole forward mechanism is deleted (commit 6d8d817, tag v3-forward-machinery-deleted):

• removed SymTab.FwdVarRef/FwdVarName/FwdVarResolve (+ tables), Check.NoteForward/FlushForward, the Design forward branch, and the dead Check.SelType/LeafName;
• language gains: a module-level body may call a procedure declared later (no FORWARD needed); a nested-local forward is a plain 201 instead of a broken image;
• verified: suite 201/201 (t_fwdproc -> 7, t_bad_nestfwd 201); fixpoint OK (3,574,843 bytes); corpus 365 vs 358 OK (-notwophase): +7 files, 0 regressions, 0 crashes.

This closes the resolve/check thread's goal. Remaining cleanup (separate): Check is now a largely vestigial walk (PropNode/EvalExpr/CheckAssign re-type/re-check nothing once no node is Ty = -1).

Cleanup done (same session) — Check removed

Instrumented PropNode first: it did re-type Ty = -1 nodes in 31 tests, but gutting Resolve left the emitted .ssa byte-identical across all 141 suite files (and the suite 201/201) — Lower recomputes what it needs and Check emitted no diagnostics. So Check.def/Check.mod were deleted along with the driver calls, the -noresolve option, and the atg import; build.sh drops Check (fixpoint.sh stages sources dynamically). Also removed the unused procNest and the always-true L7 flag twoPhase. Pipeline is now Parse -> Lower -> Emit. Suite 201/201, fixpoint OK (3,555,791 bytes).

Resume

cd .../m2compiler-V3/compiler
./build.sh && ./run_tests.sh        # 199/199
./M2 -twophase ../stdlib/convtypes.def ../stdlib/convtypes.mod \
          ../stdlib/conversions.def ../stdlib/conversions.mod
# -> spurious ".../conversions.mod:148:3: ident expected"
cd .. && ./bootstrap/fixpoint.sh     # FIXPOINT OK