# Two-phase parse (design) — eliminating the forward mechanism ## Goal Delete `SymTab.FwdVarRef`/`FwdVarName`/`FwdVarResolve`, `Check.NoteForward`/ `FlushForward` and the `Design` forward branch, plus the `Op=1` marker — by binding names before they are checked. ## Why post-parse checking is not enough `Design` defers a name it cannot find **inside a procedure** because it may be a *module-level* declaration that appears **later in the file**. The resolve/check pass happens after the parse, so it cannot help with *binding*: binding happens during the parse, and the declaration simply has not been read yet. The only fix is to read the declarations before the bodies — a two-phase (or two-pass) *parse*. ## Design A (recommended): module-level body deferral + re-scan The grammar/parser stays almost entirely as-is; only the *order* in which the parser visits text changes. 1. **Skip module-level procedure bodies.** When `ProcDecl` sees a module-level `PROCEDURE`, record the body's source span and consume its tokens without parsing them, then carry on with the remaining declarations. Result: every module-level name is in `SymTab` once the module's declarations are done. 2. **Re-scan and parse the bodies.** At the end of the module, for each recorded span: seek the scanner to its start and parse the body with the existing `Block` production. Every module-level name now resolves, so no forward reference is ever created. Consequences: - `Design`'s forward branch never fires → the whole `FwdVar*` path is dead and removable. - The body productions are **reused unchanged** (both the skip and the re-parse reuse existing machinery), unlike Design B below. ### Pieces 1. **Scanner seek** (`scanner.frm`, and expose it in `M2S.def`'s template): `PROCEDURE SeekTo (p: INT32)` — set `bp0`/`bp`/`nextPos` to `p` and recompute `line`/`col` (scan from offset 0 counting `CHR(10)`; `CharAt` already indexes the in-memory buffer, so this is O(p)). 2. **Token-level body skipper** (grammar helper): starting at the `PROCEDURE` keyword, consume tokens counting block openers `{PROCEDURE, IF, CASE, WHILE, FOR, LOOP, WITH, RECORD}`; stop when the count returns to 0 — that `END` closes the procedure. `BEGIN` is **not** an opener (a block's `BEGIN…END` shares the declaration's `END`), and `REPEAT…UNTIL`/`EXIT`/`RETURN` are not counted. Verify afterwards that the token is `END` and the following identifier equals the procedure name (guards a mis-skip). 3. **Deferral** in `ProcDecl` (module level only): skip + record the span; the declaration node is a placeholder `NkProcDecl`. 4. **Body re-parse**: a new grammar entry (reusing `Block`) driven at the end of `Unit`, walking the recorded spans. 5. **Flag**: all of this behind `-twophase` (default off) until the suite and the fixpoint are byte-identical, then flip. ## Design B (rejected): syntax-only bodies + binding walk Make every expression/statement action resolution-free and bind/type/ check in `Check`, walking with scopes mirrored from the AST. This is the "true" two-phase and is much larger: it touches every body action, and parse decisions that need types (typed brace literals, method calls, `L.x` qualified names, `HIGH`/`LEN`) would have to be deferred too. ## Risks / open questions - **Skipper exactness.** A mis-count mis-skips to the wrong `END`. The post-skip check (`END` + name) turns this into a clear error rather than silent corruption. - **Scanner state after a seek** (`line`, `col`, lookahead) and error positions inside re-parsed bodies. - **AST/`Lower` ordering.** Bodies are parsed after the declarations, so node allocation order changes. `Lower` walks by structure (order- independent), but any parse-time ordinal that assumes file order must be re-checked. - **Nested procedures** are inside the deferred span, so they are re-parsed with the same mechanism (their own locals are declared before their statements, so they need no deferral themselves). ## Milestones (each with the fixpoint green) 1. `SeekTo` + the skipper behind the flag; inert. 2. Module-level deferral + body re-parse; byte-compare against the default across the suite. 3. Flip the default; delete `FwdVarRef`/`FwdVarName`/`FwdVarResolve` and `Check.NoteForward`/`FlushForward`, the `Design` forward branch, and the `Op=1` marker. ## Cost Pieces 1–2 are a few hundred lines across the scanner frame, the grammar and the driver, and are the largest single change in the resolve/check thread — the reason `FwdVar*` could not be deleted by moving checks alone. ## Progress - **2026-10-15, milestone 1 (probe behind `-twophase`, inert).** Each outermost procedure/method body is shadow-skipped at the token level and the skip verified before the body parses normally: - `scanner.frm`: `PushPos`/`PopPos` save and restore the *full* scanner state (cursor + all published positions + `errors`, so a re-scanned bad character cannot double-count). 1-deep use, 8-deep stack. - `M2.atg`: `SetTwoPhase` switch, `procNest` counter (hooked once in `Block`, so `ProcDecl` and method bodies are both covered), `TokIs` (compares token *text*, immune to token renumbering) and `ProbeBody` (opener counting + `END ` check, silent on success, `twophase probe mismatch` to StdOut otherwise — never into the listing). - `compiler.frm` + `parser.frm`: `-twophase` flag plumbed to `M2P.SetTwoPhase` at both driver `Parse` sites. - Openers counted: `MODULE` (nested modules), `PROCEDURE`, `CLASS` (incl. `CLASS IMPLEMENTATION`), `RECORD` (incl. variant `CASE`), `IF`, `CASE`, `WHILE`, `FOR`, `LOOP`, `WITH`. `BEGIN` is not an opener; `REPEAT…UNTIL` needs none. - `run_tests.sh` gains a `twophase_same` gate (3 procedure-heavy files byte-compared flag-off vs flag-on). - Verified: suite **196/196** (193 + 3 probe gates); all 142 single-file tests plus session spot-checks (incl. Library1) and the compiler's own sources produce **byte-identical `.ssa`** with the flag on and zero mismatches; **fixpoint OK (3,517,333 bytes)**. - **Finding (V3 name resolution).** The first cut called the frame's `Get` from the new atg helper. gm2 accepts it (whole-scope), but the self-hosted V3 compiler resolves single-pass: the bare `Get` bound the *imported* `M2S.Get(VAR sym)` and stage 2 failed with 233s (plus knock-on 201s at `END M2P`). Fix: the helper advances via `Adv` (`M2S.Get(sym); INC(errDist)` — exactly the frame's `Get` body for a pragma-free grammar, where the scanner never returns a token above `maxT`). Rule going forward: atg helpers must never call a later-declared frame procedure by a name that collides with an import. - **Next**: milestone 2 — record body spans during the probe, add the body-entry production, and re-parse bodies after the declarations (driver-level restart), byte-compared against the default.