M2compiler-V3/docs/plan-two-phase-parse.md

Two-phase parse (design) — eliminating the forward mechanism

Goal

Delete SymTab.FwdVarRef/FwdVarName/FwdVarResolve, Check.NoteForward/ FlushForward and the Design forward branch, plus the Op=1 marker — by binding names before they are checked.

Why post-parse checking is not enough

Design defers a name it cannot find inside a procedure because it may be a module-level declaration that appears later in the file. The resolve/check pass happens after the parse, so it cannot help with binding: binding happens during the parse, and the declaration simply has not been read yet. The only fix is to read the declarations before the bodies — a two-phase (or two-pass) parse.

Design A (recommended): module-level body deferral + re-scan

The grammar/parser stays almost entirely as-is; only the order in which the parser visits text changes.

1. Skip module-level procedure bodies. When ProcDecl sees a module-level PROCEDURE, record the body's source span and consume its tokens without parsing them, then carry on with the remaining declarations. Result: every module-level name is in SymTab once the module's declarations are done.
2. Re-scan and parse the bodies. At the end of the module, for each recorded span: seek the scanner to its start and parse the body with the existing Block production. Every module-level name now resolves, so no forward reference is ever created.

Consequences:

• Design's forward branch never fires → the whole FwdVar* path is dead and removable.
• The body productions are reused unchanged (both the skip and the re-parse reuse existing machinery), unlike Design B below.

Pieces

1. Scanner seek (scanner.frm, and expose it in M2S.def's template): PROCEDURE SeekTo (p: INT32) — set bp0/bp/nextPos to p and recompute line/col (scan from offset 0 counting CHR(10); CharAt already indexes the in-memory buffer, so this is O(p)).
2. Token-level body skipper (grammar helper): starting at the PROCEDURE keyword, consume tokens counting block openers {PROCEDURE, IF, CASE, WHILE, FOR, LOOP, WITH, RECORD}; stop when the count returns to 0 — that END closes the procedure. BEGIN is not an opener (a block's BEGIN…END shares the declaration's END), and REPEAT…UNTIL/EXIT/RETURN are not counted. Verify afterwards that the token is END and the following identifier equals the procedure name (guards a mis-skip).
3. Deferral in ProcDecl (module level only): skip + record the span; the declaration node is a placeholder NkProcDecl.
4. Body re-parse: a new grammar entry (reusing Block) driven at the end of Unit, walking the recorded spans.
5. Flag: all of this behind -twophase (default off) until the suite and the fixpoint are byte-identical, then flip.

Design B (rejected): syntax-only bodies + binding walk

Make every expression/statement action resolution-free and bind/type/ check in Check, walking with scopes mirrored from the AST. This is the "true" two-phase and is much larger: it touches every body action, and parse decisions that need types (typed brace literals, method calls, L.x qualified names, HIGH/LEN) would have to be deferred too.

Risks / open questions

• Skipper exactness. A mis-count mis-skips to the wrong END. The post-skip check (END + name) turns this into a clear error rather than silent corruption.
• Scanner state after a seek (line, col, lookahead) and error positions inside re-parsed bodies.
• AST/Lower ordering. Bodies are parsed after the declarations, so node allocation order changes. Lower walks by structure (order- independent), but any parse-time ordinal that assumes file order must be re-checked.
• Nested procedures are inside the deferred span, so they are re-parsed with the same mechanism (their own locals are declared before their statements, so they need no deferral themselves).

Milestones (each with the fixpoint green)

1. SeekTo + the skipper behind the flag; inert.
2. Module-level deferral + body re-parse; byte-compare against the default across the suite.
3. Flip the default; delete FwdVarRef/FwdVarName/FwdVarResolve and Check.NoteForward/FlushForward, the Design forward branch, and the Op=1 marker.

Cost

Pieces 1–2 are a few hundred lines across the scanner frame, the grammar and the driver, and are the largest single change in the resolve/check thread — the reason FwdVar* could not be deleted by moving checks alone.