Inicio Explorar Ayuda
Iniciar sesión
eric
/
TurboModula2
1
0
Fork 0
Archivos Incidencias 0 Pull Requests 0 Wiki
Árbol: 63e6666f99
Ramas Etiquetas
TurboModula2
/
SESSION.md

SESSION.md 18 KB
Histórico Raw

Session save — to be continued

Date: 2026-10-02. HEAD: 7dbd516 (express-draft-v1). Tree clean.

Where we are

src/compiler/ holds the re-read compiler: COMPILER/SCANNER/ERRORS/CODEGEN fully renamed + verified clean of numbered identifiers; SYMTAB.MOD draft v1 (988 lines, 27 FIXMEs); EXPRESS.MOD draft v1 (1489 lines, 137 FIXMEs, 20 FORWARD work-list helpers); SYMTAB.DEF expanded (words 10–27); FRONTEND.md export map incl. newly found EXPRESS internals; docs/compiler/ analysis.

How we work

  • Upstream dirs git-ignored; only docs/, src/, SUMMARY.md, SESSION.md versioned.
  • Decompile in parallel background workers (3–4 groups per module), each returns Modula-2 with {/* offset */} comments; assemble + normalize via script (Compiler.word6–26, Scanner.word2–12, Doubles.proc5–11 mappings); verify with grep -nE '\b(param|local|global|proc|PROC)[0-9]+\b'.
  • Drafts are unverified by design: MCD unassemble.c diff still pending for all.
  • Commit per step, tag per milestone (*-v1).

2026-10-03: audit + proc32 applied (audit-01)

proc32 WriteOutputFiles added; 8 SYMTAB + 9 EXPRESS signature swaps applied (def headers; bodies/calls positional-consistent by construction); STMT+PASS1 all KEEP; D1/D2 resolved (map stands); CappedStrLen=StrLen alias noted. Next: GENZ80 recon + MCD-diff verification harness.

2026-10-03: GENZ80 recon + MCD-diff harness (genz80-harness-01)

GENZ80: no binary/disassembly anywhere (only call site + overlay-table strings); trigger chain partly open (no setter of nativeCodeRequested found); recovery = extract overlay from original CP/M media. Harness tools/mcdiff.sh: stage 1 green (1a clean, 1b FORWARD-OK, 1d spot-check; fixed StrLenHelper, word6, Doubles.proc5, Scanner.word2, Compiler.word3 leftovers on the way). VM builds clean (gcc, m2 binary); stage-2 run blocked on terminal I/O (documented pty approach in mcdiff.sh header). Next: GENZ80 media hunt; stage-2 pty-driven MCD diff per module.

Next steps (in order)

  1. DONE (statemen-draft-v1): STATEMEN.MOD 830 lines + frame-mechanics proof.
  2. DONE (pass1-draft-v1): PASS1.MOD 981 lines; disputes D1-D4 + proc9 recorded.
  3. Pass1 bodies (pass1.txt, 2975 lines): ~ParseType (02e6), ParseRecord, formals, decls, module heads, WriteSym — 3–4 workers.
  4. Resolve SYMTAB.MOD FIXMEs (AllocSym arity, D-group sketches, LSL) and EXPRESS.MOD FORWARDs (CentralError/PushOperand/EmitOp/BoolCondHelper/...).
  5. GENZ80 native backend (untouched so far).
  6. Recompile + unassemble.c MCD diff verification for every draft.
  7. Refresh SUMMARY.md + retag when milestones land.

SCANNER verify checkpoint (2026-10-06, uncommitted -> committing)

  • SCANNER.MOD compiles on v1.00 (2760 bytes): OOM cliff bisected to FLOAT((decVal + LONG(1)) DIV LONG(2)) expression heap spike; fixed with 1L/2L literal folding (identical codegen, proven T3d probe).
  • MCD diff vs Reloaded SCANNER.MCD: 34/41 mnemonic-identical (00-gap stripped, absolute targets masked). Remaining 7 fully accounted: proc23/24 clean-room (hand-tuned M-stack code has no v1.00 source form; algorithms decoded + restored incl. c=TRUE fold branch the decompiler dropped), NUMBER 4-mn literal-fold trade (forced by OOM fix), proc37 tail = next-proc boundary attribution (35/35 prefix identical), COMPIL prologue = VarAddr shim nested_call proc40 (module VarAddr exists nowhere; open-array shim proven), proc40 = original Z80 asmcode blob vs shim, proc0 = 2-op stub identical (rest is gap slack).
  • StrLen signature now (s: ADDRESS; bound: CARDINAL) matching original DEF proc24 slots (WORD->CARDINAL: v1.00 WORD has no comparisons); SYMTAB.MOD:221 caller fixed to pass bound 128 (proven from symtab.txt load immediate 128 at call site); CODEGEN.MOD:392 already 2-arg.
  • Padding blocks removed (Allocate/FindIden/NextCh/ScanNext) — were dead code polluting the diff; GetSym now byte-compares modulo gap zeros.

CODEGEN verify checkpoint (2026-10-12)

  • CODEGEN.MOD compiles on v1.00 (2393 bytes): 34/37 mnemonic-identical.
  • Stale renames fixed (codeBuf/EnterMod/ScanErr). keywordTable saga closed: original COMPILER.MOD also Allocate+CARDINAL-stores (proven byte-shape), CODEGEN reads low byte via limit_check+shl1+LXB = compTab[i][0] shape; v1.00 can't spell that from ADDRESS global, so local 2D-BYTE overlay (Q1/Q2-proven ADDRESS:=pointer assigns) at 2 use sites, +2mn each.
  • proc0 trailing region = embedded absolute-address tables shifted by the +6 growth (every byte -4): consequential, not semantic.

SYMTAB WIP checkpoint (2026-10-12, uncommitted -> committing as WIP)

  • SYMTAB.MOD compiles through GetAttr, dies inside EnterIdent (heap).
  • Banked fixes: TryCompile = savedEditPos>savePos (saveErr was fiction); allocHook (COMPILER.dword27, appended last); word8=scopeChain, word9=currentScope, SYMTAB.proc7=RestoreAfterEdit; transfer-index/deref bans -> Words-view params + temps; = 0/CHR(ORD())/ORD() zero-cost adaptations; unclosed comment @254 (pre-existing) fixed; DEF-leak (Files/ADDRESS) + DEF-FORWARD collisions fixed; impl-var abbreviations.
  • OOM masks type errors (VAR-mismatch, Allocate(ADR()) both OOM) — always isolate suspects in fresh-heap ZZ probes. Nesting depth tips razor-margin: flat IF+call passes, same content nested OOMs; ELSIF chain flattened.
  • Identifier chars/comments/slots/forwards all FALSIFIED as heap drivers (5604-char truncation ~1 proc; 17KB comment strip zero; +12 dummy forwards fit). Standing theory: per-scope node budget; Borland wrote smaller statements. Names/chars exonerated — readability preserved.
  • Drive holds full chain (COMPILER.SYM now WITH allocHook); ZZ probes (W1-W8, U1-U2, V1-V4, T1-T5, B-series, F1, M1, E1-E2, P1, S1-S6, Q1-Q3, H1-H2) document v1.00 codegen rules. tmp/ ignored by git (local only).

Next session plan

  1. Fold EnterIdent's IF >= 5 guard into single-depth AND (or split the arm); recompile; expect UnlinkScope/InitHeaps next.
  2. Same isolate-and-flatten loop per proc to end of SYMTAB.MOD; full MCD diff vs Reloaded SYMTAB.MCD; commit; tag symtab-verify-01.
  3. Then EXPRESS (1490 lines, biggest risk) → STATEMEN → PASS1 same loop. Budget for ZZ probes: each suspect construct gets a fresh-heap probe BEFORE bending src around it.
  4. If a module genuinely exceeds retail heap with clean nodes: timebox 2h, then switch to vm_linux instrumentation (heap high-water log) for ground truth instead of blind bisection. Do NOT rename everything terse.
  5. Refresh SUMMARY.md + retag at each verify tag.

SYMTAB session 2026-10-07 (EnterIdent heap wall)

  • GetAttr arms 101/102: dropped writebacks + sub local, reuse new (original tail-calls with jump_stack: result discarded, memoization is the effect). EnterIdent arm3: nested IF/ELSE -> two single-depth AND-IFs. Neither fixed the OOM (kept: both are the honest minimal shapes).
  • Heap-model verdict after full falsification battery (names, chars, comments, slots, forwards, locals-count, prefix-size, ASCII, $[-retention, nesting): budget is per-proc NODES; EnterIdent needs ~10 nodes over budget with every retail-legal spelling exhausted (all 6 temp-assigns proven mandatory: transfer-index/deref bans, VAR-formal bans).
  • VERSION SKEW (decisive context): retail M2.COM = "Version 1.00", writes SYM v3; shipped originals carry SYM v2 (Borland-internal/older toolchain). Only v1.00 retail on hand (other M2.COM is a different vendor product). Explains laxer-original spellings (direct ADR pushes, ADR->VAR, Buf->Identifier) that retail rejects or OOM-masks.
  • Conclusion: big modules (SYMTAB/EXPRESS/STATEMEN/PASS1) cannot compile under v1.00-retail heap with faithful sources. Next: vm_linux path — build the portable M-code VM, run retail m2.ovr under instrumented mcode.c (heap high-water log) for ground truth; heap size does not affect emitted code, so verification stays sound. Fallback: partial (prefix) verification, honestly labeled.

SYMTAB session 2026-10-07 (prefix verification via stub-tails)

  • Stub-tails method works: stub bodies from EnterIdent on (+7 DEF-stub HALTs), keep headers -> v1.00 emits MCD (1059B). Full module still OOMs in EnterIdent (unchanged wall).
  • Our MCD retains 6-char uppercase names (no directive); originals are numbered -> FINAL SYMTAB needs explicit $[+ remove (+ $[- for IMDCLS).
  • Prefix score (leave-truncated, jumps masked): 2/12 SAME* (MatchSym, EnterChained). Rest diverge STRUCTURALLY (inverted conditions, extra leaves, +30mn in PutAttr) = draft-v1 group-guess quality, not heap artifacts. Notably ours runs LONGER everywhere (temps), orig is leaner.
  • Inter-proc gap filler differs by toolchain (orig ...ff, ours ...00), excluded from scoring like dep-slots.
  • Next: re-read prefix bodies (PutAttr/GetAttr/AllocSym/FindModule first) against symtab.txt statement-by-statement; keep heap discipline (flat, minimal nodes). VM path still open for full-module compile.

SYMTAB session 2026-10-07b (nesting model + record views, prefix 7/14)

  • CALL GRAPH (mnemonic-authoritative, unassemble.c): ee=sibling/self display call (ZZ-proven under v1.00), ec=downward (parent->nested child), ed/f/ef = display-free. FORCED nest map: AllocSym{CheckNode,NewString,PutAttr}, EnterIdent{GetName,GetAttr}, CompileDispatch{35,36,37/TryCompile}, + {32}@2, {34}@33, {38}@6, {25}@24, {27}@26, {29,30,31}@28. Rest top-level. Gap-ff bytes decode as phantom calls — ignore trailing-ff "calls". proc16 dead.
  • Restructured SYMTAB.MOD accordingly (nested, forwards trimmed to the 2 cycle-breakers). v1.00 emits ee/ec exactly like the original. Zero-cost: transfers emit no code; view-field TYPES are byte-neutral (all 1-word).
  • Record-field views (LSW/SSW) replace Words indexing (LXW) everywhere: new local Entry type (CARDINAL/WORD/BITSET per dominant use); T1 direct for nodes. Proven: i * 16 folds to shl (v1.00 peephole — swap all N * x with power-of-2 N); open-array actuals push HIGH (GetName view is [0..128], not [0..127]); declared exceptions emit small codes (NodeErr=273 vs orig 272, 1B documented); ADDRESS<CARDINAL etc. need CARDINAL() views; LSWn pops base (Indirect.mod); = on arrays = inline string_comp with (addr,size) pairs.
  • Prefix score 7/14 exact (FINDMO/ENTERM/LOOKUP/MATCHS/NEWSTR/ENTERC/GETNAM). CHECKN=1B (exception code). Left: PUTATT/ALLOCS/GETATT detail (frames, guard polarity (accepted peephole class), leave-vs-jump (heap-driven flat kept), stack-threaded temps (accepted)). ENTERI wall stands (full build).
  • Slot map COMPILER.DEF re-verified 1:1 (systemScopeDesc=23 etc.).

SYMTAB session 2026-10-08 (direct-VAR arms, FindExport out, 7/14)

  • Recursive PutAttr arms now pass entry fields DIRECT as VAR (PutAttr(e^.w5, T1(sv^.word5))) + fixed SRC-vs-entry object bug (node part must be src's). GetAttr Entry views; PutAttr dedup loop is REPEAT (matches orig).
  • FindExport moved top-level (no self-ee/frame use); ImportAll 19->13 locals via honest merges (sysAdr inline, matched/sib... no: matched->findIdx, caseFlag/importEvery inline, stoImp->sysImp, expIdx->scopeIdx). All remaining overlap. 13 typed minimum exceeds retail budget (~12 plain-equiv).
  • PROVEN (6 ways): ImportAll region cannot compile under v1.00-retail (decl-count + record-descriptor weight; Borland heap ~2x). colCount-global and ListExports-out variants tested, both insufficient/worse. VM cannot raise heap (16-bit address space, high_mem already top).
  • Prefix 7/14 exact (FINDMO/ENTERM/LOOKUP/MATCHS/NEWSTR/ENTERC/GETNAM). Left: PUTATT/ALLOCS/GETATT accepted-class gaps (frames, guard polarity, leave-vs-jump, stack-temps), CHECKN 1B, SYMBOL epilogue, ENTERI wall, SYMTAB tail by construction. Next: EXPRESS (same playbook per-proc).

EXPRESS session 2026-10-08 (ground-truth verdict + compile groundwork)

  • VERDICT (byte-proven): Reversing express.txt (module base 15a5) and Reloaded COMPILE.DIR/EXPRESS.MCD (base 17fd, 6303B) are DIFFERENT BUILDS of the same source family — NOT the same file. Same proc numbering, same dep table (7 deps incl DOUBLES), 20+ procs byte-identical (e.g. proc32 43/43B), rest differs by small edits. Reloaded = newer (extra error strings " assignment", "this type"; front-proc growth +0x258 before old-proc4). Ground truth stays Reloaded MCDs (matches all verified modules + v1.00 runtime); REV txts are advisory only.
  • Old-unassemble artifacts (do NOT chase): per-file address rebase (EXPRESS REV+0x258=ours; PASS1 REV+0x3F0; STATEMEN +0x0), missed procs (EXPRESS REV lacks 1,2,3,6,10,11,22-31 = walk bug, procs exist+are called), AAAAAA ghosts, load (ix) vs load outer frame, 65535 vs -1, call_rel N vs string-print spellings. Same-dialect (old-vs-old) + immediate-normalized fp is the honest comparator.
  • PASS1 dep "skew" was a misread (truncated head): both show 11 deps incl LOADER. PASS1 old-tool output has 66 garbage heads (walk bug on that file); use new-tool disassembly as reference.
  • Committed: EXPRESS.DEF (22 exports + Desc type), PASS1.DEF stub (CondInsert/InsertEntry BOOLEAN), MOD mechanical fixes (Desc->DEF, leak-rule imports, 29x comment markers). New EXPRESS procs (no draft coverage, decompile fresh): 1,2,3,6,10,11,22-31 (tiny predicates: proc25 opt-12 guard, proc26/23/24 kind predicates, proc6/22 cond-copy, proc27 TypeSizeOf callswitch). Genuine Borland helpers, not LONGREAL.
  • Next: per-proc audit in MCD order; first fix IsOrdinal WORD>=0 via INTEGER view (MCD uses SIGNED igreater_or_equal on word5/word6).

EXPRESS session 2026-10-08 (head model + nesting map + first blood)

  • HEAD MODEL (proven): curDesc (global word2) is HeadPtr (5-word head: typ,mode,value,extra,kind) — 149 MCD accesses all fields 0-4, never 5+. Temp slots reserve 10 bytes; all copy blocks move 10 bytes (units are BYTES: R1/R2/R5/R10 -> 2/4/10/20). y := x (R10) = dest,src,20,copyblock. MOVE() is a separate opcode (not copyblock). Full descs (low/high) via full-typed params/locals; global never field-5+. curDesc never stored (link-time init). 220x curDesc^.raw[0..4] -> named fields; 6x raw[5]/[6] flagged FIXME-HEAD (draft misattribution, MCD audit pending).
  • NESTING MAP (proven by call opcodes + fct_leave bit): ec=parent->nested child, ee=nested-self/sibling (call_with_frame), ed=same/up/top-sibling. Nested fns emit plain fct_leave (no 0x80 GP-restore); top-level get 0x80 (ZZ programs always 0x80 — mask bit7 in ZZ-vs-MCD scoring!). Map: proc5>32, proc37>38, proc18>39>{40>{41,42},44,45,46,43}, EXPRES>48>49>50>51, ASSIGN>52(ghost?). Rest top-level. IsSet/IsOrdinal callers re-read (no phantom recursion: proc33 self-calls via ed).
  • AUDITED (ZZ-verified shapes): IsOrdinal nested-in-PopExprDesc byte-exact (nested-IF + INTEGER() views for signed ige + RETURN TRUE=load 1); PopExprDesc (isOrd CARDINAL via ORD(), INTEGER() wraps, IntType not Card, mode:=2, ABS(INTEGER()) wrap, EmitStandardOp(0+ORD(neg)), OR-assign); IsSet single-OR + AddressType/CardType pairs (word7/8, +2 rule) + 96 IN BITSET(field); BaseTypeOf single-OR + WordType(word18) + CARDINAL(word3).
  • Probes banked: INTEGER()/CARDINAL() zero-cost, ORD() zero-cost, CARDINAL->INTEGER assign OK, ADDRESS->RecordPtr OK, ABS needs INTEGER, CARDINAL/INTEGER mixed ops incompatible, H5Ptr(cur)^ transfer-deref banned.

EXPRESS session 2026-10-08 (ZZ verification campaign)

  • Method: per-proc ZZ modules (matching globals/imports/slots) + score.py (masks targets/procrefs; fct_leave-0x80 matches: ZZ-programs and library-top-levels both set it; nested must replicate nesting). IN-order rule: field IN BITSET(n) (not n IN ...) — pushes field first. OrElse pushes TRUE (no underflow). leaveN drops N/2 (junk cleaned). ODD() gives and; DEC/INC(v,n) give dup; BITSET{11}=2048 etc. as values.
  • VERIFIED EXACT: proc32 IsOrdinal (nested, byte-exact), proc33 IsSet (51/51), proc34 BaseTypeOf (54/54), proc5 PopExprDesc (153/153, nested IsOrdinal, CentralError not ReportError, IF/ELSIF/ELSE, OR-assign), proc35 FoldConstOp (74/74, IF/ELSIF/ELSE, inner-AND), proc7 ConstToCard, proc8 ConstToInt (13/13), proc9 MatchOpClass (14/14), proc21 ParseDesignatorBase (63/63), proc36 CheckAssignable (32/32), proc20 ParseSelector (206/206, zero mid-returns, REPEAT-extra, DEC, StrLen-tangle (S-1)=(H-L) balanced), proc16 EvalConstExpr (123/123), proc38 ParseFactor + proc37 ParseTerm (nested ee/ec). = 14 procs.
  • proc15 at 233/236 (only temp-reserve prologue missing: HeadPtr temps have no reserve source under retail strictness; Borland-lax record-to-pointer suspected — accepted toolchain delta like native proc12). WHILE (not LOOP) for outer (only EXIT-loops push_code_addr).
  • proc12 (BoolCondHelper, ×18 calls, headless) = NATIVE Z80 (accepted). proc40 (ParseSetCons) OOMs retail (heap wall); tail grind continues.
  • Nesting map confirmed by leave-bit + call opcodes; reorder + stub-tail for module-level proof after grind.

EXPRESS heap verdict (2026-10-08)

  • Full-MOD failure at PopExprDesc-END is HEAP, not syntax: identical bodies compile in small ZZs. Retail heap exhausts cumulatively during parsing; when it dies mid-parse it reports BOGUS syntax errors (not clean OOM). Clean OOM appears only if heap lasts past parsing. So full-MOD retail compile is impossible, period (even with correct sources).
  • Consequence: per-proc ZZ verification is the PRIMARY method (no heap issues, fast feedback). Full-MOD only for stub-tail prefix scoring (which also needs reorder + nesting pass first).
  • Tiny procs 23/24/25/26 bodies verified-compiling (ZZ); scoring next.
  • proc20 = 205/206 (one Borland-lax ADR(sel) delta: load local address +5 vs retail load param3; retail bans ADR on pointer-params; single-formal cannot take both value (proc15) and address-of (proc20)). Accepted.
  • proc15 = 233/236 (temp-reserve prologue only). Both near-exact, explained.
  • Verified now 21: + proc1 CentralError (7/7), proc27 TypeKindOf (24/24, CASE 98/97/94+ELSE), proc28 EmitAddress (52/52), proc23/24/25/26. proc20 205/206 (ADR delta), proc15 233/236 (reserve delta) accepted.
  • Proc24 takes NodePtr (direct from d^.typ); CheckAndStore VALUE ADDRESS (not VAR); VAR-ADDRESS accepts pointer vars/fields (not CARDINAL). Value-record params pass by reference (single push) but copy on entry (reserve_string) so not for no-copy procs. Transfers from records banned.