CALC-COMPILE.md 49 KB

CALC-COMPILE — make the TP3 compiler accept and compile CALC.PAS

The goal of this line of work is narrow and stated as a verdict:

shell/compiletest returns OK for Resources/TP3/CALC.PAS, and CmdCompile writes a .COM.

It is deliberately not the same goal as CALC-PORT.md (porting MicroCalc to native GNU Modula-2 on top of Term.mod). That is a separate piece of work and a separate session; nothing here changes it, and nothing here depends on it.

Everything below is measured against the tree at tag v-TP3-INTERRUPT-RESTORE (2ae00d1) unless it is marked inferred or unknown.


0. Scope decisions, fixed before any code

  1. Target the TURBO.COM dialect, i.e. the 6-byte binary REAL. CALC.PAS says so itself: it will not build with TURBO-87.COM without an 8087, and its help text notes that "Sqrt, Sin and friends are not implemented in the BCD compiler". So this is a TURBO.COM program and REAL is the 6-byte format of RESUME-TP3.md §2 ("REAL (6 octets)"), not the 10-byte BCD variant. Implement exactly one real format.

  2. Compile first, run second. The gate is the compiler's verdict, not a working spreadsheet. A construct may be incomplete only if it is loudly incomplete (Err (ENoLib)); it may never be accepted and mis-compiled. This project has already shipped that bug once. One of the two cases below is now closed (Stage 2); the other is not:

    • var parameters are silently accepted and passed by value — ProcFunc drops the var (DropB (MatchKey (tok)), comment "VAR parameter recorded as value in this milestone"); writeln (g) after q (g) prints 7, not 42. Ten CALC routines take var parameters.
    • {$I CALC.INC} was silently swallowed as a comment, so CALC.PAS without its include compiled green (program p; {$I part.inc} begin halt end. → OK code=32 data=4, part.inc never opened). Closed at Stage 2: the include is expanded before parsing and an unopenable file is error 90, not a comment.

The var-parameter case is still open and must be closed by implementation, or by an explicit Err until it is (Stage 10).

  1. One fixture and one ratchet per step. Every stage lands with a fixture that is provably red before and green after, plus a move of the ratchet (§1). Nothing is "done" on the strength of a compile that a human eyeballed.

1. The measuring device: the CALC ladder

A 1,763-line program (182 + 1,581 lines) that stops on the 111th line gives no sense of progress unless "how far did we get" is a number in a test. Make it one.

Implemented (Stage 1, 2026-10-19) — tests/fixtures/calc_ladder.tsv plus tests/calc_ladder.sh, run by run_all.sh as CALC ladder. Each row names a program and gives the exact verdict in the same shape as tests/fixtures/expected.tsv:

CALC.PAS   ERR   <errno>   <pos>

The script stages a copy of the named program — and of the companions that sit beside it, so CALC.INC is already there for Stage 2 — into tmp/calc_ladder/, runs the freshly built compiler on it, and compares. It does not build the compiler itself: it calls tests/run_compile_tests.sh --build-only, which is the one recipe that freshness-checks compiletest, so the ladder cannot become the second staleness test that drifts from the first.

The copy is deliberate: the include (Stage 2) resolves relative to the including file, so CALC.INC has to be beside CALC.PAS in the staged tree, and the row then measures the compiler rather than an edit-in-place of Resources/TP3. It is not a fixture in tests/fixtures/*.pas (which would enter the compile/COM/exec matrices and need a second copy of CALC.PAS kept in sync).

Each stage updates the row; the row may only move forward. Two traps are written next to the file itself:

  • pos is not a file offset. LoadFile collapses CRLF and lone LF to a single CR, so pos indexes the normalised buffer. CALC.PAS is 11,241 bytes on disk and 11,059 bytes normalised; the reported pos 7791 is file byte 7901. Every expected value in the ladder must be computed in normalised coordinates. Once {$I} lands, the buffer is CALC.PAS with the directive replaced by CALC.INC, so all downstream positions shift again — which is exactly why this must be a computed number, not a remembered one.
  • Nobody edits the ladder row without editing the fixture in the same commit, per the repo's re-baselining rule.

Current rung (measured today, normalised position):

position blocker
9202 MCFile : file of CellRec — a file type (ParseType TkFile, ERR 102)

Rungs behind it (now passed):

was blocker done by
7791 FXMax : Char = 'G' — scalar typed constant Stage 4 (scalar)
7791 EofLine = ^M — caret character constant Stage 3
7972 Numbers : set of Char = ['0'..'9'] — set typed constant sets, increment 1
8098 Attributes = (Constant,…) — enum; and SetOfAttri = Set of Attributes Stage 5 (enums)
8362 CellRec = record … end — record type Stage 7 (record types)
8705 Cells = array['A'..'G',1..FYMax], Sheet : Cells, XPOS : array[…] = (…) Stage 8 (declarations)

Rungs ahead:

position blocker
9202 the var block's file of, then {$I} at 9404, and everything in CALC.INC

Corrections the measurements force, worth stating plainly because the stage list below was written before them:

  • The include (Stage 2) does not move the rung. It is at normalised 9404, past the stop, so Stages 3-4 had to happen before the include is ever reached.
  • 7791 → 8036 in one stage was optimistic. CALC.PAS:114, Numbers : set of Char = ['0'..'9'], is a typed constant whose type is a set, and it comes before the enumeration at 8036. Sets had to go first, and now they have — the compiler stops at 8098, on the enumeration. The measured ladder is the authority; the stage numbering below is the original plan, not the dependency order.

2. The census — what CALC actually needs

Counts are from CALC.PAS + CALC.INC with comments and string literals stripped.

2.1 Type and language constructs

construct where status today
{$I CALC.INC} CALC.PAS:150 done (Stage 2) — expanded before parsing; t56/t57/t58
typed constants (X : T = v) — scalar CALC.PAS:111 done (Stage 4) — declared class kept; t59_typedconst
typed constants — set CALC.PAS:114 done (sets, increment 1) — code-resident bytes; t61_setconst
typed constants — array CALC.PAS:137; CALC.INC:777 ERR 102 — needs initialised storage
lexical ^M, #nn CALC.PAS:113 done (Stage 3) — t60_charcaret
enum type (Constant,Formula,…) CALC.PAS:120 done (Stage 5) — ordinals; bounds carried; t62_enum
subrange types 'A'..'G', 0..20, string[70] CALC.PAS:117,119,130 OK
set of Char / set of <subrange> CALC.PAS:114 type + sizing done (hybrid bytes)
set of <enum> CALC.PAS:121,127 sizing done (enum bounds → 1 byte for Attributes)
set constructor [a..b, c], in (compile-time) t61_setconst done — folding only
set in (run-time), +, *, =/<>, assignment, set variables 47 × in, 6 × + [, 2 × * ENoLib — next sets increment
record + field select Sheet[FX,FY].CellStatus CALC.PAS:126 type done (Stage 7) — CellRec = 80 bytes; field SELECT is next
with 13 sites after field selection
2-D array array['A'..'G',1..FYMax] of CellRec CALC.PAS:133 type + variable done (Stage 8) — Sheet = 11760
array typed constant XPOS : array[…] of Integer = (…) CALC.PAS:137 done (Stage 8) — code-resident
array element access b[i] (constant index), r.f t14, t65 done — width on the expression
array element access with a VARIABLE index Sheet[fx,fy], with 13 with sites ENoLib — scaled addressing next
array indexed by an enum type CALC.INC:773,927 ERR 41
pointers ^Pages, New, Dispose CALC.INC:579,587,639,690 ERR 41
var parameters 10 routines, 14 occurrences (e.g. CALC.INC:765,979,995,1036) silently by value
self-recursion Fact — Fact := I * Fact(I - 1), CALC.INC:747,754 ERR 1 at the (
mutual recursion via forward Expression fwd CALC.INC:742 ↔ Factor calls it at 820,932 forward parses; definition untested
forward definition repeating only the name CALC.INC:1036 (params commented out) to check
nested procedures / functions DisplayMessage in Help; Fact/Factor in Evaluate parses OK
local type/const/var sections in a nested function CALC.INC:767-778 ERR 41/ERR 102
case labels ^E, ^X, ^J, #27, comma lists CALC.PAS:169, CALC.INC:91,1200,1567 comma lists/^X/# unknown
file of CellRec, text CALC.PAS:144, CALC.INC:582 ERR 102
lexical #27 and ^M char constants many ERR 41 at the #/^
write/writeln/read with a leading file argument CALC.INC:562,625 not implemented

Not used by CALC, so explicitly not needed: downto, goto, absolute, shr/shl, packed, Pos, Concat, Round, Trunc, Frac, Ord.

2.2 Identifiers used but not declared in the source

This is the exact builtin surface. strPool/strOff/strLen are compiler tables, so this list is also the KProc/KFunc work list.

category builtins (count in CALC)
present today write 41, writeln 5, read 15, readln 1, halt 1
strings length 19, copy 5, delete 3, insert 1, str 3, val 5
ordinal chr 5, upcase 12, succ 5, pred 1
reals ln 5, exp 3, abs 3, sqrt 2, sqr 1, sin 1, cos 1, arctan 1, int 1, str/val above
heap new 1, dispose 1
files assign 5, reset 3, rewrite 2, close 4, endoffile 5, ioresult 1
CRT gotoxy 25, lowvideo 16, normvideo 16, clreol 10, clrscr 6, keypressed 3, delay 1, kbd (8 × Read(Kbd,Ch))
directives {$I CALC.INC} (include), {$I-}/{$I+} (2), {$A-} (1, harmless/ignored)

Also needed by name for the declarations to resolve: predefined types integer, char, boolean, real, string, text, file, and the predefined constants true/false (both work today).


3. Structural blocker: the memory layout is inverted

This is not a feature; it is a hard prerequisite. CALC's code is 16,560 bytes (§3.1, measured) and this port refuses to emit more than 4,096. It has to be settled as one of the first things, not "before the fifth stage" — until it moves, every later stage can only be developed on fixtures a fraction of CALC's size, and the ladder cannot advance past the first few rungs.

What this port does today. pc starts at rtSz (773) and dataBase := rtSz + 1000H (4869). Code occupies [rtSz, dataBase) and data occupies [dataBase, …). The data sits above the code, so the enforced limit is code ≤ 4 KiB, reported as TP3 error 98 by Ebyte and pinned by t43_codewindow (v-TP3-CODEWINDOW). MaxCode is 24,000, so the buffer is not the constraint — dataBase is.

What the original does. TPSRC7:81-82:

	MOV	pc,#$2D7C	;PC: start of code
	MOV	dc,#$0240	;DC: start of file list

pc (the code position) starts at $2D7C = 11,644 and dc (the data counter, TPSRC7:1286-1287, vardef2 uses AX := dc as the global's offset) starts at $0240 = 576. That is the opposite orientation: data low and growing up, code above it and growing up. Inferred consequence: in the original the binding limit is data running into the code base, not code running into the data. That is consistent with the original having a runtime measured in hundreds of bytes while reserving ~11 KiB between $0240 and $2D7C.

Why CALC makes this binding. Measured data budget:

item bytes
Sheet : array['A'..'G',1..21] of CellRec, 147 cells × 80 11,760
CellRec = set of Attributes 1 + String[70] 71 + Real 6 + DEC,FW 1+1 80
XPOS : array['A'..'G'] of Integer = (…) (typed const) 14
Line : string[100], FileName : string[70] 101 + 71
Help.Buffer : ^Pages with Pages = array[0..200] of string[80], New (Buffer) (CALC.INC:579,587,639) 201 × 81 = 16,281 on the heap

So ≈ 12 KiB of globals plus ≈ 16.3 KiB of heap, on top of code and stack, in one 64 KiB segment. MaxCode = 24,000 gives code room; the layout is what has to change.

This question is now closed — see §3.1, which measures the original's output directly. What remains open is which of the two candidate fixes below to implement, and that is a decision for the session that does the work, not an unknown.

Method: read the .COM write path in TPSRC7 from copyrt and the cdptr/cdbegpt/cdfoff handling through the MOV AX,dc at TPSRC7:1286, and cross-check against RESUME-TP3.md §3.1 (program skeleton) and §3.4. Resources/turbopascal3source/TP3/ is the ground truth by project rule; when the book and the disassembly disagree, the disassembly wins. Do not decide this from our dataBase constant and a plausible story.

The two candidate fixes, for reference, once the question is closed:

  • Match the original: data at a fixed low base (dataBase := rtSz), code at a fixed higher base, code unbounded above. Cheap, faithful, one orientation change — but wastes the gap and needs a real data-window number.
  • Dynamic data base: emit global addresses as fixups and resolve them once the final code size is known (FixUp kinds already exist on the Runtime side, .def comment "FixUp, kind 2"). More correct, more invasive: every emitter that bakes a DS offset has to register it.

Either way t43_codewindow and the Ebyte check must be re-baselined deliberately in the same commit as the change, with the reason in the message.

Cross-check worth making while you are in there: the original's global overflow raises $62 = 98 ("Memory overflow", TPSRC7:1286-1288), which is the same number our Ebyte uses, but CALC.PAS's own banner warns about "Compiler Overflow - Error #99". If 98 and 99 are two different limits — data into code vs. something else — the banner is telling us which one CALC hits and which one the 'C' (.COM) workaround moves. That is cheap evidence and it bears directly on the fix.

3.1 Reference measurements (resolved 2026-10-17)

Compiled on a real DOS (dosemu) with the original TURBO.COM, from a directory holding CALC.PAS, CALC.INC and CALC.HLP, using the .COM destination (O, C, Q, C, Q) exactly as CALC.PAS's banner instructs. A six-line HELLO program was compiled the same way as the baseline. The two artifacts are sitting in Resources/TP3/ for analysis and are deliberately not committed — they are derived from the proprietary TURBO.COM; the recipe above regenerates them, and the numbers are what this document keeps.

The compiler's own statistics line:

HELLO CALC
lines 6 1,763
code 0005 paragraphs = 80 B 040B paragraphs = 16,560 B
code free 0D26 par = 53,856 B 0920 par = 23,360 B
data 0002 paragraphs = 32 B 02F0 paragraphs = 12,032 B
data free 0FD9 par = 64,912 B 0CEB par = 52,400 B
stack min / max 16,384 B / 655,360 B 16,384 B / 655,360 B
.COM file 11,410 B 27,901 B

Image layout (both files share it):

  • File offset 0 is a 3-byte entry jump E9 4A 2C → address 0x2D4D (= file 0x2C4D; the .COM offset-to-address bias is 0x100, the same constant as this port's LoadBias). So the fixed TP3 runtime is file [0, 0x2C4D) = 11,341 bytes, identical in both programs.
  • At 0x2D4D: call TU_PARAGRAPHS, then the 20-byte program header (options 0006, cs 0813, ds 11D7, …), then mov bp,sp, call TU_TAILLE_CODE, the code-size word (0x4092 for CALC = 16,530, i.e. the reported 16,560 minus the 30 bytes of preamble), the SAUT_PRINCIPAL address (0x2D6D in both), and jmp PRINCIPAL.
  • Typed constants live in the code stream, exactly as RESUME-TP3.md §3.14 says. Located byte-for-byte in CALC.COM: FXMax = 'G' (47) at 0x2D70; Numbers : set of Char = ['0'..'9'] as 32 bytes with FF 03 in bytes 6-7; XPOS as the seven words 03 00 0E 00 19 00 24 00 2F 00 3A 00 44 00 at 0x2D91; and StandardfunctionNames (the nested function's string[6] array) as length-prefixed 'ABS', 'SQRT', … around 0x4700.

Consequences for this port:

  1. Q1 is answered. The original keeps the data area below the program code, so the binding limit is data running into the code base — the exact opposite of this port, whose dataBase := rtSz + 1000H caps code at 4 KiB. CALC's code is 16,560 bytes, so the port cannot emit it at all today; this is a hard prerequisite, not a nice-to-have.
  2. MaxCode = 24,000 is not the binding constraint. 16,560 + 773 (our runtime) = 17,333, comfortably inside 24,000. It is dataBase alone that must move, and the fix has to be an orientation/layout change rather than a bigger buffer — which is what t43_codewindow was built to police.
  3. The §3 data budget is confirmed, not inferred. Reported data is 12,032 B against a computed 11,760 B for Sheet alone (147 × 80 B CellRec), so the 80-byte CellRec sizing is right and the rest (Line, FileName, the file variables, scalars) accounts for ~272 B.
  4. TP3's memory model has code and data in separate address spaces (cs 0x0813 vs ds 0x11D7 in the header, and separate "free" figures). A .COM collapses them to one segment, so we have to choose one contiguous layout; the measurements bound what we must allow: ≥ 16,560 B of code and ≥ 12,032 B of data, with 16,384 B of stack and a heap large enough for Pages (16,281 B, §3).
  5. The runtime is 11,341 bytes in the original. Ours is 773 B. Not needed for this goal, but it explains where the original's fixed pc := $2D7C / dc := $0240 came from, and why TPSRC7:81-82 looks the way it does.

Feasibility, now checkable. A .COM is one 64 KiB segment holding [runtime][code][data], with the heap growing up and the stack growing down out of what is left. Using the measured figures:

runtime + code + data left for heap+stack after Pages (16,281 B)
original 11,341 16,560 12,032 25,603 9,322 for stack
this port 773 16,560 12,032 36,171 19,890 for stack

So the goal is arithmetically reachable, and the original proves it is reachable with 10,568 bytes less room than we have — our runtime is that much smaller. There is no capacity reason to doubt the layout fix; the only question is which of the two orientations to implement. The same table shows MaxCode = 24,000 is fine with 7,440 to spare, and that TextLimit's 488 bytes of slack (§5) is the one genuinely tight number in the whole budget.

Still open from this experiment: whether memory-mode compilation of CALC reports "Compiler Overflow #99" as the banner warns. That is one more run of TURBO (step 1 of the banner, R instead of the .COM path) and it is the only remaining piece of the 98-vs-99 evidence.

Not resolved by these measurements: the meaning of the two header words after ds (the book calls them code and globales; they read 730/39 for HELLO and 1760/789 for CALC, which match neither the reported code/data nor each other). They are not the code and data sizes, so the book's field list should not be trusted here — the statistics line and the file layout above are the authoritative numbers.


4. Ordered stages

There are two parallel tracks, and it is worth being explicit that they are parallel:

  • The ladder track (Stages 1-15) is front-end work. It advances the error position in CALC and needs no memory-layout change to be developed or verified — every feature can be exercised on a fixture far smaller than 4 KiB of code.
  • The layout track (Stage 16, §3) is the only thing that stops the compiler from emitting the whole program once the front end accepts it. It does not block the ladder, but it is a hard prerequisite for the goal as stated, and it changes dataBase/t43_codewindow — so it belongs to one commit of its own, with the re-baselining reasoned in the message.

Ordering rule within the ladder: cheapest-and-most-independent first, then the ladder order (each stage removes the blocker the previous one exposed).

Session state — where to resume

Last touched 2026-10-19. Stages 1-4 are landed — the measuring device, the {$I} include, the lexical character constants, and the scalar typed constant. The ladder row now sits on sets.

  • HEAD ce7e8cd (the plan) plus the Stage 1, 2 and (this one) 3+4 commits. Nothing pushed; origin/master is behind.
  • Stage 1 (the ratchet) is landed: tests/fixtures/calc_ladder.tsv + tests/calc_ladder.sh, wired into run_all.sh as CALC ladder. The row was proved red first and may only move forward; it has moved once: ERR 1 7791 → ERR 102 7972.
  • Stage 2 ({$I file}) is landed. ExpandIncludes resolves includes as a pre-pass (not in Skip, because DeclaresProc's lookahead rewinds srcPos), Compiler.SetSource gives the compiler the source path so an include resolves beside its including file, and the errors are TP3's own: 90 file-not-found, 96 illegal nesting. Fixtures t56_include (OK, and its .out proves the included procedure runs), t57_include_missing (ERR 90), t58_include_nest (ERR 96). TextBuf.InsertStr is the new bulk, capacity-checked paste.
  • Stage 3 (^X/#nn) and Stage 4 (scalar typed constants) are landed. RdConst reads ^M and #nn, ParseAtom accepts them as literals, and DefConst accepts name : Type = value for a scalar type and keeps the declared class. Fixtures t60_charcaret and t59_typedconst. The scalar typed-constant red proof is the interesting one: dropping the class keeps the compile matrix green at the same OK 65 4 and only execution sees 71 for 'G'.
  • Sets, increment 1, are landed — the hybrid type, the constructor, set typed constants, and compile-time in. Fixture t61_setconst (.out TRUE FALSE TRUE TRUE), and the storage decision is recorded in Stage 6: a typed constant's bytes are emitted in the code stream behind a jmp (as string literals already are), addressable through DS because a .COM has one segment — no seg cs, which is the single divergence from the original. Still ENoLib: run-time in, set variables, +/*/=/<>, assignment.
  • Enumerations are landed (Stage 5) — ordinals 0..n-1, one-byte type, bounds carried so set of Attributes sizes to one byte. Fixture t62_enum. RdConst now resolves a named constant, which set literals and case labels need.
  • Array TYPES, array VARIABLES and array TYPED CONSTANTS are landed (Stage 8, declarations half) — multi-range array types with row-major sizing stored in an ArrRec (Sheet = 11760), globals of array type, and XPOS : array[…] of Integer = (…) emitted code-resident like a set. Fixture t64_arrays (compile-only: its ~16 KB image is past the exec harness's 8 KB boot-sector read). t14 re-baselined: the array DECLARES now; the refusal moved to element access.
  • Array/record ELEMENT and FIELD access is landed for CONSTANT indices (constant subscript + .field), with the value's width carried on the expression (ERes.sz/VarSize). Fixture t65_array_access; t14 re-baselined ERR 102 109 → OK 41 15. A VARIABLE index (and with) is the next increment and is still loud.
  • The ladder now stops at ERR 102 at 9202 — MCFile : file of CellRec, the file type in the var block. So the next stage is files, then the {$I} at 9404 and the CALC.INC body.
  • One surprise on resuming: the working tree was found holding the nonvacuity script's frame-rule mutation of shell/Compiler.mod (the locFree := 0FFFEH / pre-decrement-base revert), i.e. a mutation from a tests/nonvacuity.sh run that the trap did not restore — presumably killed before the restore. It was reverted with git checkout -- shell/Compiler.mod before anything was measured. Worth knowing: the mutation's whole shape is byte-for-byte the same for INTEGER locals, so the compile matrix would have been green and only t40_readln (and later CALC) would have noticed.
  • Verified baseline, run on this commit: ./tests/run_all.sh → OVERALL: ALL PASS, compile matrix 66/66, CALC ladder 1/1 at ERR 102 9202, 57 .COM linked and 55 executed under both qemu and Exec86 (t64_arrays is compile-only: its image is past the 8 KB boot-sector read).
  • tests/nonvacuity.sh: re-run after this commit's cases (the set membership bit among them); see the commit message for the count. It is not in the default run_all.sh and must be run manually after touching any check.
  • Still present and deliberately untracked: Resources/TP3/CALC.COM and Resources/TP3/HELLO.COM, the two reference binaries used for §3.1. They are derived from the proprietary TURBO.COM; do not stage them. The recipe and the numbers are in §3.1, which is what this document keeps.
  • Corpus for later, already in the tree (untracked): Sources/Dezip/ holds 5,887 files, of which 4,919 are .PAS — plus 122 .EXE, 53 .OBJ, 26 .TPU and a SWAG collection (swag9702, swag9708, swag-pascal-master, …). Sources/ is a user directory and is never staged; it is here as test material only. Two things it enables, neither of which this plan currently has:
    1. A population metric — how far does the compiler get across thousands of real programs, not one. That is a far stronger regression signal than the CALC ladder alone, and it would have caught bug classes that a single hand-written fixture cannot.
    2. Dialect triage — not all of it is TP3. Expect TP1/TP2/TP4, DOS interrupt units, and units (.TPU) this compiler has no linker for. Any batch driver must report "rejected" as a result, not as a failure, or the corpus will look like a catastrophe rather than a baseline. Do this after the ladder, not before: measure the baseline once, so later stages can be shown to move it. A first pass needs a batch driver that feeds compiletest a file list (it already reads filenames from stdin, one per line) and tallies (verdict, errno) — cheap, and it turns the corpus into a number the moment it exists.
  • Cheap evidence still missing (Q6): one memory-mode run of TURBO in dosemu to see whether CALC reports "Compiler Overflow #99" as its banner warns. The .COM path succeeds; the memory path was never tried.
  • The one decision not to defer past the first sitting: which candidate fix to implement for §3 — match the original's orientation, or a dynamic data base with fixups. Everything else in the stages below is mechanical.

Next session

The next rung is an ENUM. The ladder stops at ERR 41 8098 on Attributes = (Constant, Formula, Txt, OverWritten, Locked, Calculated): a type whose definition is a parenthesised list of identifiers, which ParseType has no arm for. Land the enumeration type next (Stage 5), then the set-over-enum lines at CALC.PAS:121,127 follow immediately, and the rest of the set operations (run-time in, set variables, +/*/=/assignment) alongside.

The storage decision that sets needed is settled and recorded (Stage 6): a typed constant's bytes live in the code stream behind a jmp, addressable through DS; no seg cs, because a .COM is one segment. That is the same decision the enum-indexed array of strings (CALC.INC:767-778) and XPOS will use, so it does not need re-litigating.

Stage 1 — the ratchet (§1). DONE. tests/calc_ladder.tsv + tests/calc_ladder.sh, wired into run_all.sh; proved red against 7792 before it was set to 7791. It also resolved Q4 in favour of staging copies in tmp/calc_ladder/ rather than teaching the compile matrix about companion files.

Stage 2 — {$I file} include. DONE.

Implemented as a pre-pass, not in Skip, and that is the one design point worth recording. Skip is called by lookahead routines (DeclaresProc, PeekKw, KwAhead) that save and restore srcPos; splicing inside Skip runs once during the lookahead and again for real, and the second pass sees the first splice's text and reports error 96 (and a splice grows the buffer, which a rewind cannot undo). So ExpandIncludes resolves every include in one forward scan before parsing, and the parser — including DeclaresProc's scan for a PROCEDURE keyword, which must see a procedure that lives only in the include — sees one finished buffer. The scan recognises what the lexer does (a quoted literal, then star-paren comments, then brace comments, then a brace directive), so {$I} written inside a comment stays inert (t56 depends on it).

  • {$I <name>} is spliced at the directive (buffer grows, srcLen updated) in the expanded-buffer coordinates the ladder already measures in; {$I+} / {$I-} remain the I/O-check toggle (dirs.chk, CALC.INC:396,399).
  • Path resolved against the including file's directory — the compiler now gets the source path through Compiler.SetSource, called by the three loaders. Extension defaults to .PAS when absent (TPSRC10 kpasext).
  • An unopenable file is error 90 ("File not found", TPSRC10 cdincl errb) at the directive; a nested include is error 96 ("Illegal nesting of include files", TPSRC10 cdincl). Both measured, not assumed.
  • {$A-} is in fact a real TP3 directive (TPSRC10:783, Absolute code) that TP3 accepts and this port ignores; the "not a TP3 directive" note was wrong. The rule that mattered is kept: an include can never fall through to the comment path.
  • Fixtures: t56_include (+t56_include.inc, red 41 → green OK, and its .out proves the included procedure runs), t57_include_missing (red OK → ERR 90), t58_include_nest (+t58_include_a.inc, red OK → ERR 96). Q4 resolved: the compile matrix needs no companion-file notion, because the .inc simply sits beside the .pas in tests/fixtures/ and the include resolves there.
  • The ladder does not move at this stage. CALC.PAS's include is at normalised 9404, and the compiler stops at 7791 — the typed constant of Stage 4 — so the include is never reached for CALC.PAS yet. Stage 2 is pinned by t56/t57/t58 until Stage 4 lets the compiler get to it; the rung moving is a Stage 4 event.

Stage 3 — lexical #nn and ^X character constants. DONE.

RdConst reads ^M (the upper-cased character minus 64, so ^@ = 0 and ^[ = 27) and #nn (decimal, via RdIntConst, so #$FF works too). ParseAtom's literal test had to admit ^ and # as well, or write (^M) in an expression would still be a syntax error — only a declaration goes through RdConst. Both spellings are marked as characters, so they are written by the char writer. EofLine = ^M (CALC.PAS:113) now parses. Fixture t60_charcaret (red: ERR 41 at the ^M in write → green; its .out starts with the raw bytes 0D 1B, which is what proves they are characters and not the text "13"/"27").

Stage 4 — typed constants: the SCALAR form is DONE; structured is deferred.

DefConst takes the name : Type = value spelling (TPSRC7's constant reader) and, for a scalar type, records the declared class on the symbol. Because the value still folds to an immediate, that is the whole payoff: FXMax : Char = 'G' becomes a TChar constant holding 71, so write (FXMax) prints G rather than 71. Integer and Boolean take the same path, which also pulled in RdConst reading TRUE/FALSE (it could not before). Fixture t59_typedconst — and its red proof is worth noting: dropping the class (cls := typCls → cls := cls) leaves the compile matrix GREEN at the same OK 65 4 and only execution sees 7142 where G42 belongs.

What is not done, deliberately, is a typed constant of a structured type:

  • A set (Numbers : set of Char = ['0'..'9'], CALC.PAS:114) or an array (XPOS : array['A'..'G'] of Integer = (…), 137) is an initialised value with a layout. This compiler has no initialised data — InitMem zeroes the data area — so accepting one and discarding the initialiser would be exactly the silent mis-compile §0.2 forbids. DefConst refuses a type bigger than a word with ENoLib, and that is where the ladder now stops: ERR 102 at 7972.
  • So the plan's 7791 → 8036 was optimistic: sets (Stage 6) must be laid out before the enumeration at 8036 can be reached. The enum-indexed array of strings in a nested function (CALC.INC:767-778) needs the same storage decision, on top of sets, arrays and string[n] — it is the hardest single item in this plan and belongs to its own write-up, not this one.
  • The original keeps typed constants in the code stream, reached with seg cs, with the program jumping over them (RESUME-TP3.md §3.14); the decision between that and an init block copied at startup is still open, and is now the first thing the set/array typed-constant work must settle.

Then, in ladder order

Stage 5 — enumerations. DONE.

Attributes = (Constant,Formula,Txt,OverWritten,Locked,Calculated). Each enumerator becomes a constant holding its ordinal position (0..n-1); the TYPE is one byte (TP3 uses one below 256 values, RESUME-TP3.md §2) and carries its bounds 0..n-1 in the symbol, which is what set of Attributes sizes its base from — one byte, against 32 for set of Char. ParseType grew the ( arm, SymEntry grew lo/hi, and RdConst now resolves a NAMED constant, so [Constant, Formula] and (soon) case labels over enumerators work. Succ/Pred over enums are a later milestone (they are the standard-function milestone's job, not the type's). Fixture t62_enum. Ladder: 8098 → 8362 (the enum AND SetOfAttri = Set of Attributes).

Stage 6 — sets. Increment 1 DONE; the rest is the next milestone.

Done:

  • Type sizing — TP3's hybrid layout (coeur-tp-ocr 4.7.2): only the whole bytes covering the base range are stored, element e is bit (e MOD 8) of byte ((e DIV 8) - (base DIV 8)), high byte last. set of Char is 32 bytes, set of 12..17 is 2. elem carries the origin (a multiple of 8).
  • The constructor [a..b, c], and [].
  • Set typed constants — Numbers : set of Char = ['0'..'9'].
  • Compile-time in — a constant element against a set whose bytes are in the image, answered as TPSRC2 xsetin does (AH > 255 → FALSE, else test the bit). Fixture t61_setconst, .out TRUE FALSE TRUE TRUE.
  • The storage decision the plan demanded, and it is the original's own: a typed constant's bytes are emitted in the code stream behind a jmp, exactly as string literals already are, and the name becomes addressable. The divergence from RESUME-TP3.md §3.14 is one line: the original had separate code and data segments and therefore prefixed seg cs; a .COM has one segment, so DS already covers the code area and no prefix is needed. The bytes are ordinary RAM, so a typed constant stays assignable, as TP3's static typed constants are.

Still to do (all currently ENoLib, none of them silently accepted):

  • Run-time in — a variable element or a set variable. The bit arithmetic is the same, but it must be emitted, not folded.
  • Set variables — a 32-byte local/global (DefVar still refuses size > 2 for a non-string) and their layout in the frame.
  • + union, * intersection, =/<>, assignment, [] in context.
  • set of <enum> — needs the enum base first (the current rung).

Reference note: Resources/COMPIL4/ (Wirth's Pascal-P4, P.PAS + P41..P48) was added as a cross-check. It is a different design — a fixed setsize = 8-byte bitset over 0..sethigh (63), with in as a built-in runtime opcode — so it informs the algorithms but not the representation. TP3's hybrid bytes remain the ground truth here, per the repo rule. Ladder: 8060 → 8237 (now 8098 → the enum).

Stage 7 — records. Record TYPES done; field selection is the next increment.

Done: the record … end arm of ParseType. Fields are non-compacted and juxtaposed (RESUME-TP3.md §2); each name in an a, b : T group is its own field at the running offset. Fields live in a table of their own (FldRec, keyed per record) because a field name is scoped to its record, not global. A type carries its field range in SymEntry.fldFirst/fldLast. CellRec comes out at exactly 80 bytes, which is what §3 measured Sheet against, so the layout is pinned by t63_record's data=84. This also required subrange sizing: 0..20 is one byte (both bounds in 0..255, RESUME-TP3.md §2), not the two this compiler used to give it.

Not yet: field selection (c.DEC, Sheet[FX,FY].CellStatus) and with. A field access needs the expression record to carry a width (a field's size is not its variable's), which is a small ERes change plus the addressing; today it is a loud error, not a wrong address. Records as array elements come with the 2-D array in Stage 8 — Sheet[FX,FY].CellStatus is both.

Stage 8 — arrays at their point of use. Declarations DONE; element access next.

Done: multi-range array types (array['A'..'G',1..FYMax] of T), row-major with the last index varying fastest (RESUME-TP3.md §2); the shape is kept in an ArrRec (n, lo[], hi[], elem) and the symbol points at it (SymEntry.arrIdx), so indexing has everything it needs. Array variables (global, and of any size) are laid out, and array typed constants (XPOS : array['A'..'G'] of Integer = (…)) are accepted: their bytes are emitted code-resident, exactly like a set, each element little-endian. Sheet is 11760 bytes and XPOS's 14 bytes are in the code stream, which is why t64_arrays's data=11764 is Sheet plus the two @@T temporaries. t14 is re-baselined: the array declares now, and the refusal moved from the declaration (pos 83) to the subscript (pos 109).

Element and field access are DONE for constant indices. An expression now carries its own WIDTH (ERes.sz, via VarSize), because an element's size is not its array's: a constant subscript folds (index - lower bound) * element size into the address and hands the element's size and class to the store, and r.f adds the field's offset and takes the field's size and class. The width is threaded through LoadAtom, Assignment and IoCall, and Statmnt (which builds an assignment target directly, not through ParseVar) had to set it too

  • that omission is what the first cut got wrong. Fixture t65_array_access (v[1] := 11, v[3] := v[1] + v[2], r.A, r.B; .out 33 then 12), and t14 is re-baselined from ERR 102 109 to OK 41 15.

Not yet: a VARIABLE index (Sheet[FX,FY], XPOS[Ch], b[i]) - scaled run-time addressing, including the second dimension - which is a loud ENoLib today, and with. Both wait on the same scaled-addressing work.

Two consequences for the harness, both now recorded:

  • check_8086 clause F had to narrow. It asserted "this region sweeps clean, therefore it is pure code, therefore no 0F". A code-resident set or array constant breaks that premise - the data can decode cleanly and then present an 0F that is a data byte. Fixtures that emit code-resident constants are now listed in CODE_RESIDENT_CONST and clause F is skipped for them; the runtime sweep (the backbone) and the execution oracles still cover the emitted instructions.
  • The execution harness reads 8192 bytes. tests/exec/bootcom.s reads 16 sectors, and t64_arrays's image is already ~16 KB - so it is compile-only, and CALC's measured .COM is 27,901 bytes. Running CALC therefore needs the boot sector's read raised as well as the layout change; both are on the execution path.

Stage 9 — pointers and the heap. ^T (4 bytes, offset then segment), New/Dispose, Buffer^[i] read/write, Buffer^[0] := st (assignment to a string through a pointer). The heap must hold ≥ 16,281 bytes for Pages, which is a §3 budget item, not a detail.

Stage 10 — var parameters, by reference. RESUME-TP3.md §3.11: the caller pushes SEG then OFFSET (4 bytes), the callee reads LES DI,[BP+off], and ret imm frees 2 bytes per value parameter and 4 per var parameter. Ten CALC routines need it (14 occurrences). Also remove the silent by-value acceptance in ProcFunc in this same commit — until the SEG:OFF push exists, var must Err, not quietly pass by value.

Stage 11 — real, 6-byte. The largest single item, and the one to plan in its own write-up.

  • storage (declaring a real variable is ERR 102 today), literals (0.0, 1.0, 34.0, 1.5, exponentials).
  • + - * / and the six comparisons over the 6-byte format.
  • and/or on the results (e.g. (I > 0.0) and (I < 34.0)).
  • integer↔real conversions.
  • Str/Val with widths and decimals (CALC's display and input path).
  • ln, exp, abs, sqrt, sqr, sin, cos, arctan, int.
  • a recursive real function with a by-value real parameter is the natural end-to-end test: Fact (CALC.INC:747), Fact := I * Fact (I - 1) (754).
  • Runtime: nothing real-related exists in the 20 current RT_Entry selectors. TU_* equivalents are in TPSRC1/TPSRC2.

Stage 12 — the string builtins. length, copy, delete, insert, str, val, plus named string constants (const s = 'abc' is ERR 102 today) and string-typed-constant arrays. Also revisit the data cost: each + currently reserves a fresh 256-byte global scratch (NewScratchStr), which for CALC's concat sites buys ~2 KiB of the budget in §3 — it needs to be sized to the operand, or reused.

Stage 13 — files. text and file of CellRec; Assign, Reset, Rewrite, Close, EndOfFile, IoResult; read/write with a leading file argument (writeln (MCFile, Line), Readln (H, Buffer^[count])). DOS INT 21h open/create/read/write/close in the runtime, with text-mode newline and ^Z semantics (RESUME-TP3.md §2). {$I-}/{$I+} (Stage 2) already gate the checking that IoResult reports.

Stage 14 — the CRT identifiers. GotoXY (25), LowVideo/NormVideo (16 + 16), ClrEol (10), ClrScr (6), KeyPressed (3), Delay (1), Read (Kbd, Ch). There is no unit system, so these become builtins with runtime entries: GotoXY and ClrEol from the cursor position, video attributes, a Delay busy loop, KeyPressed/Kbd from INT 16h. A compiling CALC needs them to resolve; a running CALC needs them to be right, which is a later question and explicitly not this goal.

Stage 15 — recursion, forward, nesting. Two distinct shapes, both in Evaluate:

  • Direct self-recursion, Fact: Fact := I * Fact (I - 1) (CALC.INC:747,754). The name must be visible inside its own body.
  • Mutual recursion through forward: Expression is declared function Expression(var ExpressionCh : Char): Real; forward; at CALC.INC:742 and defined at 1036 — with the parameter list commented out, function Expression{(var ExpressionCh : Char):Real;forward}; — while Factor calls it two ways (CALC.INC:820,932). So the definition may repeat only the name and must inherit the forward's parameters, result type and level; keep the var-ness intact (Stage 10).

Today a self-call is ERR 1 at the ( (proved with a probe). Grouping a callee's parameters must agree with Stage 10's 2-vs-4-byte accounting, so land this after parameters — or land the symbol-visibility half first and gate the rest.

Stage 16 — the layout (§3, the layout track). Re-orient the image so the compiler can emit 16,560 B of code and hold 12,032 B of data (both measured, §3.1), choosing between the two candidate fixes in §3 and re-baselining t43_codewindow in the same commit. This is the only stage that blocks the goal rather than the ladder. Then capacity (§5), which follows from it: TextLimit is 488 bytes from overflowing and should rise with the include stage.


5. Capacity — the numbers that will bite

limit value CALC needs verdict
TextLimit (source buffer) 62,903 CALC.PAS + CALC.INC, normalised = 62,415 488 bytes of slack
MaxCode 24,000 16,560 B (measured, §3.1) enough in itself; dataBase's 4,096 B window is the blocker (§3)
code window (dataBase − rtSz) 4,096 ≥ 16,560 must change orientation (§3)
data area unbounded past dc 12,032 B (measured, §3.1) must be able to hold it under/over the code
stack 16,384 min / 655,360 max 16,384 min matches the original's minstk/maxstk
MaxSym 3,000 47 routines + locals + params inferred OK, verify
strOff/strLen [0..255] 57 distinct string literals (67 occurrences) OK
strPool 4,096 chars ≤ 69 bytes each OK
heap hdrHeap = dc to the stack 16,281 for Pages must be checked in §3's budget
data scratch per + 256 bytes/site ~9 sites trim (Stage 12)

The TextLimit number is the sharpest: a 62 KB program in a 62,903-byte buffer is not a design, it is luck. Raising TextLimit (and the buf array in TextBuf.mod) costs only compiler memory and should happen in the same commit as the include, because the include is what consumes the slack.


6. Open questions, each with a method

  • Q1 — RESOLVED (§3.1). The data area sits below the program code; the binding limit is data running into the code base, and this port has it the other way round. Remaining as a decision, not an unknown: implement "match the original" or "dynamic data base", and re-baseline t43_codewindow in the same commit.
  • Q2. Is {$A-} really a no-op for TURBO.COM? Check TPSRC for an A directive before treating it as ignorable.
  • Q3. Pages = array[0..200] of string[80] — confirm the element size is 81 and that New (Buffer) gets 16,281 contiguous bytes from the heap the way TP3's New does.
  • Q4 — RESOLVED (Stage 1). run_compile_tests.sh/CompileTest.mod still have no fixture-with-companion-file notion, and they do not need one: the ladder is a small dedicated script (tests/calc_ladder.sh) that stages CALC.PAS and CALC.INC as copies in tmp/calc_ladder/. That is also what makes the Stage 2 include resolve, since it opens the .inc relative to the including file.
  • Q5. Are range/index checks on by default for TURBO.COM? CALC sets neither {$R} nor {$V}; if the default is on, TU_RANGE_CHECK and TU_INDEX_CHECK are on the critical path for every index in CALC, not an optional hardening. RESUME-TP3.md §4 (compiler options) is the place to check.
  • Q6 (cheap, one run in dosemu). Does memory-mode compilation of CALC report "Compiler Overflow #99" as CALC.PAS's banner warns? The .COM path succeeds (§3.1); the memory path is the only untested claim about which limit bites. Banner step 1 (R) is all it takes.
  • Q7. What the two header words after ds actually mean. They read 730/39 for HELLO and 1760/789 for CALC, matching neither the reported code/data sizes nor each other, so RESUME-TP3.md §3.1's field list is unreliable there. Only matters if we adopt the original's header shape; until then the statistics line and the file layout are authoritative (§3.1).

7. What not to do

  • Do not "support" a construct by parsing it and emitting nothing. That is the var-parameter bug and the {$I} bug, and both were green.
  • Do not raise dataBase/MaxCode and call §3 solved; the orientation is the question, and t43_codewindow exists precisely to catch a silently corrupt image.
  • Do not implement BCD reals, Pos, Concat, Round, Trunc or goto for this goal. CALC uses none of them.
  • Do not let this plan absorb CALC-PORT.md. Same program, different target, different session.

8. Pointers

  • SUMMARY.md — current state, the fixture tables, "Next steps".
  • RESUME-TP3.md — §2 types and layouts, §3.1 skeleton, §3.4 assignment, §3.11 parameters (the SEG:OFF push), §3.14 typed constants, §4 options.
  • TP3-COMPILER.md — what the compiler does today and why.
  • Resources/turbopascal3source/TP3/TPSRC1-10 — the ground truth.
  • Resources/coeur-tp-ocr/coeur-tp-03-d.txt §4.7 — sets explained: the hybrid byte layout this port implements (4.7.2) and the memory order (4.7.3).
  • Resources/COMPIL4/ — Wirth's Pascal-P4 (P.PAS, P41..P48), added as a cross-check. Its sets are a fixed 8-byte bitset over 0..63 with in as a runtime opcode, so it informs the algorithm, not the representation.
  • Resources/TP3/CALC.PAS, CALC.INC, CALC.HLP — the target.
  • shell/tests/probe/README.md — read before touching any emitter.