Bladeren bron

docs: record open-array M4 session and refresh status

Eric Streit 1 dag geleden
bovenliggende
commit
707b7e4014

+ 13 - 10
README.md

@@ -47,24 +47,27 @@ docs/              STATUS.md, plan.md, plan-*.md, analysis_*.md, summary_*.md
 
 ## Status (2026-10-23)
 
-- Suite **283/283**; self-hosting **fixpoint OK** (byte-identical
-  stage2/stage3, 4,198,600 bytes); corpus **512/612 compile-OK,
+- Suite **295/295**; self-hosting **fixpoint OK** (byte-identical
+  stage2/stage3, 3,480,366 bytes); corpus **518/612 compile-OK,
   0 crashes** (`tools/v3-corpus`; the corpus is gm2's testsuite, so it
   mixes conforming Modula-2 with gm2 leniencies — see `docs/gaps/`).
-  `lazyunique` now runs; the remaining `except*` corpus files still hit
-  V3's declaration-order policy.
-- Active branch **`master`**, feature state **`v3-finally`**.
+  The remaining `except*`/`onebyte*`-style corpus files still hit V3's
+  declaration-order policy.
+- Active branch **`master`**, feature state **`v3-open-arrays-m4`**.
   Recent milestones: `PACKEDSET`, ISO exceptions (`EXCEPT`/`RETRY`,
   `EXCEPTIONS`, `M2EXCEPTION`, catchable language traps including
-  `functionException`, module `FINALLY`), and
-  self-referential pointer records/chained fields.
+  `functionException`, module `FINALLY`), self-referential pointer
+  records/chained fields, and multi-dimensional fixed/open arrays
+  (M1–M4): flat payloads + per-level views, open-array calls, and
+  payload `SIZE`/view `ADR`.
 
 Lowers today: program modules; `CONST`/`VAR`/`TYPE` (scalars,
 subranges, enums, arrays, records incl. variants and self-referential
 pointer records, sets incl. `PACKEDSET`, pointers);
-`ARRAY`/`RECORD`/`SET`/`POINTER`, `WITH`, `IN`; whole array/record
-copy; `NEW`/`DISPOSE`; all scalar arithmetic, relations, boolean
-connectives; `IF/CASE/WHILE/REPEAT/LOOP/FOR`, `GOTO`/labels;
+`ARRAY`/`RECORD`/`SET`/`POINTER`, `WITH`, `IN`; fixed and open
+multi-dimensional arrays (view descriptors, row/subarray conversion,
+payload `SIZE`/`ADR`); whole array/record copy; `NEW`/`DISPOSE`; all
+scalar arithmetic, relations, boolean connectives; `IF/CASE/WHILE/REPEAT/LOOP/FOR`, `GOTO`/labels;
 procedures with value/`VAR` params, functions, recursion, `FORWARD`,
 nested procedures (static-link display), procedure values;
 ISO exceptions (`EXCEPT`/`RETRY`, `SYSTEM.THROW`, `EXCEPTIONS`,

+ 8 - 3
docs/INDEX.md

@@ -20,7 +20,7 @@ bisecting *why* a decision was made, not for reading end-to-end).
 - **`features.md`** — the feature list.
 - **`comparison-PIM4-gm2-V3.md`** — PIM4 / GNU Modula-2 / **M2R10** /
   **TopSpeed V3** / V3 feature comparison (2026-10-23; V3 at
-  `v3-packedset`; filename kept for link stability).
+  `v3-open-arrays-m4`; filename kept for link stability).
 - **`analysis_gaps.md`** — language gaps + corpus notes.
 - **`gaps/README.md`** — minimal reproducers, one per unsupported gap,
   and the gm2-extension policy.
@@ -45,8 +45,9 @@ bisecting *why* a decision was made, not for reading end-to-end).
 - **`design-open-arrays.md`** — multi-dimensional open-array ABI:
   per-level view descriptors over a flat backing block, payload
   `SIZE`/`ADR`, fixed/open 2-D and 3-D arrays, row/byte flattening,
-  records/constructors and corpus mapping.  Design; M1 type/ABI
-  helpers are implemented.
+  records/constructors and corpus mapping.  M1–M4 are implemented;
+  M5 (record fields/constructors) and the optional pointer-based 1-D
+  view remain.
 - `blaise-phases-map.md`, `language-report.md`, `grammar.md` — the
   grammar/phases reference.
 - `summary_l3_selfhost_fix.md` — the self-host root cause; also where
@@ -54,6 +55,10 @@ bisecting *why* a decision was made, not for reading end-to-end).
 
 ## Recent session summaries (newest first)
 
+- **`summary_session-2026-10-23-open-arrays.md`** — fixed/open
+  multi-dimensional arrays M1–M4: flat payload + views, open calls,
+  row/subarray conversion, payload `SIZE`/`ADR`; tags
+  `v3-open-arrays-m4` and `v3-open-arrays-m4-summary`; next M5.
 - **`summary_session-2026-10-23-exceptions.md`** — ISO exceptions
   M1–M5: parser/AST, runtime/codegen, `EXCEPTIONS`/`M2EXCEPTION`,
   index/range/CASE traps; tag `v3-exceptions`; corpus caveats.

+ 18 - 14
docs/STATUS.md

@@ -4,18 +4,18 @@ This answers "where are we now?".
 
 ## Numbers
 
-- Suite **283/283** (`compiler/run_tests.sh`).
+- Suite **295/295** (`compiler/run_tests.sh`).
 - Self-hosting **FIXPOINT OK** — stage2 and stage3 images are
-  byte-identical (4,198,600 bytes).
-- Corpus **512 / 612** sources compile with 0 errors, **0 crashes**
+  byte-identical (3,480,366 bytes).
+- Corpus **518 / 612** sources compile with 0 errors, **0 crashes**
   (`tools/v3-corpus/corpus.sh` over the gm2 testsuite; the harness now
   resolves V3's own shipped libraries); the rest are genuine language
   gaps, **gm2 leniencies V3 deliberately rejects**, or unresolved
-  imports of gm2-only modules.  `lazyunique` now compiles and runs
-  (`rc=0`); the corpus stays 512 because it was already compile-OK.
-  The pointer-chain (`p^.next^.value`) layout/codegen gap is fixed
-  (`t_ptrchain`).
-- Branch **`master`**, feature state **`v3-exceptions` + `v3-ptrchain` + `v3-function-exception` + `v3-finally`**; tree clean.
+  imports of gm2-only modules.  Open-array corpus cases `unbounded`,
+  `unbounded2`, `unbounded3`, `unbounded4` and `array9` now compile;
+  `unbounded`/`unbounded2`/`unbounded3`/`unbounded4` also run.
+- Branch **`master`**, feature state **`v3-open-arrays-m4`**; tree
+  clean.
 
 ## Architecture (current)
 
@@ -53,7 +53,11 @@ parameters, constraints, nested generics, generic classes.
 
 Multi-dimensional arrays use the PIM/ISO repeated-bracket form
 (`ARRAY [lo..hi], [lo..hi] OF T`) as well as V3's earlier one-bracket
-comma list.  Subranges keep their ordinal base: `['A'..'Z']` is a
+comma list.  Rank-2/3 fixed/open arrays use one flat payload plus
+per-level 40-byte views (`count`, `elemBytes`, `stride`, `base`,
+`next`); nested open formals, row/subarray conversion, forward
+open-to-open calls, payload `SIZE`/`TSIZE` and view `ADR` are
+implemented.  Subranges keep their ordinal base: `['A'..'Z']` is a
 character type and `[red..blue]` belongs to its enumeration, so
 base-typed values assign and compare; `FOR` accepts any ordinal
 control variable (`FOR c := red TO blue`).  The ISO empty string
@@ -174,11 +178,11 @@ declarations still work (`t_fwdproc`, `t_fwdok`).
    `INC`/`DEC`, leading CASE `|`, `__FUNCTION__`).  Corpus is **511/612**
    at the stop point.  The remaining `LANG` entries are mostly:
    **declaration order** cases V3 rejects by policy (`docs/gaps/`); ISO
-   exceptions (M4 + `functionException` + module `FINALLY`; coroutine/
-   protection follow-ups — see `docs/design-exceptions.md`);
-   multi-dimensional
-   open arrays (representation, see `docs/handoff-2026-10-23-next.md`);
-   and gm2-only extensions.  See `docs/analysis_gaps.md`.
+   exceptions (coroutine/protection follow-ups — see
+   `docs/design-exceptions.md`); gm2-only extensions; and the M5
+   array tail (record fields/constructors plus the optional
+   pointer-based 1-D view migration).  See
+   `docs/handoff-2026-10-23-next.md` and `docs/analysis_gaps.md`.
 2. ~~**The size-sensitive V3 miscompile**~~ — **root-caused and fixed**
    (2026-10-19; see `docs/summary_l3_selfhost_fix.md`).  It was never
    size-sensitive: the fixpoint gate matched the literal string

+ 20 - 15
docs/comparison-PIM4-gm2-V3.md

@@ -1,7 +1,7 @@
 # PIM4 vs GNU Modula-2 vs M2R10 vs TopSpeed V3 vs V3 — feature comparison
 
-**Date:** 2026-10-23 · **V3 state:** `v3-finally` (suite
-283/283, self-hosting fixpoint OK 4,198,600 bytes, corpus 512/612,
+**Date:** 2026-10-23 · **V3 state:** `v3-open-arrays-m4` (suite
+295/295, self-hosting fixpoint OK 3,480,366 bytes, corpus 518/612,
 0 crashes).
 
 **Scope and sources.**
@@ -75,7 +75,7 @@ change individual cells.
 | `PACKEDSET` | ❌ (ISO) | ✅ | ❌ | ❌ | ✅ (1/2/4 bytes, span ≤ 32) |
 | Fixed / multi-dimensional arrays | ✅ `[a], [b] OF` | ✅ | ✅ `ARRAY count, count OF` (zero-based) | ✅ | ✅ both `[a], [b]` and `[a, b]` |
 | Open arrays (1-D) | ✅ | ✅ | ✅ (`BARE ARRAY OF`) | ✅ | ✅ |
-| Multi-dimensional open arrays | ❌ (ISO addition) | ✅ | 🔄 one-dimensional `BARE` arrays | ❌ | ❌ (descriptor-per-level representation) |
+| Multi-dimensional open arrays | ❌ (ISO addition) | ✅ | 🔄 one-dimensional `BARE` arrays | ❌ | ✅ (flat payload + per-level views; legacy 1-D ABI adapter) |
 | Records | ✅ | ✅ | ✅ extensible | ✅ | ✅ (incl. self-referential pointer fields) |
 | Variant records | ✅ | ✅ | ❌ removed (extensible records replace them) | ✅ | ✅ (plus empty arms/`ELSE`) |
 | Pointers, `NIL`, opaque types | ✅ | ✅ | ✅ (`OPAQUE` keyword) | ✅ | ✅ (incl. chained `p^.next^.value`) |
@@ -99,7 +99,7 @@ change individual cells.
 | Empty string `""`/`''` as `CHAR` | ❌/unclear (ISO rule) | ✅ | 🔄 (char literals compatible with `ARRAY OF CHAR`; empty-to-`CHAR` not explicit) | ✅ | ✅ (ISO rule) |
 | `LENGTH` | ❌ (ISO adds it) | ✅ | 🔄 library/pervasive, not in the base grammar | ❌ | ✅ |
 | `HIGH` | ✅ | ✅ | ❌ (not in the R10 grammar) | ✅ | ✅ (including string `CONST`) |
-| `TSIZE`/`SIZE`/`ADR` | ✅ | ✅ | 🔄 SYSTEM/library forms | ✅ | ✅ |
+| `TSIZE`/`SIZE`/`ADR` | ✅ | ✅ | 🔄 SYSTEM/library forms | ✅ | ✅ array payload `SIZE`; multi/open `ADR` = payload base; fixed 1-D `ADR` keeps descriptor compatibility |
 | Runtime subrange range checks | ⚠️ error per language; checks implementation-defined | ✅ | ✅ (R10 safety) | 🔄 | ✅ assignment/value actual/`RETURN`/`INC`/`DEC`; catchable `rangeException`; `FOR` final increment exempt |
 | `SHIFT`/`ROTATE` | ⚠️ implementation SYSTEM extension | ✅ | ❌ | ✅ (`<<`/`>>` operators) | ✅ |
 | `TBITSIZE` | ❌ | ✅ | ❌ | ❌ | ❌ |
@@ -154,12 +154,14 @@ change individual cells.
   extensions, set `CASE` labels, libc prototypes).  ISO exceptions are
   **mostly complete** (`EXCEPT`/`RETRY`/`SYSTEM.THROW`, `EXCEPTIONS`/
   `M2EXCEPTION`, index/range/CASE/function traps, and module
-  `FINALLY`).  Its hard gaps are **complex arithmetic,
-  finalization/protection, multi-dimensional open arrays, ISO OO** and
-  parts of the **word model** (`SHORTREAL`, `REAL` width), plus a
-  **65536-bit `SET` span limit** and a **32-bit `PACKEDSET` limit**.
-  Self-referential pointer records and chained pointer-field
-  designators (`p^.next^.value`) are supported.
+  `FINALLY`).  Fixed and open multi-dimensional arrays use a flat
+  payload with per-level views; payload `SIZE`/`ADR` and row/subarray
+  conversion are implemented.  Its hard gaps are **complex arithmetic,
+  finalization/protection, ISO OO** and parts of the **word model**
+  (`SHORTREAL`, `REAL` width), plus a **65536-bit `SET` span limit**
+  and a **32-bit `PACKEDSET` limit**.  Self-referential pointer records
+  and chained pointer-field designators (`p^.next^.value`) are
+  supported.
 
 ## V3-specific deviations and limits
 
@@ -183,9 +185,11 @@ change individual cells.
 - **Module `FINALLY`** runs on normal `main` return in reverse
   initialization order.  `HALT`, unhandled aborts and the ISO
   `TERMINATION` module are not covered.
-- **Multi-dimensional open arrays** are not supported: V3 represents an
-  array level as its own descriptor, whereas gm2 passes a flat block —
-  accepting the corpus files would compile but miscompile.
+- **Multi-dimensional arrays** use a flat payload and per-level
+  40-byte views.  A legacy 1-D `{count,data}` descriptor is still used
+  at ABI boundaries; row/`ARRAY OF BYTE` conversions are copy-based
+  there, so row `ADR` identity through an internal 1-D call remains a
+  follow-up.
 - **Single-invocation compilation:** definition, implementation and
   program units are compiled together into one image; there is no
   incremental object/archive format.
@@ -195,8 +199,9 @@ change individual cells.
 ## Sources
 
 - V3: `docs/STATUS.md`, `docs/features.md`, `docs/gaps/README.md`,
-  `docs/analysis_gaps.md`, `docs/summary_session-2026-10-23-exceptions.md`; the
-  gates in this document were run at `v3-finally`.
+  `docs/analysis_gaps.md`,
+  `docs/summary_session-2026-10-23-open-arrays.md`; the gates in this
+  document were run at `v3-open-arrays-m4`.
 - gm2: the GCC Modula-2 testsuite and `gm2-libs`/`gm2-libs-iso`
   definition modules in the sibling `gcc-git` tree.
 - M2R10: the published R10 grammar (m2r10-grammar.gll) and

+ 9 - 4
docs/features.md

@@ -1,4 +1,4 @@
-# m2compiler-V3 — feature status (2026-10-22; suite 255/255 green)
+# m2compiler-V3 — feature status (2026-10-23; suite 295/295 green)
 
 Pipeline: Coco/R `M2.atg` (1748 lines, 73 productions) → `gm2`-built
 `M2` → QBE `.ssa` → `qbe` → `cc` → run. `SymTab.mod` 1623 lines,
@@ -26,8 +26,10 @@ Legend: ✅ done · 🔄 partial · ⏸ not started / deferred.
   actuals, `RETURN` and `INC`/`DEC` (the `FOR` final increment is
   exempt).
 - ✅ `ARRAY [lo..hi,…]` (multi-dim, one bracket) and the PIM/ISO
-  repeated-group form `ARRAY [lo..hi], [lo..hi] OF T`, plus open
-  `ARRAY OF` formals.
+  repeated-group form `ARRAY [lo..hi], [lo..hi] OF T`; fixed and open
+  multi-dimensional formals (`ARRAY OF ARRAY OF …`) use flat payloads
+  + per-level views, with row/subarray conversion and open-to-open
+  forwarding.
 - ✅ ISO empty string literal: `""`/`''` is compatible with `CHAR`
   (value 0) in assignment, comparison, `CASE` labels, `FOR` bounds,
   actuals and `RETURN`; in an `ARRAY OF CHAR` context it stays the
@@ -140,7 +142,10 @@ Legend: ✅ done · 🔄 partial · ⏸ not started / deferred.
   (`ORD(a > b)` in a `CONST`). UString `+`/`LEN` likewise.
 - ✅ `__FILE__`, `__LINE__`, `__FUNCTION__` predefined literals.
 - ✅ `NIL` comparisons/assignment.
-- ✅ `VAL`/`ADR`/`TSIZE`/`SIZE`/`HIGH`/`LEN` work. ✅ Result suffixes
+- ✅ `VAL`/`ADR`/`TSIZE`/`SIZE`/`HIGH`/`LEN` work.  Array
+  `SIZE`/`TSIZE` use payload bytes (no descriptor header/term slot);
+  `ADR` of a multi/open view or row is its payload base, while fixed
+  1-D `ADR` keeps descriptor compatibility.  ✅ Result suffixes
   (`F()^`, `F()[i]`, `F().field`): components after a call result,
   loaded like Design components.
 

+ 3 - 0
docs/summary_session-2026-10-23-exceptions.md

@@ -1,5 +1,8 @@
 # Session 2026-10-23 — ISO exceptions (M1–M5)
 
+> Continued later the same day with multi-dimensional open arrays:
+> see `summary_session-2026-10-23-open-arrays.md`.
+
 Follows the same-day range-check/PACKEDSET session
 (`summary_session-2026-10-23.md`) and the earlier design commit
 `2a30d8c`.

+ 127 - 0
docs/summary_session-2026-10-23-open-arrays.md

@@ -0,0 +1,127 @@
+# Session 2026-10-23 — multi-dimensional open arrays (M1–M4)
+
+Same-day continuation of the range-check/PACKEDSET and ISO-exception
+sessions.  This session implemented the open-array ABI in four steps:
+fixed storage, open views/calls, and payload `SIZE`/`ADR`.
+
+**Branch:** `master`.  **Feature state:** open-array M4
+(`v3-open-arrays-m4`), on top of `v3-finally`.
+**Gates at the end:** suite **295/295**, fixpoint **OK
+(3,480,366 bytes)**, corpus **518/612 compile-OK, 0 crashes**.
+
+---
+
+## Numbers
+
+| gate | start (M1) | end (M4) |
+| --- | --- | --- |
+| suite (`compiler/run_tests.sh`) | 283/283 | **295/295** |
+| fixpoint (`bootstrap/fixpoint.sh`) | 4,198,600 B | **3,480,366 B** |
+| corpus (`tools/v3-corpus/corpus.sh`) | 512/612 | **518/612, 0 crashes** |
+
+Corpus additions: `unbounded`, `unbounded2`, `unbounded3`,
+`unbounded4` and `array9` compile; the first four also run.
+`unbounded6`/`unbounded8`/`adrunbounded3` remain declaration-order
+cases under V3 policy (their variables are declared after the
+procedures).  The corpus is compile-only; it is not a run gate.
+
+---
+
+## What landed
+
+### M1 — type/ABI helpers (`2cc6776`)
+
+- `SymTab.ArrayRank`, `SymTab.IsMultiArray`.
+- `SymTab.ArrayLeafBytes`, `SymTab.ArrayStride`, and
+  `SymTab.ArrayPayloadSize`.
+- `MaxDefer` 256 → 512 and the session SSA buffer 4 → 8 MiB.
+- Design document: `docs/design-open-arrays.md`.
+
+### M2 — fixed multi-dimensional storage (`4616e51`)
+
+- Rank-2/3 fixed arrays use one contiguous payload and a chain of
+  40-byte views:
+  `{count, elemBytes, stride, base, next}`.
+- Static multi arrays emit a view chain, per-level templates and a flat
+  zero payload.  Locals allocate views/templates + payload and
+  initialize them in the frame.
+- `ViewIndex` returns a copied next-level view for array selectors and
+  a scalar address at the leaf.
+- Pointer-to-multi indexing and `NEW` initialization work.
+- Whole-array copy uses the flat payload; old nested static
+  constructors are converted by `CopyOldNestedToView`.
+- Rank-1 row views are adapted to the legacy 1-D `{count,data}`
+  descriptor (copy-in/copy-back for `VAR`) as an interim ABI bridge.
+- Regression: `t_multidim_local`; existing `t_multidim`, `t_array`,
+  `t_ctor3`, `t_nestidx`, `t_nestarr`, `t_ptrnest` and `showcase6` stay
+  green.
+- Tag **`v3-open-arrays-m2`**.
+
+### M3 — open view descriptors (`557addb`)
+
+- Nested open formals (`ARRAY OF ARRAY OF …`) parse.
+- Type compatibility accepts same-rank fixed/open arrays with the same
+  leaf type, and `ARRAY OF BYTE`/`SBYTE` accepts any array rank for
+  flattening.
+- Fixed/open multi actuals and rank-2/3 subviews pass as `ArrayView`
+  pointers; open-to-open forwarding and `VAR` writes go straight to
+  `base`.
+- Open formals index through `count/stride/base/next`;
+  `HIGH`/`LEN` read the view count.
+- Added: `t_open2d`, `t_open2d_value`, `t_open2d_high`,
+  `t_open2d_nested`, `t_open3d`, `t_open3d_row`,
+  `t_open_row_to_1d`, `t_open_flatten_byte`.
+- Tag **`v3-open-arrays-m3`**.
+
+### M4 — array builtins (`df72d03`)
+
+- `SIZE`/`TSIZE` now use payload bytes for arrays:
+  - fixed 1-D `count * leaf`;
+  - open 1-D `count * element`;
+  - multi views runtime `count * elemBytes`;
+  - descriptor headers and the hidden 1-D CHAR terminator are
+    excluded.
+- `ADR`:
+  - multi view/row → `view.base`;
+  - 1-D open descriptor → `descriptor + 8`;
+  - fixed 1-D → existing descriptor address (pointer-to-array
+    compatibility);
+  - scalar/selected element → its address.
+- External 1-D compatibility descriptors include the CHAR/UCHAR
+  terminator; the temporary descriptor is still freed after `VAR`
+  copy-back.
+- Added: `t_open2d_size`, `t_open2d_adr`, `t_open1d_size_adr`.
+  `unbounded3` runs with `SIZE` 60/12/2/2.
+- Tag **`v3-open-arrays-m4`**.
+
+---
+
+## Remaining
+
+- M5: multi-dimensional **record fields**, arrays of records with
+  multi fields, and constructors using the flat/view layout.
+- Optional follow-up: migrate the legacy 1-D descriptor itself to a
+  pointer-based view.  Row → 1-D still uses a copy-in/copy-back bridge,
+  so row `ADR` identity through an internal 1-D call is not preserved.
+  Writes are preserved.
+- `unbounded6`/`unbounded8`/`adrunbounded3` still fail declaration
+  order by policy; do not relax it for corpus count.
+- Comparison/README/STATUS/features/INDEX were refreshed for M4.
+
+---
+
+## Commits and tags
+
+| commit | tag | work |
+| --- | --- | --- |
+| `2cc6776` | — | M1 type/ABI helpers |
+| `0f422ae` | — | open-array design document |
+| `4616e51` | `v3-open-arrays-m2` | fixed rank-2/3 flat storage + views |
+| `7c02646` | — | M2 docs/tag map |
+| `557addb` | `v3-open-arrays-m3` | open multi-array views + call conversion |
+| `fb90c8f` | — | M3 docs/tag map |
+| `df72d03` | `v3-open-arrays-m4` | payload `SIZE`/`TSIZE`, view `ADR` |
+| this docs commit | `v3-open-arrays-m4-summary` | session summary |
+| final docs commit | `v3-session-2026-10-23-open-arrays-end` | session tag |
+
+No push was performed.

+ 3 - 2
docs/summary_session-2026-10-23.md

@@ -1,7 +1,8 @@
 # Session 2026-10-23 — range checks + PACKEDSET
 
-> Continued later the same day with ISO exceptions: see
-> `summary_session-2026-10-23-exceptions.md`.
+> Continued later the same day with ISO exceptions
+> (`summary_session-2026-10-23-exceptions.md`) and then multi-dimensional
+> open arrays (`summary_session-2026-10-23-open-arrays.md`).
 
 Branch **`master`**.  Continues `summary_session-2026-10-22.md` from the
 2026-10-22 stop point (`v3-smallfix`).