|
|
@@ -1,92 +1,95 @@
|
|
|
-# Step: V3 compiler corpus run (gm2 testsuite)
|
|
|
+# Step: V3 compiler corpus runs
|
|
|
|
|
|
-Tag `v3-corpus-run`. V3 suite **151/151**, fixpoint OK (unchanged —
|
|
|
-this only adds a harness and a doc).
|
|
|
+Tag `v3-corpus-run`. V3 suite **152/152**, fixpoint OK.
|
|
|
|
|
|
## Why
|
|
|
|
|
|
The TopSpeed corpus validated the *sidecar grammar*, not V3 the
|
|
|
compiler (the sidecar is a separate `TSM2` binary built from
|
|
|
`TopSpeed-V3-M2.atg`; it reuses V3's `SymTab`/`QbeGen`/`FileIO` sources
|
|
|
-but never runs `compiler/M2`). So this step runs **V3 itself** over a
|
|
|
-real standard-Modula-2 corpus: the gm2 testsuite
|
|
|
-`testsuite/gm2/pim/pass` and `testsuite/gm2/iso/run/pass`.
|
|
|
+but never runs `compiler/M2`). So this work runs **V3 itself** over two
|
|
|
+real Modula-2 corpora.
|
|
|
|
|
|
-Harness: `tools/v3-corpus/corpus.sh` (copies each file to a scratch
|
|
|
-tree, compiles alone, splits failures into CRASH / LIB / LANG — see its
|
|
|
-README).
|
|
|
+Harness: `tools/v3-corpus/corpus.sh` — compiles each file with
|
|
|
+`compiler/M2` alone (pairing an IMPLEMENTATION with its sibling
|
|
|
+DEFINITION), in a scratch tree, and splits failures into CRASH (M2
|
|
|
+died) / LIB (imports a module outside V3's known stdlib) / LANG (real
|
|
|
+gap).
|
|
|
|
|
|
-## Baseline (after the crash fix)
|
|
|
+## Corpora and results
|
|
|
+
|
|
|
+**1. gm2 testsuite** — `testsuite/gm2/pim/pass` + `iso/run/pass`
|
|
|
+(612 files):
|
|
|
|
|
|
| | files |
|
|
|
| --- | --- |
|
|
|
-| total | **612** |
|
|
|
-| **compile-OK** (0 errors) | **219** |
|
|
|
-| CRASH (M2 died, rc ≥ 128) | **0** |
|
|
|
-| LIB (imports a gm2 library V3 lacks) | **187** |
|
|
|
-| LANG (real V3 gap) | **206** |
|
|
|
-
|
|
|
-(Before the fix: 219 OK / **7 CRASH** / 182 LIB / 204 LANG.) So ~187
|
|
|
-of the 393 failures are simply **missing gm2 libraries** (`StrIO`,
|
|
|
-`NumberIO`, `STextIO`, `SWholeIO`, `FpuIO`, `WholeStr`, `StdIO`,
|
|
|
-`M2RTS`, `DynamicStrings`, `Processes`, …) — expected, not language
|
|
|
-gaps. The signal is the **206 LANG** files.
|
|
|
-
|
|
|
-## Fix: long-line buffer overflow (the 7 "set crashes")
|
|
|
-
|
|
|
-The 7 crashes (`largeset`, `program2`, `set9`, `sets3`,
|
|
|
-`smallset4/5/6`) were **not** set bugs: those files contain a ~860-char
|
|
|
-line (a 76-literal enumeration), and
|
|
|
-
|
|
|
-- `PrintListing`'s line buffer was `ARRAY [0 .. 255] OF CHAR`, and
|
|
|
-- `GetLine` wrote **without bounding** to `HIGH(line)`.
|
|
|
-
|
|
|
-A source line longer than 255 chars therefore **overflowed the stack**
|
|
|
-(corrupting locals and the return address) — hence the garbage line
|
|
|
-number in the `.LST` and the segfault.
|
|
|
-
|
|
|
-Fixed in `compiler/src/compiler.frm`:
|
|
|
-- `GetLine` now truncates at `HIGH(line)` (still consuming the rest of
|
|
|
- the line so positions stay correct);
|
|
|
-- `PrintListing`'s buffer is now `ARRAY [0 .. 4095] OF CHAR`;
|
|
|
-- `PrintErr`'s caret loop is bounded by `HIGH(line)`.
|
|
|
-
|
|
|
-Regression test: `compiler/tests/t_longline.mod` (406-char line, exit
|
|
|
-0). Suite **152/152**; fixpoint OK (2,339,115 bytes).
|
|
|
-
|
|
|
-## Top remaining LANG gaps (from the corpus)
|
|
|
-
|
|
|
-- **48 `not supported yet`** — deliberate V3 `230`s; the corpus shows
|
|
|
- which constructs matter, e.g. **array constructors**
|
|
|
- (`array {'h','e','l','l','o'}` in `arrayconst1`).
|
|
|
-- **`undeclared identifier`** — mostly **`SYSTEM` facilities** V3 does
|
|
|
- not expose (`FROM SYSTEM IMPORT BYTE;` → `BYTE`), plus small library
|
|
|
- holes.
|
|
|
-- **Anchored subranges** `INTEGER [-1 .. 24]` (`array2`, `array3`) —
|
|
|
- PIM form V3's `Subrange` does not parse (`';' expected` at `[`).
|
|
|
-- **Unnamed parameters in procedure types**
|
|
|
- (`PROCEDURE (VAR ARRAY OF REAL)` in `another.mod`) — PIM allows
|
|
|
- them; V3 requires a name (`ident expected`).
|
|
|
-- Assorted: `incompatible assignment`/`comparison` on char arrays,
|
|
|
- `invalid ArrayType`, `invalid call`, `FOR … BY`, etc.
|
|
|
+| **compile-OK** | **242** |
|
|
|
+| CRASH | 0 |
|
|
|
+| LIB (missing gm2 library) | 185 |
|
|
|
+| LANG (real V3 gap) | 185 |
|
|
|
+
|
|
|
+**2. gm2 standard libraries** — `$HOME/bin/Modula2/Gm2/lib/gcc/…/m2/`
|
|
|
+(`m2pim`/`m2iso`/`m2log`/`m2cor`/`m2min`, 336 files, `.def`+`.mod`):
|
|
|
+
|
|
|
+| | files |
|
|
|
+| --- | --- |
|
|
|
+| **compile-OK** | **78** |
|
|
|
+| CRASH | 0 |
|
|
|
+| LIB | 176 |
|
|
|
+| LANG | 82 |
|
|
|
+
|
|
|
+(LIB is high in both because each file is compiled alone — cross-module
|
|
|
+imports are not resolved; the real signal is LANG.)
|
|
|
+
|
|
|
+## Fixes landed
|
|
|
+
|
|
|
+### A. Long-line buffer overflow (the 7 "set" crashes) — `v3-fix-longline-crash`
|
|
|
+
|
|
|
+`PrintListing`'s line buffer was `ARRAY [0 .. 255]` and `GetLine` wrote
|
|
|
+without bounding to `HIGH(line)`, so a source line > 255 chars (an
|
|
|
+860-char enum line in the gm2 set tests) overflowed the stack. Fixed
|
|
|
+in `compiler/src/compiler.frm` (truncate in `GetLine`, buffer →
|
|
|
+4096, bounded caret loop); regression `t_longline.mod`; CRASH → 0.
|
|
|
+
|
|
|
+### B. `EXPORT QUALIFIED` in DEFINITION modules — this step
|
|
|
+
|
|
|
+gm2/PIM definition modules carry an export list
|
|
|
+(`EXPORT QUALIFIED ReadString, WriteString, …;`). V3's `DefUnit` did
|
|
|
+not accept it → `'END' expected` on the first such file. Added the
|
|
|
+clause (parsed; the names are already exported by the module scope).
|
|
|
+This raised **both** corpora: testsuite compile-OK **219 → 242**, gm2
|
|
|
+libs **42 → 78**.
|
|
|
+
|
|
|
+## Remaining LANG gaps
|
|
|
+
|
|
|
+- Tests suite: **185** files, mostly gm2-specific libraries' call sites
|
|
|
+ and deliberate V3 `230`s (array constructors, …).
|
|
|
+- gm2 libs: **82** files, dominated by **gm2-specific syntax** —
|
|
|
+ `PROCEDURE __BUILTIN__ name(...)` (`Builtins`/`libc`/`libm`/…),
|
|
|
+ C-name pragmas, ISO module attributes — which are *not* V3's dialect.
|
|
|
+- Genuine V3 gaps still seen: **`SYSTEM` facilities** (`FROM SYSTEM
|
|
|
+ IMPORT BYTE`), **anchored subranges** `T[lo..hi]`, **unnamed
|
|
|
+ procedure-type parameters**.
|
|
|
|
|
|
## Reproduce
|
|
|
|
|
|
```sh
|
|
|
cd tools/v3-corpus
|
|
|
-./corpus.sh # -> 219 compile-OK / 0 CRASH / 187 LIB / 206 LANG
|
|
|
+./corpus.sh # gm2 testsuite -> 242 OK / 185 LANG
|
|
|
+G=$HOME/bin/Modula2/Gm2/lib/gcc/x86_64-pc-linux-gnu/16.0.1/m2
|
|
|
+./corpus.sh $G/m2pim $G/m2iso $G/m2log $G/m2cor $G/m2min # 78 OK
|
|
|
```
|
|
|
|
|
|
## Suggested next steps
|
|
|
|
|
|
-1. **`SYSTEM` facilities**: expose at least `BYTE` / `WORD` /
|
|
|
- `SHORTADDR` etc. so `FROM SYSTEM IMPORT …` works.
|
|
|
+1. **`SYSTEM` facilities**: expose `BYTE`/`WORD`/`SHORTADDR` etc.
|
|
|
2. **Anchored subranges** `T[lo..hi]` (small grammar addition).
|
|
|
3. **Unnamed proc-type parameters** (small grammar addition).
|
|
|
-4. **Array constructors** `array{…}` (bigger: codegen).
|
|
|
+4. Resolve cross-module imports in the harness (pass each file's
|
|
|
+ imported `.def`s) to remove the LIB noise and isolate LANG.
|
|
|
|
|
|
## Files
|
|
|
|
|
|
-`tools/v3-corpus/{corpus.sh,README.md}`,
|
|
|
-`compiler/src/compiler.frm` (long-line fix),
|
|
|
+`tools/v3-corpus/{corpus.sh,README.md}`, `compiler/src/compiler.frm`
|
|
|
+(long-line fix), `compiler/src/M2.atg` (`EXPORT QUALIFIED`),
|
|
|
`compiler/tests/t_longline.mod`, `compiler/run_tests.sh`, this doc.
|