Преглед на файлове

corpus: EXPORT QUALIFIED in DEFINITION modules; pair .mod with its .def

- M2.atg DefUnit: accept the gm2/PIM definition-module export list
  EXPORT [QUALIFIED] name, ...; (parsed; already exported by the module
  scope).  This was the top 'END' expected gap on the gm2 libraries.
- tools/v3-corpus/corpus.sh: compile an IMPLEMENTATION together with
  its sibling DEFINITION when present; README notes the gm2 stdlib
  corpus location.
- Results: gm2 testsuite compile-OK 219 -> 242; gm2 standard libraries
  (m2pim/m2iso/m2log/m2cor/m2min, 336 files) 42 -> 78.
- Suite 152/152; fixpoint OK (2,339,870 bytes).
Eric Streit преди 1 седмица
родител
ревизия
2ee8d4afa9
променени са 5 файла, в които са добавени 2635 реда и са изтрити 2604 реда
  1. 5 0
      compiler/src/M2.atg
  2. 2543 2538
      compiler/src/M2.lst
  3. 68 65
      docs/summary_v3-corpus.md
  4. 9 0
      tools/v3-corpus/README.md
  5. 10 1
      tools/v3-corpus/corpus.sh

+ 5 - 0
compiler/src/M2.atg

@@ -106,6 +106,11 @@ PRODUCTIONS
                                            QbeGen.SetModule(m1); .)
       ";"
       { Import }
+      [ "EXPORT" [ "QUALIFIED" ]        (. (* definition-module export
+                                              list: parsed, and the
+                                              names are already exported
+                                              by the module scope *) .)
+        GetIdent<pn> { "," GetIdent<pn> } ";" ]
       { ConstBlock | TypeBlock<TRUE> | VarBlock
       | ProcHeading<pn, SymTab.InvalidType> ";"
                                         (. SymTab.CloseProc;

Файловите разлики са ограничени, защото са твърде много
+ 2543 - 2538
compiler/src/M2.lst


+ 68 - 65
docs/summary_v3-corpus.md

@@ -1,92 +1,95 @@
-# Step: V3 compiler corpus run (gm2 testsuite)
+# Step: V3 compiler corpus runs
 
-Tag `v3-corpus-run`. V3 suite **151/151**, fixpoint OK (unchanged —
-this only adds a harness and a doc).
+Tag `v3-corpus-run`. V3 suite **152/152**, fixpoint OK.
 
 ## Why
 
 The TopSpeed corpus validated the *sidecar grammar*, not V3 the
 compiler (the sidecar is a separate `TSM2` binary built from
 `TopSpeed-V3-M2.atg`; it reuses V3's `SymTab`/`QbeGen`/`FileIO` sources
-but never runs `compiler/M2`).  So this step runs **V3 itself** over a
-real standard-Modula-2 corpus: the gm2 testsuite
-`testsuite/gm2/pim/pass` and `testsuite/gm2/iso/run/pass`.
+but never runs `compiler/M2`).  So this work runs **V3 itself** over two
+real Modula-2 corpora.
 
-Harness: `tools/v3-corpus/corpus.sh` (copies each file to a scratch
-tree, compiles alone, splits failures into CRASH / LIB / LANG — see its
-README).
+Harness: `tools/v3-corpus/corpus.sh` — compiles each file with
+`compiler/M2` alone (pairing an IMPLEMENTATION with its sibling
+DEFINITION), in a scratch tree, and splits failures into CRASH (M2
+died) / LIB (imports a module outside V3's known stdlib) / LANG (real
+gap).
 
-## Baseline (after the crash fix)
+## Corpora and results
+
+**1. gm2 testsuite** — `testsuite/gm2/pim/pass` + `iso/run/pass`
+(612 files):
 
 | | files |
 | --- | --- |
-| total | **612** |
-| **compile-OK** (0 errors) | **219** |
-| CRASH (M2 died, rc ≥ 128) | **0** |
-| LIB (imports a gm2 library V3 lacks) | **187** |
-| LANG (real V3 gap) | **206** |
-
-(Before the fix: 219 OK / **7 CRASH** / 182 LIB / 204 LANG.)  So ~187
-of the 393 failures are simply **missing gm2 libraries** (`StrIO`,
-`NumberIO`, `STextIO`, `SWholeIO`, `FpuIO`, `WholeStr`, `StdIO`,
-`M2RTS`, `DynamicStrings`, `Processes`, …) — expected, not language
-gaps.  The signal is the **206 LANG** files.
-
-## Fix: long-line buffer overflow (the 7 "set crashes")
-
-The 7 crashes (`largeset`, `program2`, `set9`, `sets3`,
-`smallset4/5/6`) were **not** set bugs: those files contain a ~860-char
-line (a 76-literal enumeration), and
-
-- `PrintListing`'s line buffer was `ARRAY [0 .. 255] OF CHAR`, and
-- `GetLine` wrote **without bounding** to `HIGH(line)`.
-
-A source line longer than 255 chars therefore **overflowed the stack**
-(corrupting locals and the return address) — hence the garbage line
-number in the `.LST` and the segfault.
-
-Fixed in `compiler/src/compiler.frm`:
-- `GetLine` now truncates at `HIGH(line)` (still consuming the rest of
-  the line so positions stay correct);
-- `PrintListing`'s buffer is now `ARRAY [0 .. 4095] OF CHAR`;
-- `PrintErr`'s caret loop is bounded by `HIGH(line)`.
-
-Regression test: `compiler/tests/t_longline.mod` (406-char line, exit
-0).  Suite **152/152**; fixpoint OK (2,339,115 bytes).
-
-## Top remaining LANG gaps (from the corpus)
-
-- **48 `not supported yet`** — deliberate V3 `230`s; the corpus shows
-  which constructs matter, e.g. **array constructors**
-  (`array {'h','e','l','l','o'}` in `arrayconst1`).
-- **`undeclared identifier`** — mostly **`SYSTEM` facilities** V3 does
-  not expose (`FROM SYSTEM IMPORT BYTE;` → `BYTE`), plus small library
-  holes.
-- **Anchored subranges** `INTEGER [-1 .. 24]` (`array2`, `array3`) —
-  PIM form V3's `Subrange` does not parse (`';' expected` at `[`).
-- **Unnamed parameters in procedure types**
-  (`PROCEDURE (VAR ARRAY OF REAL)` in `another.mod`) — PIM allows
-  them; V3 requires a name (`ident expected`).
-- Assorted: `incompatible assignment`/`comparison` on char arrays,
-  `invalid ArrayType`, `invalid call`, `FOR … BY`, etc.
+| **compile-OK** | **242** |
+| CRASH | 0 |
+| LIB (missing gm2 library) | 185 |
+| LANG (real V3 gap) | 185 |
+
+**2. gm2 standard libraries** — `$HOME/bin/Modula2/Gm2/lib/gcc/…/m2/`
+(`m2pim`/`m2iso`/`m2log`/`m2cor`/`m2min`, 336 files, `.def`+`.mod`):
+
+| | files |
+| --- | --- |
+| **compile-OK** | **78** |
+| CRASH | 0 |
+| LIB | 176 |
+| LANG | 82 |
+
+(LIB is high in both because each file is compiled alone — cross-module
+imports are not resolved; the real signal is LANG.)
+
+## Fixes landed
+
+### A. Long-line buffer overflow (the 7 "set" crashes) — `v3-fix-longline-crash`
+
+`PrintListing`'s line buffer was `ARRAY [0 .. 255]` and `GetLine` wrote
+without bounding to `HIGH(line)`, so a source line > 255 chars (an
+860-char enum line in the gm2 set tests) overflowed the stack.  Fixed
+in `compiler/src/compiler.frm` (truncate in `GetLine`, buffer →
+4096, bounded caret loop); regression `t_longline.mod`; CRASH → 0.
+
+### B. `EXPORT QUALIFIED` in DEFINITION modules — this step
+
+gm2/PIM definition modules carry an export list
+(`EXPORT QUALIFIED ReadString, WriteString, …;`).  V3's `DefUnit` did
+not accept it → `'END' expected` on the first such file.  Added the
+clause (parsed; the names are already exported by the module scope).
+This raised **both** corpora: testsuite compile-OK **219 → 242**, gm2
+libs **42 → 78**.
+
+## Remaining LANG gaps
+
+- Tests suite: **185** files, mostly gm2-specific libraries' call sites
+  and deliberate V3 `230`s (array constructors, …).
+- gm2 libs: **82** files, dominated by **gm2-specific syntax** —
+  `PROCEDURE __BUILTIN__ name(...)` (`Builtins`/`libc`/`libm`/…),
+  C-name pragmas, ISO module attributes — which are *not* V3's dialect.
+- Genuine V3 gaps still seen: **`SYSTEM` facilities** (`FROM SYSTEM
+  IMPORT BYTE`), **anchored subranges** `T[lo..hi]`, **unnamed
+  procedure-type parameters**.
 
 ## Reproduce
 
 ```sh
 cd tools/v3-corpus
-./corpus.sh            # -> 219 compile-OK / 0 CRASH / 187 LIB / 206 LANG
+./corpus.sh                       # gm2 testsuite -> 242 OK / 185 LANG
+G=$HOME/bin/Modula2/Gm2/lib/gcc/x86_64-pc-linux-gnu/16.0.1/m2
+./corpus.sh $G/m2pim $G/m2iso $G/m2log $G/m2cor $G/m2min   # 78 OK
 ```
 
 ## Suggested next steps
 
-1. **`SYSTEM` facilities**: expose at least `BYTE` / `WORD` /
-   `SHORTADDR` etc. so `FROM SYSTEM IMPORT …` works.
+1. **`SYSTEM` facilities**: expose `BYTE`/`WORD`/`SHORTADDR` etc.
 2. **Anchored subranges** `T[lo..hi]` (small grammar addition).
 3. **Unnamed proc-type parameters** (small grammar addition).
-4. **Array constructors** `array{…}` (bigger: codegen).
+4. Resolve cross-module imports in the harness (pass each file's
+   imported `.def`s) to remove the LIB noise and isolate LANG.
 
 ## Files
 
-`tools/v3-corpus/{corpus.sh,README.md}`,
-`compiler/src/compiler.frm` (long-line fix),
+`tools/v3-corpus/{corpus.sh,README.md}`, `compiler/src/compiler.frm`
+(long-line fix), `compiler/src/M2.atg` (`EXPORT QUALIFIED`),
 `compiler/tests/t_longline.mod`, `compiler/run_tests.sh`, this doc.

+ 9 - 0
tools/v3-corpus/README.md

@@ -23,4 +23,13 @@ A "pass" is syntactic+semantic cleanup, not a linkable image (many
 testsuite files are DEFINITION/IMPLEMENTATION modules with no program;
 V3 reports those as `Incorrect source` but with 0 errors).
 
+An IMPLEMENTATION file is compiled together with its sibling
+DEFINITION (same basename, `.def`/`.DEF`) when present, so a `.mod`
+does not fail on its own interface.
+
+Corpora used:
+- gm2 testsuite `pim/pass` + `iso/run/pass`;
+- the gm2 standard libraries, e.g.
+  `$HOME/bin/Modula2/Gm2/lib/gcc/x86_64-pc-linux-gnu/16.0.1/m2/{m2pim,m2iso,m2log,m2cor,m2min}`.
+
 Current baseline: `docs/summary_v3-corpus.md`.

+ 10 - 1
tools/v3-corpus/corpus.sh

@@ -32,7 +32,16 @@ for dir in "$@"; do
     n=$((n+1))
     w="$WORK/f$n"; mkdir -p "$w"; base=$(basename "$f")
     cp "$f" "$w/$base"
-    "$M2" "$w/$base" >/dev/null 2>&1
+    # if this is an IMPLEMENTATION, compile its sibling DEFINITION first
+    files="$w/$base"
+    case "$base" in
+      *.mod|*.MOD)
+        sib="$(echo "$base" | sed -E 's/\.[^.]*$/.def/')"
+        [ -f "$dir/$sib" ] || sib="$(echo "$base" | sed -E 's/\.[^.]*$/.DEF/')"
+        if [ -f "$dir/$sib" ]; then cp "$dir/$sib" "$w/$sib"; files="$w/$sib $w/$base"; fi
+        ;;
+    esac
+    "$M2" $files >/dev/null 2>&1
     rc=$?
     lst="$w/$(echo "$base" | sed 's/\.[^.]*$//').LST"
     if [ "$rc" -ge 128 ]; then echo "CRASH||$f" >> "$WORK/results.txt"; continue; fi

Някои файлове не бяха показани, защото твърде много файлове са промени