M2compiler-V3/docs/separate-compilation.md

Separate compilation — implementation note

Status: core shipped (2026-10-24).

V3 now supports per-unit objects. The original one-shot multi-file mode is still the default for the test suite and corpus; separate compilation is additive.

Workflow

The compiler itself emits QBE IR, one unit at a time. M2make --separate drives the whole object build and link:

M2make --separate --dialect iso \
       --m2bin ./compiler/M2 --shim runtime/syslib/shim.c \
       -o Prog Prog.mod

It resolves the imported .def files transitively, compiles each stale .mod against interfaces only, writes objects to gen_ssa/<Module>.o, and links them with cc plus the shared runtime shim. No .def is passed to cc. A second run reports up to date; touching a source recompiles only that object and relinks. A newer compiler binary is treated as an ABI event, and changes to the shim trigger a relink.

For a single unit, M2 -c emits QBE IR directly:

M2 -c Lib.mod                       # picks up Lib.def automatically
M2 -c Lib.def Prog.mod              # explicit imported interface
qbe -o gen_ssa/Lib.s gen_ssa/Lib.ssa
cc -c gen_ssa/Lib.s -o Lib.o

M2 -c requires the imported interfaces on the command line (except the target's own .def, which is found beside the .mod). M2make is the transitive-interface resolver.

Object model

• Library objects contain declarations and bodies for one module and no main. Every library object exports <Module>_init, even when the module has no BEGIN body.
• The program object is the only object with main; it calls all imported module inits in dependency order, then the program's own init, then returns ExitCode.
• Module-level procedures use stable symbols <Module>__<name>. Globals stay <Module>_<name>; literals, constructors and sets are module-qualified (<Module>_strN, <Module>_ctoN, <Module>_setN); class vtables are <Module>__<Class>__vt.
• All top-level data is emitted with QBE export data, so a program object can reference an imported module's exported variable.
• Nested procedures remain object-local (module prefix + uid).
• Module init order comes from the interface order; M2make's topological sort supplies dependencies first.

The implementation is split across:

Area	Change
SymTab	uid → module/depth/class tables; stable type/class names
QbeGen	stable Mangled, module-qualified pools, export data, library/program split, external init list
Lower	-c mode, empty init for bodyless objects, target .def data emission, class/vtable stability
compiler.frm / M2.mod	-c, -I transitive .def scanner, sibling .def lookup, imported-def init list, target def matching
M2make	--separate: per-module M2 -c -I + qbe + cc, incremental stale checks and final link

Limits / next phases

• Build freshness uses mtimes plus a small content ABI/flags stamp (gen_ssa/.m2abi). The stamp covers the ABI version and --m2flag settings; per-object binary metadata is not implemented.
• Type/value/procedure and generic-class instances are emitted as separate objects by M2make: it detects MODULE X = G(actuals), builds X.o from the template plus instance declaration, and skips the abstract generic object. Imported instance implementations are parsed for method symbols but are not lowered into the client object.
• Classes have stable vtable/method symbols for normal imported class use. M2make now also builds a definition-only object for a module with no .mod, so a class declared only in a .def gets its (zero-method) vtable object.
• Per-object self-hosting is green: bootstrap/fixpoint-separate.sh builds the compiler as objects with stage1, links stage2, rebuilds with stage2, links stage3, and compares all per-object .ssa. Compiler units use -legacyopen1d; user programs use pointer views.
• M2 -I scanning is conservative: import files are read up to a 64 KiB buffer and interfaces are ordered by dependency.

Per-object fixpoint

TMP="$PWD/.scratch/fixpoint-separate" ./bootstrap/fixpoint-separate.sh
# SEPARATE FIXPOINT OK: stage2 == stage3 (11 objects)

The script passes --m2flag -legacyopen1d because the compiler itself still uses the legacy 1-D call ABI; user programs use pointer-based views.

Tests

compiler/run_tests.sh has separate-mode gates:

• SepProg — exported data, procedure call, library BEGIN, link and run (42), library object must not define main.
• SepEmptyProg — a library with no BEGIN still exports SepEmpty_init and links (42).

Manual class check: a class in CLib.def/.mod linked into CProg returns 42 through a virtual call, with vtable/method symbols resolved across objects.