30-scanner-module.md 2.5 KB
Scanner (scanner.txt 1901 lines, SCANNER.DEF/MOD)
Lexer + source window + tiny allocators. All parsers call
GetSym/Accept/Expect/Test.
State
| old | proposed | role |
|---|---|---|
| global2 | scanOptions |
0=list, 9=caseSens, 12=noHalt |
| global3 | curChar |
377C=EOF |
| global5 | curSym |
token 0..60; 10=ident |
| global6 | isLiteral |
after number/string |
| global7 | cardVal |
integer literal |
| global8/9 | curNode/identKind |
found node / 0=ident,1=num/str,6=with,7=other |
| global11 | tokenBuf[0..128] |
ident/number text + 0C |
| global12 | curType |
CARD/REAL/LONGREAL/... of literal |
| global13 | curScopeWalk |
SymTab.global9 chain |
| global14/17 | realVal/longVal |
via Doubles.StrToDouble |
| global16 | srcFile |
Files.ReadBytes/SetPos/Open |
| global4 | codeFile |
output of Compile |
| global21/22/23 | codeBuf/codeSize/listing |
GENZ80 4K buffer |
| global24 | charClass[0..255] |
10=letter, 11=digit, 12=quote ... |
| global25/26 | tokPos/tokCol |
start of token |
| procPtr27 | keywordHash |
(ADR(tokenBuf),case)->CARD |
| buffer/bufIndex/filePos/column | same | 128B window at 0080H/006CH/006EH/0070H |
| stackLimit | same | heap guard at 0316H, FREEMARKER=3AE3H |
Pipeline: OpenSource(proc29) -> InitScan(proc28) -> NextChar(proc2) ->
scanNextToken(proc39) -> GetSym(proc1) + Number/SkipComment helpers.
128B window: bufIndex=128 = empty; NextChar calls Refill(proc34)
(ReadBytes 128, pad EOT=032C), else buffer[bufIndex++],
INC(filePos), echo if 0 IN options (EchoChar proc33).
Procedures (proposed)
| old | proposed |
|---|---|
| proc1 | GetSym |
| proc39 | ScanNextToken |
| NUMBER | ParseNumber (base *10/*8/*16, .->real, D/E exp, H/O/C/L suffix) |
| proc38 | SkipComment (nested (* *), $L+- toggles options) |
| proc2/34/28/29 | NextChar/RefillBuffer/InitScan/OpenSource |
| proc32/33 | ListPrefix/EchoChar |
| proc4/8/9/10/11 | Accept/Expect/TestSet/Test40/Test13 |
| proc12/13/21 | NeedIdent/ExpectKind/ExpectString |
| proc3 | PushWithScope |
| proc25 | AddExtension |
| proc26 | ScanError -> Errors.proc1 |
| proc27 | EnterSymtabKey |
| proc7=Allocate | HeapAlloc (was Z80) |
| proc5/6 | GetStackMark/CheckStack |
proc22/23/24=FindIdent/StrCmp/StrLen |
keyword/scope search (were Z80) |
| proc16/15/14 | CopyString/NewNode/NewSizedNode |
| proc18/19/20/17 | MkIdent/DeclIdent/InsertSym/NewScope |
proc30=COMPIL |
Compile (top driver) |