diff --git a/docs/reference/v4-dictionary.tsv b/docs/reference/v4-dictionary.tsv new file mode 100644 index 00000000..0999e72d --- /dev/null +++ b/docs/reference/v4-dictionary.tsv @@ -0,0 +1,321 @@ +# docs/reference/v4-dictionary.tsv -- the dictionary of v4 (#157): name, stack notation, +# description, tab-separated. The names are the live system's (v4/tools/mkdictionary.py +# check refuses a missing or a stale one). A row whose description starts TODO is a draft. +! TODO +# TODO +#> TODO +#S TODO +' ( -- addr ) TODO FORTH-79: the next word's parameter field address; if compiling, compiled as a literal +( TODO ---- comments (5.15) ------------------------------------------------------- PAREN skips to the closing parenthesis; BACKSLASH skips the rest of the line. +(.") TODO The run time of ." : print the string after the call, go on after it. It takes the return address off before it calls anything and keeps the count there instead, so a word that prints text goes no deeper than one that calls any other word. +(ABORT") ( flag -- ) TODO the run time of ABORT" : if the flag is not zero print the string after the call, start a new line and ABORT; otherwise go on after the string. +(BITS) ( -- n ) TODO +(CONSOLE) TODO +(LOG") ( level -- ) TODO the run time of LOG-xxx" : the text is after the call, as for ." ; log it at that level and go on after it. +(LOST) TODO +(ME) TODO +(NB) TODO +(REFUSED) TODO +(S") TODO The run time of S" : leave the address and length of the string after the call, and go on after it. +(SEAL) ( -- ) TODO WHAT IS IN THE DICTIONARY NOW IS THE SYSTEM: COLD comes back to it and FORGET will not go below it. Whoever has loaded a node's capsules has it do this (system.f18, (BOOT) and FENCE). +(UD) TODO +(WORD-DEFINED) TODO +(WORD-FORGOTTEN) TODO +* TODO The low cell of the product, by N steps of +* with nothing but the count on the return stack. UM* drop gives the same cell but needs two more return entries, which a word inside two DO loops does not have (D-2). The low cell is exact: what +* loses at the top of T (D-3) takes N shifts to reach the bit that moves into A, and the loop ends first. +*/ TODO +*/MOD TODO ++ TODO ---- arithmetic ---- ++! TODO ++LOOP TODO +, TODO , calls nothing: the code generator calls it from as deep as anything goes. +- TODO +-! TODO +--> ( -- ) TODO go on with the next block. Only in a block being loaded: at the terminal it is an error (code 14). +-ROT TODO +-TRAILING TODO +. TODO +." TODO FORTH-79: ." text" prints the text -- now, if interpreting; when the word it is compiled into runs, if compiling. +.R TODO +.S ( -- ) TODO as v3: the depth between < and >, then every value on the stack, the deepest first, each followed by a blank; then a new line. The stack is left as it was. It reads each value with PICK, so it needs a few cells of the stack free to work in. +/ TODO +/MOD TODO +0< TODO +0<> TODO ---- comparison ---- +0= TODO +0> TODO +1+ TODO +1- TODO +2! TODO +2* TODO +2+ TODO +2, TODO +2- TODO +2/ TODO +2>R TODO +2@ TODO ---- memory ---- +2DROP TODO +2DUP TODO +2OVER TODO +2R> TODO +2R@ TODO +2ROT TODO +2SWAP TODO ---- stack ---- +79-STANDARD TODO FORTH-79: execute to be sure a FORTH-79 system is there. It is: nothing happens. (v3's prints two lines and leaves a flag.) +: TODO FORTH-79: start a definition; it cannot be found until ; ends it. The CONTEXT vocabulary is set to the CURRENT one. (C)+2 holds LATEST while the entry is made, to tell whether one was. +; TODO FORTH-79: compile the return, let the word be found, stop compiling +< TODO +<# TODO ---- pictured output ---- +<= TODO +<> TODO += TODO +> TODO +>= TODO +>BODY TODO +>IN TODO +>LINK TODO +>NAME TODO +>R TODO ---- return stack: in line, and only inside a definition ---- +? ( addr -- ) TODO print what the cell holds +?DO TODO +?DUP TODO +?TERMINAL TODO ---- strings and memory ---- +@ TODO ---- memory ---- +ABORT TODO FORTH-79: clear the data and return stacks, set execution mode, return control to the terminal. As v3, the line it stops ends with " ok". Both are emptied before anything is called: ABORT may be run with the return stack full. +ABORT" ( flag -- ) TODO ABORT" text" as v3 (it is FORTH-83's, not FORTH-79's): if the flag is not zero, print the text and ABORT. +ABS TODO +ACL-ALLOW! TODO +ACL-ALLOW@ TODO +ACL-HEAT@ ( xt -- heat ) TODO how often the word has been used. The node's heat counters are not readable from a programme yet (D-6): 0. +ACL-HOOK TODO +ACL-INHERIT ( src dst -- ) TODO as v3: dst takes src's mode; it is not pinned, its TTL is 0 and it is allowed. This is the one word that unpins. +ACL-INIT-PRIMITIVES ( -- ) TODO as v3: do that to every word in the dictionary, in every vocabulary +ACL-MODE! TODO +ACL-MODE@ TODO +ACL-PIN TODO +ACL-PINNED? TODO +ACL-TTL! TODO as v3: below 0 is 0. The TTL is sixteen bits here: above 65535 is 65535. +ACL-TTL@ TODO +ACL-WORD-ID ( xt -- n ) TODO a number for the word that never changes: its address +AGAIN TODO +ALIGN TODO +ALLOT TODO +AND TODO ---- logic and comparison ---- +AWAIT ( node -- how ) TODO WAIT FOR THE NODE'S WORD OF HOW TEXT ENDED: 0 QUIT, 1 completed, 2 an error. (AWAIT-FROM) holds the node while this node waits, and 0 otherwise. It sleeps, executing nothing, until a message comes; then it passes on what is for other nodes, (SERVE), and goes through what is for itself (MESH.md 7d.4): the answer -- a message of type 3 from that node -- is taken, and the wait is over; a NACK is taken: about that node, the wait ends in error 19, Message refused; about another it is counted, as at any time; a GONE is taken: believed (its centre sent it: (GONE?)) and about that node, the wait ends in error 20, Node gone; text from its console ends the wait in error 21, Interrupted (7b.6). The text is not taken: it stays on its wire and is done when this text has been finished with. It is the one way to end a wait on a node that is alive and never answers, short of Hera's killing that node; anything else stays on its wire, in its order, for when this node has nothing to do. WHAT HAS COME ALREADY is on the wires and is found the first time through: the node looks before it sleeps. A message it left on a wire does not keep it awake. The stack what it does needs is tried first, (ROOM), core.f18: text that has left too little ends "Stack overflow" with nothing taken off a wire. +BASE TODO +BEGIN TODO A loop that tests at its end jumps back with the flag still there, so it jumps back to a drop laid just ahead of the body: BEGIN a UNTIL jump L0 Ld: drop L0: a if Ld drop BEGIN a WHILE b REPEAT jump L0 Ld: drop L0: a if Lx drop b jump L0 Lx: drop BEGIN a AGAIN jump L0 Ld: drop L0: a jump L0 The control-flow stack holds Ld, L0 and 3. +BL TODO ---- in-line addresses and constants ---- +BLANK TODO +BLK TODO +BLOCK ( n -- baddr ) TODO FORTH-79: the address of a buffer that holds block n, read from storage if it is not already in one. +BUFFER ( n -- baddr ) TODO FORTH-79: the address of a buffer assigned to block n. What the buffer holds is not read from storage. +BYE TODO +C! TODO +C, TODO +C@ TODO +CASE TODO +CELLS TODO +CFA TODO +CMOVE TODO ---- section 5.9: strings ------------------------------------------------ +CMOVE> TODO +COLD ( -- ) TODO the system as it was when the capsule had just been loaded: everything defined since is gone, there is one vocabulary, numbers are decimal, the block buffers are empty. Then as WARM. +COMPARE TODO +COMPILE TODO COMPILE xxx FORTH-79: when the word this is in runs, xxx is compiled +CONSTANT ( n -- ) TODO FORTH-79: an entry whose word leaves n +CONTEXT TODO ---- vocabularies (FORTH-79) ---------------------------------------------------- A vocabulary is two cells: its head -- the xt of its newest entry, or 0 -- and the address of the vocabulary defined before it, or 0. VOC-LINK holds the newest one's address. FORTH's head is (LATEST) and it is in no chain. +CONVERT ( d1 baddr1 -- d2 baddr2 ) TODO FORTH-79: the characters from baddr1 + 1 on are taken as digits in the current BASE, each accumulated into the double after it is multiplied by BASE, until one is not a digit; baddr2 is its address. (P)+2 holds the digit while the double is multiplied and (P)+5 the address, so nothing waits on the return stack across the calls. +COUNT TODO +CR TODO +CREATE TODO FORTH-79: an entry whose word leaves the address of its parameter field +CURRENT TODO +D+ TODO ---- section 5.7: doubles ------------------------------------------------ +D- TODO ---- doubles and mixed ---- +D. TODO +D.R TODO +D0< TODO +D0= TODO +D2* TODO +D2/ TODO +D< TODO +D= TODO +DABS TODO +DECIMAL TODO +DEFAULT-ROUTE ( port -- ) TODO the port, by its number, leads toward every node not told of +DEFER TODO +DEFER@ TODO +DEFINITIONS TODO FORTH-79: new definitions go into the CONTEXT vocabulary from now on. +DEPTH TODO FORTH-79: how many values were on the stack before DEPTH +DMAX TODO +DMIN TODO +DNEGATE TODO +DO TODO +DOES> TODO +DROP TODO +DUMP TODO +DUP TODO ---- stack ---- +ELSE TODO +EMIT TODO EMIT puts the character with what has been printed. Everything the node prints goes through EMIT. A is kept: the words that print have always been free to use it across an EMIT. As v3, only the low 8 bits of the character are printed. A BUFFER FOUND FULL is sent first. It is full when a flush of it ended in a fault -- the stack too deep to begin a message -- and what is printed next is that fault's own message: put where (OUT^) points, it would go past the buffer's end, and (OUT^) would never again be at the end to be flushed (MESH.md 7c.6). +EMPTY-BUFFERS TODO FORTH-79: forget what is in the buffers; nothing is written. +ENCLOSE ( baddr c -- baddr n1 n2 n3 ) TODO in the zero-terminated string at baddr: n1 the offset of the first character that is not c, n2 the offset of the first c after it, n3 the offset of the first character after that run of c. +ENDCASE TODO +ENDOF TODO +ERASE TODO +EXECUTE TODO +EXIT TODO FORTH-79: return from the definition at this point +EXPECT ( baddr n -- ) TODO FORTH-79: characters from the terminal are stored from baddr upward until a new-line or until n have been received; the new-line is taken but not stored. A zero is added after the text, so the buffer must hold n + 1 bytes. No action for n <= 0. SPAN is how many characters were stored (SPAN is not FORTH-79; v3 has it). A line longer than n leaves the rest, its new-line included, to be read next. It is what the prompt reads every line with, with whatever the user has left on the stack beneath it, so it works in (P) -- (P)+0 where the next character goes, (P)+1 how many more may come, (P)+2 where the first went -- and has at most three cells of its own on the data stack (D-2). +FALSE TODO +FENCE TODO +FILL TODO +FIND ( -- xt | 0 ) TODO FORTH-79: the compilation address of the next word in the input stream, or 0 if it is not in the dictionary. +FLUSH TODO As v3: the same. +FORGET TODO FORGET xxx FORTH-79: remove xxx and every word defined after it, in whatever vocabulary; the space they took is free again. A vocabulary that goes is no longer CONTEXT or CURRENT: FORTH is. A word that is not there is an error, and so is one below FENCE, which is where the capsule's own words are (code 10, "Protected word"). +FORTH TODO FORTH-79: the primary vocabulary; it becomes the CONTEXT vocabulary. +GONE ( n node -- ) TODO tell the node that node n is gone. It acts on that only if this node is its centre: it forgets the way to n, and if it is waiting for n's answer the wait ends. +HERE TODO +HEX TODO +HIDDEN TODO +HOLD TODO +I TODO +IF TODO Native if leaves the flag, so both ways out of it drop it (section 2): IF a THEN if L1 drop a jump L2 L1: drop L2: IF a ELSE b THEN if L1 drop a jump L2 L1: drop b L2: +IMMEDIATE TODO +INTERPRET ( -- ) TODO FORTH-79: take words from the input stream until it is exhausted. A word in the dictionary is executed, or, when compiling and not immediate, compiled. Anything else must be a number in the current BASE: it is left on the stack, or compiled as a literal. What is neither, or a compile-only word met while interpreting, abandons the line, with v3's message: UNKNOWN WORD: 'xxx' xxx: compile-only +INVERT TODO +IS TODO +J TODO +KERNEL-WORD ( n -- ) TODO KERNEL-WORD xxx A WORD THAT ASKS THE KERNEL (docs/v4.0.0/ ENGINE.md 3.3). xxx is made a word whose whole body is "write n to the port": n is the number of a request, the port is the node's way to its kernel, and the write blocks the node until the kernel has served it (DECOMPOSITION.md section 6). xxx's arguments are whatever it finds on the data stack and its results are what the kernel leaves there. This is what a word that is a C function in v3 -- BIRTH, BLOCK, BYE -- is on a node. Not FORTH-79 and not in v3, where such a word is registered from C. The constant (PORT) is the port's word address. +KEY TODO +LATEST TODO +LEAVE TODO +LFA TODO +LINK> TODO +LIST TODO ---- LIST ---------------------------------------------------------------------- ( n -- ) as v3: a new line, "Block n", then sixteen lines of 64 characters, each after its number, and an empty line. SCR is left holding n. A character that is not printable is shown as a blank. (Q)+0 is the line, (Q)+1 the column, (Q)+2 the block's number. +LITERAL ( x -- ) TODO FORTH-79: if compiling, compile x as a literal +LOAD ( n -- ) TODO FORTH-79: interpret block n, then go on with what follows. +LOG-DEBUG TODO +LOG-DEBUG" TODO +LOG-DEBUG-STR ( baddr u -- ) TODO +LOG-ERROR TODO +LOG-ERROR" TODO +LOG-ERROR-STR ( baddr u -- ) TODO +LOG-INFO TODO +LOG-INFO" TODO +LOG-INFO-STR ( baddr u -- ) TODO +LOG-LEVEL! ( n -- ) TODO as v3: a level below 0 is 0, one above 4 is 4 +LOG-LEVEL@ TODO +LOG-TEST TODO +LOG-TEST" TODO +LOG-TEST-STR ( baddr u -- ) TODO +LOG-WARN TODO +LOG-WARN" TODO +LOG-WARN-STR ( baddr u -- ) TODO +LOOP TODO +LSHIFT TODO ---- shifts ---- As v3, a count that is negative or as large as the cell is wide is an error (code 9, "Shift count out of range"). +M* TODO The product's sign waits in (F), not on the return stack, for the same reason as SM/REM's. +M+ TODO +M- TODO +M/MOD TODO +MAX TODO +MIN TODO +MINT-ZUSE TODO +MOD TODO +MOVE TODO FORTH-79: n cells from addr1 to addr2, the cell at addr1 first. +NAME> TODO +NEGATE TODO +NEIGHBOUR ( node port -- ) TODO the node on the other end of the port, by its number; 0: none, or a device. Whoever wires the node tells it, and tells it again when the wiring changes. It is how a node knows who its centre is, the one node whose GONE it believes: (CENTRE). +NFA TODO +NO-ROUTE ( node -- ) TODO forget the way to that node, if one is known: what is for it then goes by the way for every node not told of, if there is one. The last way in the table takes its place. +NO-ROUTES ( -- ) TODO forget every way: for when the wiring has changed +NODE-ERROR TODO FORTH-79: clear the return stack, set execution mode, return control to the terminal; no message is given. The data stack is left as it is. NODE-ERROR, by name: how a definition written in FORTH raises an error. A store of a code that is not zero is the trap of D-18 -- the word goes no further, the message for the code is printed (-1: the word has printed its own) and the line ends with ERROR. The codes are listed in core.f18. +NOP TODO +NOT TODO +NUMBER ( baddr -- d ) TODO FORTH-79. What is not a number gives 0 and sets NODE-ERROR. +OCTAL TODO +OF TODO +OR TODO +ORDER ( -- ) TODO as v3's, but true: where names are looked for, and where new ones go +OVER TODO +PAD TODO +PAGE ( -- ) TODO clear the screen and put the cursor at the top, as v3: ESC [2J ESC [H +PFA TODO +PICK ( n -- x ) TODO FORTH-79: a copy of the n-th value, not counting n itself. 1 PICK is DUP, 2 PICK is OVER. n less than one is an error. +PORT! ( w port -- ) TODO write a cell to the port, by its number. It waits until the neighbour has taken it. This is how a node is sent a capsule of F18 code (docs/v4.0.0/MESH.md section 5): the cells go as they are, not as a message, to a node that is executing from its ports. +Q.* TODO ---- multiply ---- +Q.+ TODO ---- the double words under other names ---- +Q.- TODO +Q./ TODO +Q.0 TODO +Q.0= TODO +Q.1 TODO +Q.< TODO +Q.= TODO +Q.> TODO +Q.ABS TODO +Q.COS TODO +Q.EXP TODO ---- e to the x ---- +Q.FROM-INT TODO ---- conversion and constants ---- +Q.LOG TODO ---- natural logarithm ---- +Q.MAX TODO +Q.MIN TODO +Q.NEG TODO +Q.PRINT TODO ---- printing ---- The integer part, a point, five digits, a blank; always in decimal. +Q.SCALE TODO +Q.SIN TODO +Q.SQRT TODO ---- square root ---- +Q.TO-INT TODO +QUERY ( -- ) TODO FORTH-79: up to 80 characters, or a line, into TIB; >IN to 0. +QUIT TODO +R> TODO +R@ TODO +REPEAT TODO +ROLL ( n -- ) TODO FORTH-79: the n-th value, not counting n itself, is taken out and put on top. 3 ROLL is ROT, 2 ROLL is SWAP, 1 ROLL does nothing. n less than one is an error. +ROT TODO +ROUTE ( node port -- ) TODO the port, by its number, leads toward that node +RSHIFT TODO +S" ( -- baddr u ) TODO S" text" as v3. In a definition the text is compiled into it. At the prompt it is copied to PAD, where it stays until PAD is used again: WORD's buffer is overwritten by the very next word of the line. +S>D TODO +SAVE-BUFFERS TODO FORTH-79: write every block that has been UPDATEd. +SCAN TODO +SCR TODO +SEARCH TODO +SEND ( baddr u node -- ) TODO send the node u characters of text to interpret, by the way this node knows to it. What it prints there goes to that node's console. With no way to the node it is error 12. +SEND-ON ( baddr u node port -- ) TODO the same by a port this node names, by its number, whatever ways it knows: for a neighbour that has no number of its own yet, to which text is sent as to node 0. +SIGN TODO +SKIP TODO +SMUDGE TODO +SOURCE ( -- baddr u ) TODO the input buffer and how much of it is in use +SPACE TODO +SPACES TODO ---- spaces and the number base ---- +SPAN TODO +STATE TODO +SWAP TODO +THEN TODO +THRU ( first last -- ) TODO as v3: LOAD each block from first to last. +TIB TODO +TRAVERSE ( baddr n -- baddr' ) TODO as v3: for n > 0, from a name's count byte to the byte after its last character; otherwise the address unchanged. +TRUE TODO +TYPE TODO +U* TODO +U. TODO +U.R TODO +U/MOD TODO +U< TODO +U> TODO +UM* TODO ---- section 4: multiply ------------------------------------------------ +UNLOOP TODO +UNTIL TODO +UPDATE TODO FORTH-79: mark the block most recently asked for as changed, so that it is written before its buffer is used for another. +VARIABLE TODO FORTH-79: CREATE and one cell. The standard leaves its first value to the programme; here it is 0. +VERSION ( -- ) TODO which system this is, and its cell width +VLIST TODO +VOCABULARY TODO +WARM ( -- ) TODO as v3's message, and then ABORT: both stacks emptied, the prompt +WHILE TODO +WITHIN TODO +WORD TODO +WORDS ( -- ) TODO the names in the CONTEXT vocabulary, the newest first, a blank after each, a new line whenever one has passed column 64. Hidden entries -- a definition under way -- are not shown. (Q)+0 is the column. +XOR TODO +[ TODO +['] TODO ['] xxx the same inside a definition, whatever STATE is; and [LITERAL], which v3 has as another name for LITERAL. +[COMPILE] TODO [COMPILE] xxx FORTH-79: compile xxx even though it is immediate +[LITERAL] TODO +\ TODO +] TODO diff --git a/docs/v4.0.0/plans/2026-11-10-alpha4-documents.md b/docs/v4.0.0/plans/2026-11-10-alpha4-documents.md index 9b102e64..0fa63526 100644 --- a/docs/v4.0.0/plans/2026-11-10-alpha4-documents.md +++ b/docs/v4.0.0/plans/2026-11-10-alpha4-documents.md @@ -14,9 +14,9 @@ Captain Bob, 2026-11-10: "Do it" -- the rest of alpha.4 that lives in the tree, PDF in `$DOCS`, so the release carries it. ## #157 -- the dictionary of v4, and the .com's Documents page -- [ ] the live word list of the built system (318 words) -- [ ] the table in the tree: name, stack effect, description, where defined -- [ ] the check: every live word has an entry, every entry is a live word +- [x] the live word list of the built system (318 words): `v4/tools/mkdictionary.py words` -- the hosted v4's `WORDS` (317) plus the kernel's `sk_v4_words[]` (`MINT-ZUSE`) +- [~] the table in the tree, `docs/reference/v4-dictionary.tsv`: drafted for all 318 (`mkdictionary.py draft`; the source gave a stack comment for 54, a description for 105); the rest written by hand in batches, alphabetical -- next: batch 1 +- [x] the check: `mkdictionary.py check` -- every live word has a finished entry, every entry a live word (fails on all 318 drafts today, as it should); not yet in the build - [ ] the PDF, alphabetical; `make dictionary`; the nightly's docs stage, classifier `dictionary` - [ ] the Documents page: canvas artboard, then `site/com` diff --git a/v4/tools/mkdictionary.py b/v4/tools/mkdictionary.py new file mode 100644 index 00000000..7aa3fc6f --- /dev/null +++ b/v4/tools/mkdictionary.py @@ -0,0 +1,170 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: LicenseRef-Starship-1.0 +# Copyright (c) 2023-2026 Robert A. James +"""mkdictionary.py -- the dictionary of v4 (#157): every word the system boots with, alphabetical, +with its stack notation, a description, and where it is defined. + + v4/tools/mkdictionary.py words the live word list, one per line + v4/tools/mkdictionary.py draft TSV a table for every live word not in TSV yet, appended, + its stack and description taken from the source where + the source has them (to be read and finished by hand) + v4/tools/mkdictionary.py check TSV every live word has an entry with a stack and a + description, and every entry is a live word: 0, or 1 + with what is wrong + v4/tools/mkdictionary.py md TSV OUT.md the dictionary as Markdown, alphabetical, for the book + +The live word list is the system's own: the hosted v4 (`make -C v4 hosted`) booted and asked +`WORDS`, plus the kernel's words of the lone node (`sk_v4_words[]` in kernel/src/v4/sk_v4.c) -- +the words a booted StarshipOS has. The table (docs/reference/v4-dictionary.tsv) holds the stack +notation and the description, which only a person can write; the names are never typed by hand. +""" +import os, re, subprocess, sys + +ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")) +HOSTED = os.path.join(ROOT, "v4", "build", "starforth4-amd64") +SOURCES = ([os.path.join("capsules/v4/sys", f) for f in sorted(os.listdir(os.path.join(ROOT, "capsules/v4/sys"))) + if f.endswith(".f18")] + + [os.path.join("capsules/v4", f) for f in sorted(os.listdir(os.path.join(ROOT, "capsules/v4"))) + if f.endswith(".4th")]) +KERNEL_WORDS_SRC = "kernel/src/v4/sk_v4.c" + + +def live_words(): + """The hosted system's WORDS, then the kernel's own words not among them.""" + if not os.path.exists(HOSTED): + sys.exit(f"mkdictionary: {HOSTED} is missing: make -C v4 hosted") + out = subprocess.run([HOSTED], input="WORDS\n", capture_output=True, text=True, timeout=120).stdout + lines = out.split("\n") + try: + start = next(k for k, l in enumerate(lines) if l.startswith("ok> ") and l[4:].strip()) + except StopIteration: + sys.exit("mkdictionary: the hosted system printed no WORDS") + got = [] + for l in lines[start:]: + if l.startswith("ok> ") and got: + break + got.append(l[4:] if l.startswith("ok> ") else l) + text = " ".join(got) + if " ok" in text: + text = text[:text.rfind(" ok")] + words = text.split() + src = open(os.path.join(ROOT, KERNEL_WORDS_SRC)).read() + m = re.search(r"sk_v4_words\[\]\s*=\s*\{([^}]*)\}", src) + for k in re.findall(r'"([^"]+)"', m.group(1) if m else ""): + if k not in words: + words.append(k) + return words + + +def definitions(): + """Where each name is defined, and what its source says of it: {name: (file, stack, text)}.""" + found = {} + for rel in SOURCES: + lines = open(os.path.join(ROOT, rel), errors="replace").read().split("\n") + for i, l in enumerate(lines): + names = [] + if rel.endswith(".f18"): + names = re.findall(r'",\s+(\S+)\s+#,', l) + else: + names = re.findall(r"(?:^|\s):\s+(\S+)", l) + names += re.findall(r"\b(?:CONSTANT|VARIABLE|DEFER|CREATE|VOCABULARY)\s+(\S+)", l) + for n in names: + if n in found: + continue + stack, text = "", "" + inline = re.search(r":\s+" + re.escape(n) + r"\s+(\([^)]*\))", l) + if inline: + stack = inline.group(1) + # the comment block just above + j, block = i - 1, [] + while j >= 0 and lines[j].startswith("\\"): + block.insert(0, lines[j].lstrip("\\").strip()) + j -= 1 + comment = " ".join(b for b in block if b) + cm = re.match(r"(\([^)]*--[^)]*\))\s*(.*)", comment) + if cm and not stack: + stack, comment = cm.group(1), cm.group(2) + text = comment + found[n] = (rel, stack, text) + for k in re.findall(r'"([^"]+)"', re.search(r"sk_v4_words\[\]\s*=\s*\{([^}]*)\}", + open(os.path.join(ROOT, KERNEL_WORDS_SRC)).read()).group(1)): + found.setdefault(k, (KERNEL_WORDS_SRC, "", "")) + return found + + +def read_table(path): + table = {} + if os.path.exists(path): + for l in open(path).read().split("\n"): + # a comment is a line beginning with # that has no tab: a row always has tabs, and + # some words begin with # (#, #>, #S) + if not l.strip() or (l.startswith("#") and "\t" not in l): + continue + f = l.split("\t") + while len(f) < 3: + f.append("") + table[f[0]] = (f[1].strip(), f[2].strip()) + return table + + +def main(): + if len(sys.argv) < 2: + sys.exit(__doc__) + mode = sys.argv[1] + if mode == "words": + print("\n".join(live_words())) + return + path = sys.argv[2] + table = read_table(path) + words = live_words() + if mode == "draft": + defs = definitions() + new = [w for w in words if w not in table] + with open(path, "a") as f: + if not table: + f.write("# docs/reference/v4-dictionary.tsv -- the dictionary of v4 (#157): name, stack notation,\n" + "# description, tab-separated. The names are the live system's (v4/tools/mkdictionary.py\n" + "# check refuses a missing or a stale one). A row whose description starts TODO is a draft.\n") + for w in sorted(new): + rel, stack, text = defs.get(w, ("", "", "")) + desc = ("TODO " + text) if text else "TODO" + f.write(f"{w}\t{stack}\t{desc}\n") + print(f"mkdictionary: {len(words)} live words; {len(new)} drafted into {path}") + return + if mode == "check": + bad = [] + for w in words: + if w not in table: + bad.append(f"no entry: {w}") + elif not table[w][0] or not table[w][1] or table[w][1].startswith("TODO"): + bad.append(f"unfinished: {w}") + for w in table: + if w not in words: + bad.append(f"not a live word: {w}") + if bad: + print("\n".join(bad)) + print(f"mkdictionary: {len(bad)} problem(s) in {path} ({len(words)} live words)") + sys.exit(1) + print(f"mkdictionary: {path} covers all {len(words)} live words") + return + if mode == "md": + defs = definitions() + out = sys.argv[3] + with open(out, "w") as f: + f.write("# The dictionary of v4\n\n") + f.write(f"Every word StarForth v4 boots with -- {len(words)} of them -- in alphabetical order: its " + "stack notation (what it takes and leaves: `( before -- after )`), what it does, and " + "where it is defined.\n\n") + f.write("| Word | Stack | Description | Defined in |\n|---|---|---|---|\n") + for w in sorted(words, key=lambda x: (x.upper(), x)): + stack, desc = table.get(w, ("", "")) + rel = defs.get(w, ("", "", ""))[0] + cell = lambda x: x.replace("|", "\\|") + f.write(f"| `{cell(w)}` | `{cell(stack)}` | {cell(desc)} | `{rel}` |\n") + print(f"mkdictionary: {out}, {len(words)} words") + return + sys.exit(__doc__) + + +if __name__ == "__main__": + main()