CLI and cache: nish, nish run and the files and processes they touch

The security audit's record for the CLI stage (issue #363). It says what was checked, how, what was found, and which test pins each answer. Base is main at 083339e.

Scope

File Functions
src/run-cache.ts runCacheRoot, runBinaryName (new), fnv1a64Hex, fileFingerprint, runCacheKey
src/compile.ts main (the run split and --link probe), makeDirectory, makeDirectoryFor, planOutputs, perModule, cCompiler, missingToolchain, programOnPath, packageRootCandidates, runProgram, buildIntoCache, linkProgram, writeSidecars; and, since #546 (at 02d90e7), distrustOf, refuseCacheRoot, considerRoot, findPackage and PackageSearch, which replaced the audit's packageRoot and libraryRoot
src/run-cache.ts, src/compile.ts, scripts/build.sh (CLI-11, the runtime cache) runtimeCacheRoot, runtimeUnits, addRecipeFingerprints, addUnitFingerprints, runtimeCacheKey; cachedRuntime, pushRecipeFlags, and linkProgram's call into them; build.sh's --runtime-objects and --runtime-from modes, join_jobs and stop_jobs
src/build-cache.ts, src/compile.ts, scripts/build.sh (CLI-12, the module cache) moduleCacheRoot, moduleCacheKey; StagedModules, stageModules, keepModules, and linkProgram's call into them; build.sh's --module-objects
src/compile.ts (CLI-13, profile-guided links) pgoProblem, startsWithMagic, main's --pgo-generate and --pgo-use parse, refusals and llvm-profdata probe, and linkProgram's cache bypass for them
std/cli.ts paletteOf, styleFor, agentDriven: added after the audit, with the grouped --help (in src/cli-style.ts until they moved to the standard library, which the compiler imports by a relative path), and read for process and environment access only (below)
src/compilation.ts Compilation.resolveSpecifier (where nish/<module> is read from); the rest of the file was read for file and process access and has none beyond readFileSyncOrNull on module paths

Read but not owned, so any finding in them is recorded as open rather than fixed: scripts/build.sh (how the link's arguments reach clang), scripts/bootstrap.sh (where the intermediate stages live), and runtime/runtime-os.c (nish_put_file, nish_mkdir, nish_spawn_impl, nish_realpath: what each builtin actually does to the file system).

Added after the audit, with the grouped --help: paletteOf (std/cli.ts) starts /bin/sh -c by absolute path on the paths that print the usage, unless a variable has already forced colour on or off, with a constant script that takes nothing from the arguments: [ -t N ], N the descriptor 1 or 2, then tput colors, which reads how many colours TERM's terminfo entry declares.

tput is run as /usr/bin/tput or /bin/tput, never looked up on PATH, so a PATH holding . or an empty entry cannot run a tput a checkout planted; with neither there the probe answers without it ("--help: the colour probe never runs a tput found on PATH").

The shell and tput inherit the environment as every spawn of nish does, so TERM, TERMINFO and TERMINFO_DIRS choose the terminfo file tput parses: the victim's own settings, the misconfiguration case of the threat model, and the answer is only ever a colour count. usageText (src/compile.ts) calls findPackage for an agent's help only to print the absolute path of the package's docs/AI.md, which nish neither reads nor runs. The root it settles on is the one CLI-9 already covers.

Threat model

The victim runs nish <file.ts> ... or nish run <file.ts> as themselves. The attacker is one of:

  • another local user on a shared machine, who can write to world-writable directories (/tmp) and to anything the victim's umask leaves open, and can read anything the victim's modes leave readable;
  • the author of a checkout the victim compiles or runs a script from: they control every file in the working directory, every file and directory name in it, and the source being compiled. Compiling such a checkout must not run any of its code; nish run must run the program its source describes and nothing else;
  • the environment, only as far as a misconfiguration goes (a relative XDG_CACHE_HOME, a CC that names something else). The attacker does not set the victim's environment.

The goals considered: run code as the victim (a planted binary, a planted build.sh, a planted library module, shell text in an argument), start a stale binary for a changed program, write outside the outputs the command line names, and read what the victim's cache holds.

Method

  • Read every line of the three owned files that touches a path, a directory, an environment variable or a process, and followed each builtin into runtime/runtime-os.c to learn its real behaviour: mkdirSync is mkdir(path, 0777) (so the umask decides the mode, and an existing directory or a symlink to one counts as success), writeFileSync is open(O_WRONLY|O_CREAT|O_TRUNC, 0644) (it follows a symlink and does not insist on a new file), spawnSync is posix_spawnp with no shell (a PATH search happens only for an argv[0] without a /), and realpathSync is realpath(3).
  • Listed every input of the link — scripts/build.sh's options and the variables it reads (CC, WASI_SYSROOT, WASI_BUILTINS, NODE, NODE_INCLUDE, LLVM_PROFILE_FILE), the runtime files it adds beside runtime.c, and every #include "..." in runtime/ — and compared that list with what runCacheKey hashes.
  • Read scripts/build.sh for quoting: every expansion of a path is quoted, and the one bash -c in linkProgram takes its operands positionally.
  • Drove build/nish and the base compiler by hand in a scratch directory for each hypothesis below, then turned every confirmed one into a check in tests/nish/cli.ts, which npm test builds and runs against the compiler it tests (and npm run test:cli against build/nish). Each check was run against the compiler built from the base SHA and failed there, and passes on this branch.

Findings

Id Severity Where Description Disposition
CLI-2 High src/compile.ts:798 packageRootCandidates; src/compilation.ts:765 A compiler that does not sit in its package (a binary copied to ~/bin, a broken install) took the working directory as its package root. --link then ran ./scripts/build.sh from whatever directory it was started in, and nish/<module> compiled ./std/<module>.ts: compiling someone else's checkout executed their script. Reproduced: a copy of build/nish in another directory, run in a directory holding a scripts/build.sh that touches a file, touched it and exited 0. Rated High rather than Critical because it needs the compiler outside its package layout. Fixed. . is a candidate only when the compiler's own real path is inside the working directory, which is the checkout's own build/ and build/selfhost/ stages that the bootstrap relies on. When no candidate has the package, nish/<module> is resolved against the root the compiler's path implies (libraryRoot), never the working directory, and fails naming it. Tests: CLI-2: --link does not run the working directory's scripts/build.sh, CLI-2: and is refused with the toolchain band, since there is no package, CLI-2: nish/<module> is not read from the working directory's std/
CLI-1 Medium src/compile.ts:863 runProgram nish run key.ts linked the binary to <entry>/key, then wrote the key file over it, then started the key file. The run always failed with exit 3, on every attempt, and on a C library whose posix_spawnp falls back to /bin/sh on ENOEXEC the key text (the program's IR) would be handed to a shell instead. Not verified on such a libc; glibc refuses. Fixed. runBinaryName keeps the binary as key.bin (and tmp-* as tmp-*.bin), and the key carries the binary's name, so one entry only ever holds one script's binary. Tests: CLI-1: a script called key.ts runs, and answers its own status, CLI-1: and its own output, CLI-1: and runs again from the cache
CLI-3 Low src/run-cache.ts:38 runCacheRoot A relative XDG_CACHE_HOME (or HOME) resolved against the working directory, so a run started in a checkout looked its binary up in a cache the checkout could supply: a planted nish/run/<hash>/key and binary for the checkout's own script ran in place of the source. The XDG base directory rules say a relative value is invalid and is ignored. Needs the victim's environment to be misconfigured, hence Low. Fixed. A value that does not start with / is ignored, XDG_CACHE_HOME falling back to HOME and HOME to the refusal. That also makes every path the miss hands to mv and rm absolute, and both now take -- besides. Tests: CLI-3: and keeps nothing under the relative path, CLI-3: but under $HOME/.cache, as if it were unset, CLI-3: with no absolute HOME either, the run is refused with the toolchain band
CLI-4 Low src/compile.ts:929 buildIntoCache The cache root and every entry were made with mkdir(0777) under the umask, commonly 0755, and each key with 0644: on a machine whose home directories other users can enter, every user could read each cached program's whole IR, string literals included, and its binary. Fixed. A miss makes the cache root and then runs chmod 700 on it, and refuses the run when that fails, which it does for a root this user does not own. Every entry is made on a miss, so every entry is behind a private root. Test: CLI-4: the cache root is private to its owner
CLI-5 Low src/compile.ts:929 buildIntoCache The entry is named by a 64-bit FNV-1a, which is not collision resistant, so two programs' keys can name one entry. A run that moved its binary in and then died before writing its key left the other program's key beside it, and that program's next run was a hit that started the wrong binary. Fixed. The entry's key is emptied before the new binary is moved in; an empty key never equals a real one. Test: CLI-5: a run killed after its binary is moved in leaves no other program's key beside it
CLI-6 Medium runtime/runtime-os.c:127 nish_put_file, used by src/compile.ts:688 and writeSidecars Every output (.ll, sidecars, the cache key) is opened with `O_CREAT O_TRUNCand noO_NOFOLLOW. A source compiled with no -owrites .llbeside it, and a symlink at that name redirects the write to whatever file it names. The author of a checkout controls every name in it, so they need no race and no world-writable directory: they ship the source and a symlink at .ll, and the victim's plain nish file.tsdeterministically truncates and overwrites any file the victim can write. Linux'sfs.protected_symlinksonly applies in sticky world-writable directories such as/tmp`, so it does not help in a checkout or any other shared directory. A destructive write triggered by attacker input, hence Medium; not High, because there is no memory corruption and the attacker does not choose the content written.
CLI-7 Low src/compile.ts:863 runProgram at the base 083339e; the fix at 02d90e7: :1361 runProgram, :1437 refuseCacheRoot, :1121 distrustOf A hit reads the key and starts the binary without checking who owns the cache root, because that check has to stay cheap and the language has no lstat. CLI-4's chmod covers every root a miss reaches, so the gap is a root that another user created at the victim's cache path (only possible when that path is under a directory they can write) and filled with a key and binary for a program whose IR they can predict. Fixed (#486). Every run, a hit as much as a miss, asks about the root first (refuseCacheRoot → distrustOf in src/compile.ts) and refuses one that is not a directory (a symbolic link is not followed), that this user does not own, or that its group or every user can write, with exit 3 and NL0002 (run: refusing the cache root <root>, which ...). The miss asks again after its chmod 700, since chmod succeeds for root on a directory it does not own. The question is answered by a process, one /bin/sh running /bin/ls -ldn and the shell's own test -O, not by the lstatOwnerModeSync and geteuid builtins, because those are refused on a wasm32 build and src/ is also the web compiler; see "Status after the owner checks" for what the probe trusts and how it fails closed. It closes the static plant this row describes, a root made and filled before the run; it is not atomic with the key read and the spawn, so it assumes the root's parent is not writable by the attacker, and an attacker who can write that parent is out of scope. Tests: CLI-7: a cache root its group can write is refused with the toolchain band, ... every user can write is refused, ... that is a symbolic link is refused, ... another user owns is refused (run as root only, a counted skip otherwise, since only root can give a directory away), CLI-7: a cache root another user owns is refused, run by any user with ... whose message says this user does not own it (a non-root run whose root is /run through a link to /, which reaches the owner refusal in CI: it fails with that refusal removed; a counted skip as root, or where /run is not a directory root owns), each with and the binary in it never ran, all failing on e1b8d0a, main when #546 opened; and CLI-7: a hit checks its root with no tool found on PATH, ... runs no ls or sh found on PATH, ... a PATH ls that says the root is private does not make it so.
CLI-8 Low src/run-cache.ts fnv1a64Hex The entry name is a 64-bit FNV-1a. A hit still compares the full key, so a collision alone only costs a relink, and CLI-5 closes the crash window, but two colliding programs started at the same moment can still interleave mv and the key write. Both programs are the victim's own runs. Fixed (#386). sha256Hex in src/run-cache.ts, FIPS 180-4 written in plain Nish because src/ reads nothing from std/, names every entry and fingerprints every runtime file in the key; fnv1a64Hex is gone. Checked against sha256sum at every padding boundary (0, 1, 55, 56, 57, 63, 64, 65, 119, 120, 128 bytes, a source file and 20 MB of random bytes). Test: CLI-8: the entry is named by the SHA-256 of its key in tests/nish/cli.ts, against std/crypto/sha256, which fails on the base (expected "f1e0…97a9", got "a4b2492e51146f75")
CLI-9 Low src/compile.ts:726 programOnPath, :798 packageRootCandidates at the base 083339e; the fix at 02d90e7: :1054 programOnPath, :1312 findPackage, :1280 considerRoot, :1121 distrustOf The package root is the parent of the compiler's directory, so a compiler kept at /tmp/x/nish trusts /tmp/scripts/build.sh, which any user can create. And programOnPath takes the first readable nish on PATH, where the shell took the first executable one, so the two can disagree. Fixed (#486). programOnPath takes the first entry whose nish is executable and not a directory, the one the shell ran, asked of one /bin/sh running test -x and test ! -d over every entry. The package root the compiler settles on (findPackage: the first candidate holding scripts/build.sh, or, with none, the one the library is read from) is refused when it is not owned by this user or by root, or every user can write it, by the same probe as CLI-7's, and the refusal names it and why (refused <dir>, which can be written by every user). This applies to every compile, not only --link: std/ comes from the same root, so when no trusted candidate holds the package and one was refused, every compile stops with exit 3 and NL0002 (compile: no package root this user may trust (...; refused <dir>, which ...)). A checkout or install that is world-writable (a 0777 bind mount, say) or owned by a user other than you and root (a container running as root over a checkout owned by your uid) compiled on the base and is refused now. A root its group can write is still trusted: whoever can write it can already replace the bin/ that holds the compiler, so refusing it protects nothing and would refuse every checkout made under a 0002 umask. Limits: only the root directory itself is probed, not scripts/, std/ or scripts/build.sh inside it, and write permission an ACL grants is not seen, since the probe reads the mode bits. Tests: CLI-9: --link does not run scripts/build.sh from a package root every user can write, CLI-9: a compile that imports nish/<module> from that root is refused with the toolchain band (on the base it compiled the root's std/), CLI-9: a directory called nish on PATH does not name the package, CLI-9: a nish on PATH with no execute bit does not name the package, CLI-9: the first executable one does, which fail on e1b8d0a, main when #546 opened. The negative test CLI-9: a package root only its group can write is still the package passes on both.
CLI-10 Low src/compile.ts:960 buildIntoCache The cache root's mkdir and its chmod 700 shared one condition and one message, and `
CLI-11 Medium (a new surface, by design) src/compile.ts cachedRuntime; src/run-cache.ts runtimeCacheRoot, runtimeCacheKey; scripts/build.sh --runtime-objects, --runtime-from Added after this audit with the runtime cache: compiled runtime objects in …/nish/runtime are linked into every binary the victim builds, so a planted, stale or partial entry would put the wrong code in every program. Mitigated as the run cache is: an absolute root only, chmod 700 and owner-checked on every link, a hit only on a byte-equal key, a scratch directory and one rename with the key written last. The record, its threat model and its tests are in The runtime cache (CLI-11) below.
CLI-12 Medium (a new surface, by design) src/compile.ts stageModules, keepModules; src/build-cache.ts moduleCacheRoot, moduleCacheKey; scripts/build.sh --module-objects Added after this audit with nish run's module cache: the object each module compiled to, in …/nish/module, is linked into a later debug build of any program whose module has the same IR, so a planted, stale or partial entry would put the wrong code in that program. Mitigated as the runtime cache is: an absolute root only, chmod 700 and owner-checked on every link, a hit only on a byte-equal key, the key emptied before the object moves in and written last. The record and its tests are in The module cache (CLI-12) below.
CLI-13 Low (a new surface, by design) src/compile.ts pgoProblem, main, linkProgram Added after this audit with --pgo-generate and --pgo-use: --link starts llvm-profdata from PATH on a file the user names, and an instrumented runtime is a different object under the same cache key. Mitigated; see Profile-guided links (CLI-13).

Properties verified

Property Pinned by
nish run never keeps a binary under /tmp: with neither HOME nor XDG_CACHE_HOME absolute, it refuses with exit 3 and NL0002. tests/run.js "nish run: with neither HOME nor XDG_CACHE_HOME set, an NL0002 refusal under --json, exit 3"; tests/nish/cli.ts "CLI-3: with no absolute HOME either, ..."
A hit is the stored key being byte-for-byte equal to this run's key, not the entry's name matching: an entry holding another key is relinked, and the binary it held never runs. tests/nish/cli.ts "an entry whose key is not this run's is relinked, not started", "and the binary it held never ran"
The key covers every input of the link: the compiler version, the binary's name, the profile, -g, --threads, CC, scripts/build.sh, all eighteen runtime C files (runtime.c, its sixteen halves and runtime-async.c; the seventeen elsewhere in the docs leaves out runtime-async.c) (eighteen since strings and number formatting moved from runtime.c into runtime-text.c; seventeen since runtime-async.c joined them; sixteen since runtime-chacha20.c did; fifteen since runtime-region.c did; fourteen since runtime-shm.c did; thirteen since runtime-sha256.c did; twelve since runtime-sandbox.c did; eleven since runtime-checksum.c did; ten since runtime-bytes.c did; nine since runtime-term.c did; eight since runtime-cross.c did; seven since runtime-aes.c did, at c29cd40) and nish.h (the only header they include), and every module's stem and IR, each IR framed by its length. scripts/build.sh reads no CFLAGS-style variable that could change a native binary unseen. What is not covered is the version of the compiler CC names; upgrading clang in place keeps the old binary until the next edit, which is the victim's own build and not a security question. tests/run.js "nish run: an edited script is rebuilt, ...", "nish run: each program has its own entry"
No argument reaches a shell as text. Every spawn is an argv through posix_spawnp; the one bash -c takes its operands as positional parameters; scripts/build.sh quotes every expansion. tests/nish/cli.ts "a --link path full of shell syntax links", "and none of it is run"
The binary a run starts is named by a path with a /, so posix_spawnp never searches PATH for it. A hit spawns nothing else. tests/run.js "nish run: a second run of the same program is a hit -- it runs with no C compiler and no shell on PATH"
No path the compiler builds can be read as an option: per-module IR is always below a directory (-o <dir>/, <exe>.modules/), the cache paths are absolute, and mv and rm take --. code review of planOutputs and buildIntoCache; CLI-3's tests
A failed link leaves no key and no scratch directory, so the next run links again rather than starting a binary that is not there. The scratch directory is inside the entry, so concurrent misses never share one. tests/run.js "nish run: a failed link is the link's report, exit 3, and leaves no key and no scratch directory behind"
A script's name cannot collide with the entry's own files. tests/nish/cli.ts CLI-1's checks
The cache root is private to its owner after any miss. tests/nish/cli.ts "CLI-4: the cache root is private to its owner"
The runtime cache (…/nish/runtime) links only objects whose stored key equals this link's byte for byte, from a root that is absolute, 0700 and this user's on every link, filled through a scratch directory and one rename with the key last; and the stage that compiles into it stops its compiles when it is stopped. CLI-11's tests, below
The module cache (…/nish/module) links only objects whose stored key equals this module's byte for byte, from a root that is absolute, 0700 and this user's on every link, with the key emptied before an object moves in and written after; the speed and size recipes never read it. CLI-12's tests, below

Status after the runtime stage

The runtime stage (runtime.md) merged after this record and changed what is open. The table keeps this stage's findings as they stood; this is what is true now.

  • CLI-6 is fixed in the runtime (RT-4): writeFileSync, appendFileSync and spawnSyncTo's two paths open with O_NOFOLLOW and refuse a symbolic link as the last component. The compiler is built by the last release, so the fix reached nish itself with the next release, 0.16.0; 0.15.0 and earlier still follow the link.
  • CLI-7 and CLI-9 have their primitives (RT-9): nish_lstat_owner_mode, nish_euid and nish_is_executable in runtime/runtime-host.c. src/ may call them only once a release declares them, so both stay open until the release after the runtime stage, and then need the change in src/compile.ts that uses them.
  • CLI-8 is unchanged. Since fixed by #425 (the next section).

Status after the owner builtins (#386, phase 1)

  • The RT-9 primitives are builtins: lstatOwnerModeSync(path): i64, geteuid(): i64 and isExecutableSync(path): boolean (LANGUAGE.md). src/ may call them once a release ships them (the rolling freeze), so CLI-7 and CLI-9 stay open for the release after this one: a hit that refuses a cache root it does not own or that is group- or world-writable, and a programOnPath that takes the first executable nish.
  • CLI-8 is fixed: the entry is named by a SHA-256 of its key (the row above).
  • scripts/bootstrap.sh no longer needs the working-directory fallback. Its stages live in <work>/bin/, and <work> holds scripts, runtime and std as symbolic links to the checkout by absolute path, so each stage finds the checkout through its own path. On the base, a --work outside the checkout failed at stage2 with --link: cannot find scripts/build.sh; it now builds and verifies. The . candidate in packageRootCandidates is still there, and can go with CLI-9's change.

Status after the owner checks (#486)

  • CLI-7 and CLI-9 are fixed (their rows above).

  • The checks are processes, not builtins. lstatOwnerModeSync, geteuid and isExecutableSync answer the same questions in-process, but the checker refuses all three on a wasm32 build, and src/ is also linked as the web compiler (web/), which Nish-0 cannot leave a call out of. So src/compile.ts asks /bin/sh, through spawnSyncTo, which links under wasi.

    Each owner-and-mode question is one /bin/sh that runs one /bin/ls -ldn (the type, the mode and the numeric owner, without following a link) and the shell's own test -O (the effective uid), so no id runs; the answer is the exit status. The PATH question is one /bin/sh that runs test -x and test ! -d over every entry.

  • What the probe trusts. /bin/sh and /bin/ls, by absolute path, both present on Linux and macOS; test, case and set are the shell's builtins. No PATH is searched, so an ls or sh earlier on it is never run (CLI-7: and runs no ls or sh found on PATH). Every path reaches the shell as an operand, never as script text, and set -f stops a * in it from being expanded.

  • It fails closed. A probe that cannot be started, whose ls fails for a path that exists, or whose answer does not have the shape of a mode string and a numeric uid, refuses the directory, with the same NL0002 and the reason could not be checked (the /bin/sh and /bin/ls probe of its owner and mode failed). A path that is not there is no answer: a missing cache root is made as before, and asked about again after its chmod 700. No test drives a failed probe, since that needs a broken /bin/ls.

  • The web compiler. In nish.wasm every spawn fails, so every probe would refuse. None runs there: the web host starts the compiler as a bare nish with no PATH, so there is no package-root candidate to ask about and no PATH to search, and nish run and --link are not reachable in a wasm build. A wasi host that does name the compiler by a path gets the fail-closed refusal, which is the right answer for a compiler that cannot check where its package is.

  • What it costs. One probe per nish run (two on a miss), one per compile for the package root, and one more when the compiler is found on PATH: the figures are in the change's Measured: trailer.

  • The ownership rule differs on purpose between the two. The cache root is this user's alone and is refused when its group can write it, because a run never makes it so: such a root predates CLI-4 or was changed by someone else, and rm -rf of it costs one relink. The package root may be root's, as every system install is, and only every-user write is refused.

  • The . candidate in packageRootCandidates is still there, now under the same check; removing it is left to a change of its own.

Doc corrections for the security-policy stage

  • docs/INSTALL.md, the nish run section: "It needs HOME or XDG_CACHE_HOME set" should say set to an absolute path (a relative one is now ignored), and should say that the cache root is made 0700 on the first run that builds into it.
  • docs/INSTALL.md and the contributor notes: a compiler outside its package no longer falls back to the working directory for scripts/build.sh and std/ unless the compiler itself is inside that directory. An install has to keep bin/nish beside scripts/, runtime/ and std/, which every documented install already does; CLI-9's advice (a parent directory only you can write) belongs beside it.
  • The supply-chain stage owns scripts/bootstrap.sh: its intermediate stage at build/selfhost/stage still finds the checkout only through the narrowed . fallback. Building that stage one level below the checkout's root, or passing the root explicitly, would let the fallback go entirely. (Done in #386 phase 1: see "Status after the owner builtins".)

The runtime cache (CLI-11)

Added with the runtime cache, against main at 53608c5; its anchors are function names in the commit that adds this section rather than line numbers. --link and every nish run link with the debug, speed or size profile used to compile the runtime's six translation units on every link. They are now compiled once per recipe into $XDG_CACHE_HOME/nish/runtime/<sha256 of the key>/ (runtimeCacheRoot, runtimeCacheKey and runtimeUnits in src/run-cache.ts; cachedRuntime in src/compile.ts; the --runtime-objects and --runtime-from modes of scripts/build.sh), and every later link of any program links those objects into its binary.

Id Severity Where Description Disposition
CLI-11 Medium (by design, as a new surface) src/compile.ts cachedRuntime; src/run-cache.ts runtimeCacheRoot, runtimeCacheKey Cached code is linked into every binary the victim builds, so the entry is a place to plant code that runs as them in every program, and a stale entry is a wrong runtime in every program. The threats are CLI-3's, CLI-4's, CLI-5's and CLI-8's, now against objects rather than one binary: a root another user or a checkout can write, an entry another recipe filled, a partial entry, a race between two links. Mitigated as the run cache is. The root is runCacheRoot's sibling and follows its rules: absolute XDG_CACHE_HOME or HOME only, never relative and never /tmp; with none, --link compiles the runtime itself rather than refusing a link that works without the cache. The root is made, chmod 700'd and its owner and mode checked on every link before anything is written in it (a hit reads only, but the key needs $CC --version, captured through a file in the root), and a root that cannot be made private, or is not this user's directory, is not used. The key holds the compiler's name and version, the profile, -g, --threads, CC, what $CC --version prints (the LLVM version LTO bitcode is tied to, and the target), the runtime's real directory (which -g writes into the objects), and the SHA-256 of scripts/build.sh, all fifteen runtime files and nish.h (fifteen since runtime-region.c joined them; fourteen since runtime-shm.c did; thirteen since runtime-sha256.c did; twelve since runtime-sandbox.c did; eleven since runtime-checksum.c did; ten since runtime-bytes.c did; nine since runtime-term.c did; eight since runtime-cross.c did; seven since runtime-aes.c did, at c29cd40); the entry is named by the SHA-256 of the key and a hit is the stored key equal to this one byte for byte. A miss compiles into a scratch directory inside the entry, empties the entry's key, moves the fifteen objects in with one mv (a rename each, inside the entry), and writes the key last, so a key on disk always stands beside fifteen whole objects of its recipe; an object is never deleted, only replaced whole, so two concurrent misses both succeed. build.sh links exactly the fifteen objects by name, never a glob, so a file planted beside them is never read. Tests (tests/run.js, each name prefixed "runtime cache: "): "a miss creates one entry, named by 64 hex digits, holding the fifteen objects and the key and nothing else", "the root is private to its owner (0700)", "a second link, of a different program, compiles no runtime source and links", "`nish run` links against the same entry, and compiles no runtime source either", "the debug entry's runtime.o is native code, with no thread-local arena and no DWARF", "<input> is an entry of its own, compiled with it, and its binary links and runs" for each of --threads, -g, --profile speed, --profile size and "a CC of another name", "<file> edited in place is an entry of its own, compiled again" for each of runtime.c, nish.h and scripts/build.sh, "an entry with no key and a missing object is compiled again, and the link runs", "an entry whose key is not this recipe's is compiled again, and its planted object is never linked", "two links on a cold cache at once both miss, both succeed, and both binaries run", "a relative XDG_CACHE_HOME is ignored, and the entry goes under $HOME/.cache/nish/runtime", "a symbolic link at the root's path is not used -- the link compiles the runtime itself and keeps nothing there", "with neither HOME nor XDG_CACHE_HOME, --link compiles the runtime itself and links", and "the miss compiled each of the fifteen runtime units once", "a copied package links, and a second link of it reuses its entry", "<program> under --profile <profile> prints what the uncached recipe's binary prints, and is its size" for each of examples/hello.ts, examples/argv.ts, examples/sets.ts, examples/wordcount.ts and examples/multi/main.ts under speed and size, "build.sh <mode> refuses <what> with exit 2" for each of --runtime-objects and --runtime-from against --profile wasm, --profile napi, --profile wasi and --pgo-generate, "build.sh --runtime-objects stopped by TERM stops every compile it started", and "build.sh --runtime-from stopped by TERM stops every compile it started".

Like a run's (CLI-7), every link asks who owns the root after the chmod, hits included, with the same /bin/sh and /bin/ls probe (distrustOf): chmod succeeds for root on any directory, so it alone proves nothing. A root that is not a directory this user owns, a symbolic link at its path included, or that its group or every user can write, is not used, and the link compiles the runtime itself. The objects are linked, never executed by the compiler.

The module cache (CLI-12)

Added with nish run's module cache (wp52-run-rebuilds.md); its anchors are function names in the commit that adds this section. A debug link used to compile every module of the program, unchanged or not. It now keeps each module's object in $XDG_CACHE_HOME/nish/module/<sha256 of the key>/<stem>.o beside its key (moduleCacheRoot and moduleCacheKey in src/build-cache.ts; stageModules and keepModules in src/compile.ts; the --module-objects mode of scripts/build.sh), and a later link of a module with the same IR under the same recipe links that object.

Id Severity Where Description Disposition
CLI-12 Medium (by design, as a new surface) src/compile.ts stageModules, keepModules; src/build-cache.ts moduleCacheRoot, moduleCacheKey; scripts/build.sh --module-objects Cached code is linked into later binaries, so an entry is a place to plant code that runs as the victim, and a stale or partial entry is a wrong module. The threats are CLI-11's, against one module's object rather than the runtime's: a root another user or a checkout can write, an entry another recipe or another module filled, a partial entry, a race between two links. Mitigated as the runtime cache is. The root is runCacheRoot's sibling and follows its rules (absolute XDG_CACHE_HOME or HOME only); with none, the link compiles every module, as it did before. The root is made, chmod 700'd and its owner and mode checked with distrustOf on every debug link before anything is read from it, and a root that fails is not used. The key is the runtime entry's key — the compiler's name and version, the profile and its flags, CC and what $CC --version prints, scripts/build.sh and the runtime's files — then the module's stem and the SHA-256 of its IR; the entry is named by the SHA-256 of the key, and a hit is the stored key equal to this one byte for byte (a key whose object a cleaner removed is dropped when its copy fails, and any part of the object that did copy is deleted so a truncated copy is never linked, so that link compiles the module and the next keeps it again, rather than a failed copy turning the cache off). The IR enters as a digest, so a hit rests on SHA-256's collision resistance, as the entry's name already does, and an entry holds a key of a few hundred bytes rather than a second copy of the module. A hit's object is copied beside the module's .ll in the link's own per-module directory, and build.sh links the object it finds at <dir>/<stem>.o there and compiles the module when there is none, so nothing outside that directory and the matched entries is read. A miss's object is moved in only after the link succeeds: the entry's key is emptied first and written after the mv, so a key on disk stands beside a whole object of its recipe, and two concurrent links both succeed (an object is replaced whole by a rename, never written in place). A key that cannot be written ends the process, as the run cache's does, so an old key never stands beside a new object. A binary named <stem>.o beside the modules turns the cache off for that link, so it is never linked over a module's object and kept as it. build.sh --module-objects refuses any profile but debug and any link without --runtime-from, with exit 2. Only the debug recipe uses the cache: speed and size compile to LTO bitcode and their binaries stay what build.sh links without a cache, and --link passes no IR and builds none. Tests (tests/run.js, each name prefixed "module cache: "): "the first run prints what the program says", "the first link keeps one entry per module it compiled, named by 64 hex digits and holding the object and its key", "the root is private to its owner (0700)", "a rebuild after an edit to one module compiles that module alone and links the others from the cache", "an entry whose stored key differs is not linked; the module is compiled and the entry rewritten", "an entry with an empty key is compiled again, not linked", "an entry whose object is missing is compiled and repaired by the next link, and the other modules still come from the cache", "a key holds the IR's SHA-256, not the IR", "--profile speed neither reads nor writes the module cache", "a change to the recipe (another CC, or -g) compiles every module again; the old recipe's entries still hit", "an edit that keeps the IR's length is a miss, because the key holds the IR's digest", "a root left group- and world-writable is made 0700 again before it is read", "scripts/build.sh refuses --module-objects outside the debug profile (exit 2)", and "scripts/build.sh refuses --module-objects without --runtime-from (exit 2)".

Nothing prunes the cache, as nothing prunes the run cache: it grows by one entry each time a module's IR changes. Deleting nish/module costs one full compile of every module and nothing else.

Added with nish --link --pgo-generate and --pgo-use <file.profdata> (docs/LANGUAGE.md, "Profile-guided builds"); its anchors are function names in the commit that adds this section.

Id Severity Where Description Disposition
CLI-13 Low (by design, as a new surface) src/compile.ts pgoProblem, startsWithMagic, main, linkProgram Three new things happen on a profile-guided link. nish reads the bytes of a file the user names. It starts llvm-profdata, found on PATH, on that file. And it compiles the runtime and SQLite with -fprofile-generate or -fprofile-use, which makes objects that differ from a plain link's under the same recipe. If those objects reached the runtime cache (CLI-11), every later plain link would carry the instrumentation, or another profile's layout. Mitigated. pgoProblem refuses the line first, before any file is read or process started, when the flags would do nothing: both flags, no --link, a profile other than speed or size, a non-host --target, or nish run, which refuses them as it refuses every link-only flag. The file is resolved with realpath, must be a regular file, and must begin with the indexed profile's eight magic bytes (startsWithMagic). A raw .profraw is told apart by its own magic, because llvm-profdata show accepts one with exit 0 while clang -fprofile-use refuses it. Only then is llvm-profdata started, through spawnSyncTo (no shell), with the fixed argv show <file> and its stdout to /dev/null; a non-zero exit is a usage refusal (exit 2). llvm-profdata and clang come from the same PATH, so this trusts nothing --link did not already trust; a missing llvm-profdata is exit 3 before anything compiles. The real path reaches build.sh as one argv element, which puts it in one quoted array element of the compile line (-fprofile-use=<path>), so a space or a shell character in the name is never split or run. linkProgram passes a profile-guided link no runtime or SQLite cache entries (new CachedObjects("", "", "")), so build.sh compiles every unit from source and the cache is neither read nor written; the module cache (CLI-12) serves debug links only, which refuse the flags. Tests (tests/run.js, each name prefixed "pgo: "): a refusal row with exit 2 and one line for each rule above, "nish run --pgo-generate is refused with exit 2 before anything runs", "--pgo-generate with no llvm-profdata on PATH is a toolchain failure (exit 3) naming it, and links nothing", "a profile with the indexed magic that llvm-profdata cannot read is refused with exit 2, and links nothing", "--pgo-generate puts -fprofile-generate on the program, every runtime unit, SQLite and the link", "--pgo-use with a relative profile puts its real path on every unit and the link, and the binary prints the same", "an instrumented link between two plain ones adds no runtime cache entry, and only it is instrumented", and "the plain links before and after the instrumented one are byte-identical".

The magic read loads the whole file the user named into memory. That file is the user's own, and the same file is the one clang then reads whole.