Builtins

No import is needed; builtins are resolved by name. The libc-backed ones may also be imported from a nish: module, which is the same builtin under a name that cannot be shadowed (see Builtin modules below). Effects (none, read, write) are what the call does to the purity of the enclosing function (ARCHITECTURE.md).

Builtin modules (nish:)

The builtins backed by the C runtime can be imported instead of reached for as globals. nish: is the only bare specifier the language accepts; but for nish:secret and nish:region, which have source behind them (Secrets, Regions), it resolves to no file, and the import renames a builtin rather than introducing one — the call checks through the same rule and emits the same IR as the global spelling, which tests/run.js pins by compiling tests/cases/io_nish_import and io_nish_import_global and comparing the two bodies.

Module For
nish:fs files and directories
nish:process processes, environment, clocks
nish:io standard output and error, panic
nish:secret secrets that are wiped
nish:region region allocation
nish:net sockets and the readiness loop
nish:term terminal input
nish:unsafe unchecked access, wrapping arithmetic
nish:async tasks: spawn, await, wait, cancel
nish:simd sixteen-byte vectors

nish:fs exports readFileSync, readFileSyncOrNull, readFileBytesSync, writeFileSync, appendFileSync, writeFileBytesSync, renameSync, unlinkSync, writeFileSyncMode, createFileExclusiveSync, mkdirSync, mkdirSyncMode, isDirectorySync, readdirSync, realpathSync, statMtimeSync.

nish:process exports exit (the global process.exit), getenv, spawnSync, spawnSyncTo, spawnPiped, spawnPipedWithin, spawn, spawnWithFds, kill, waitPid, availableParallelism, totalMemory, setNice, setIdlePriority, setCpuAffinity, monotonicNanos, wallClockNanos, signalFd, readSignal, argv, platform, arch.

nish:io exports write, writeError, writeBytes, panic.

nish:secret exports Secret, secret, expose, exposeWith, wipe. It is the one module with source behind it (Secrets).

nish:region exports Region, region, regionWith, enter, enterWith, free. It has source behind it too (Regions).

nish:net exports netAddress, netLocalPort, tcpListen, tcpListenFlags, tcpAccept, netRead, netWrite, netWrite2, netShutdown, netClose, netMss, netSetOption and its option ids NET_TCP_NODELAY, NET_TCP_QUICKACK, NET_SO_KEEPALIVE, NET_SO_SNDBUF, NET_SO_RCVBUF, tcpConnect, connectResult, udpBind, udpSendTo, udpRecvFrom, pollCreate, pollAdd, pollModify, pollRemove, pollWait, unixListen, unixListenMode, unixConnect, unixAccept, unixSocketPair, sendFd, recvFd, peerCredentials, netRingState and the constants NET_UNIX_UNLINK_STALE, NET_UNIX_STREAM, NET_UNIX_SEQPACKET, NET_LISTEN_RING. Each is a global too (nish:net).

nish:term exports readStdin, waitStdin, isTerminal, terminalSize, setRawMode, setTerminalRestore. Each is a global too (nish:term).

nish:unsafe exports uncheckedGet, uncheckedSet, wrappingAdd, wrappingSub, wrappingMul, readCString, readStruct, cErrno. They are reached only through the import (nish:unsafe).

nish:async exports spawn, awaitAll, sleep, readable, writable, readableWithin, writableWithin, taskId, cancel, cancelled, cancelAfter. They are reached only through the import (Async functions).

nish:simd exports the vector types u8x16, u16x8, i32x4, u32x4, f32x4, f64x2 and their constructors, splatU8x16 … splatF64x2, loadU8x16 … loadF64x2, store, extract, replace; the masks mask8x16 … mask64x2; the lane operations add … asF64x2, among them shl, shr, rotl and rotr. They are reached only through the import (nish:simd, lane operations).

import { readFileSync } from "nish:fs";
import { argv } from "nish:process";
import { write } from "nish:io";
  • The forms are the ones every other import allows: a named import, with as (tests/cases/io_nish_rename). Default, namespace, side-effect and type-only imports are refused under the same messages a relative specifier gets.
  • An unknown module is Unknown builtin module `nish:sqlite` (reject_nish_unknown_module); a name a module does not export is Module `nish:fs` has no export `statSync` (reject_nish_unknown_export). Both list what does exist.
  • An imported builtin cannot be shadowed, and that is the reason to prefer one. A global builtin is consulted only when no user function of that name is in scope, so declaring one silently replaces it; declaring one that collides with an import is `panic` is already declared in this module instead (reject_nish_import_clash).
  • A name keeps its kind. argv, platform and arch are values, and calling one is refused (reject_nish_property_call); the functions are not values: a builtin is not a capture-free top-level function of this program, so it cannot be a reference (reject_nish_builtin_value).
  • exit terminates control flow exactly as process.exit does, so a non-void function may end with it (io_nish_rename).
  • Diagnostics name the canonical builtin whatever the call site calls it: exit(1, 2) reports `process.exit` expects exactly 1 argument, which is the rule to look up.
  • Everything else stays global — Math.*, the width conversions, console.log / console.error, String.fromCharCode, Arena.*, Ok and Err. They lower to an LLVM intrinsic or a single instruction and cost no runtime at all, so they read as language rather than as library. String.fromBytes is global too, beside String.fromCharCode on the same String, though both make their answer with one nish_str_new. Date.now() and crypto.getRandomValues are global too, though each is a runtime call, because JavaScript spells them as globals and a program that runs under Node unchanged has to find them where Node has them (The host).

console

The reference for each of these, with its parameters, its answer and its capability, is its doc comment in runtime/nish.d.ts; the rules are here.

Function Takes Gives Effect
console.log string | number | i64 | f64 | boolean void write
console.error as console.log void write
write string void write
writeError string void write
writeBytes readonly u8[], i32, i32 i32 write
panic string void write

console.log(x: string | number | i64 | f64 | boolean): void writes x and a newline to stdout. It is allowed in statement position only, and takes exactly one argument.

  • Lowering: one write(2).
  • Tests: str_console_log, i64_basic; reject_console_log_noargs, reject_console_log_two_args, reject_console_log_as_value; arrays and objects are rejected (`console.log` accepts string, number, or boolean, got i32[] (CLI only)).

console.error(x): void is the same, on stderr: the stream a compiler's diagnostics belong on (WP14 B2). It has identical rules and identical messages under its own name.

  • Tests: io_streams; reject_console_error_type.

write(s: string): void writes s to stdout with no trailing newline and no conversion (a string only). It is allowed in statement position.

  • Tests: io_streams.

writeError(s: string): void is the same, on stderr.

  • Tests: io_streams.

writeBytes(bytes: readonly u8[], off: i32, len: i32): i32 writes the bytes of bytes[off..off+len) to stdout as they are. It answers len, or a negative errno if the stream refuses them, never a panic. A pipe whose reader has gone raises SIGPIPE natively, where Node answers -32 EPIPE. A range outside bytes panics with slice out of range: [off, off+len) of length n and exits 1 (a slice site), under Node too. It needs no capability, as for write. A wasm32 build has no stdout to reach and refuses it.

  • Tests: io_write_bytes, io_write_bytes_bounds; reject_write_bytes_arity, reject_write_bytes_type (NL2268), reject_write_bytes_wasm.

panic(message: string): void writes message and a newline to stderr, then exits 1: the same ending an out-of-range index has. It is allowed in statement position, and it terminates control flow like process.exit, so a non-void function may end with it (WP14 D1, WP16).

  • Tests: io_streams; reject_panic_value.

Numbers print as JavaScript's String(x): exact decimal for i32/i64, shortest round-trip digits for f64 (0.1, 1e+21, 1e-7, NaN, Infinity, -0 as 0) (tests/runtime-test.c, tests/cases/str_f64_mode, math_intrinsics). Booleans print true/false.

Reference: nish:io.

Math

The reference for each member of Math, with its parameters, its answer and its capability, is its doc comment in runtime/nish.d.ts; the rules are here.

Function Takes Gives Effect
Math.sqrt/floor/ceil/trunc/sin/cos/exp/log f64 f64 none
Math.tan f64 f64 none
Math.atan2, Math.hypot f64, f64 f64 none
Math.pow f64, f64 f64 none
Math.round f64 f64 none
Math.abs any numeric T T none
Math.min, Math.max T, T T none
Math.clz32 i32 | u32 integer<0, 32> none
Math.sign any numeric T T none
Math.imul two of i32 | u32 i32 none
Math.random nothing f64 write
Math.PI, Math.E — f64 none

Math.sqrt/floor/ceil/trunc/sin/cos/exp/log(x: f64): f64 take and answer an f64. floor, ceil and trunc have two paths on x86-64 (see Lowering), and the answers are the same bits on either path.

  • Lowering: LLVM intrinsics (llvm.sqrt.f64, ...), except floor, ceil and trunc in an x86-64 module that is not a wasm build: there they call the runtime's nish_floor, nish_ceil and nish_trunc, which run SSE4.1's roundsd on a CPU that has it and libm's routine on one that does not, chosen once before main (NISH_SIMD=scalar pins libm); with no --target that is an x86-64 host. On a page module (a wasm build with no WASI, or an --emit-dts compile, whose IR --profile wasm or wasm-host links with no libm), sin, cos, exp and log are calls to std/math.ts's sinF64, cosF64, expF64 and logF64 instead, fdlibm's, copied in like tan, within one ulp of V8; --profile wasi keeps the intrinsics. Under --deterministic they are those calls on every target, --emit-ts included.
  • Tests: math_intrinsics, math_rounding, wasm_math_exp, wasm_math_std; reject_wasm_math_f32 (`Math.sin` requires an f64 argument, got f32); reject_math_i32 (requires an f64 argument, got i32 (use --number-mode f64 or toF64(x))).

Math.tan(x: f64): f64 is exact near every multiple of π/2 and within one ulp.

  • Lowering: a call to std/math.ts's tanF64, fdlibm's, copied into the module as internal @nish.tanF64, so no target calls libm.
  • Tests: math_tan, math_std_edges.

Math.atan2(y: f64, x: f64): f64, Math.hypot(x: f64, y: f64): f64 follow ECMAScript's special cases (atan2(0, -0) is π, atan2(-0, -0) is −π, hypot(Infinity, NaN) is Infinity). Math.hypot takes exactly two arguments, not JavaScript's any number.

  • Lowering: calls to std/math.ts's atan2F64 and hypotF64, copied in like tan.
  • Tests: math_atan2, math_hypot; reject_math_atan2_f32 (`Math.atan2` requires an f64 argument, got f32 (use --number-mode f64 or toF64(x))), reject_math_hypot_arity (`Math.hypot` expects exactly 2 arguments, got 3).

Math.pow(x: f64, y: f64): f64 gives ECMAScript's answer where it and C99 differ: pow(x, NaN) and pow(±1, ±Infinity) are NaN (C gives 1); pow(x, ±0) is 1, pow(NaN, 0) is 1, and every other special case agrees with C.

  • Lowering: llvm.pow.f64 plus a select for the cases where ECMAScript and C99 differ. On a page module it is a call to std/math.ts's powF64, copied in like tan, which answers ECMAScript's cases itself, within one ulp of V8.
  • Tests: math_intrinsics, wasm_math_std; tests/differential/corpus/f64_pow_spec; reject_wasm_math_arity (`Math.pow` expects exactly 2 arguments, got 1).

Math.round(x: f64): f64 is JavaScript's round-half-up, and keeps x's sign: 2.5 -> 3, -2.5 -> -2, 0.49999999999999994 -> 0, and a value in [-0.5, 0) or -0 itself -> -0, as in JavaScript; NaN -> NaN.

  • Lowering: Math.floor's floor + compare + select, then llvm.copysign.f64 puts x's sign back.
  • Tests: math_intrinsics, math_round_negzero, math_rounding.

Math.abs(x: T): T for any numeric T answers the magnitude, wrapping for INT_MIN. On an unsigned type the value is its own magnitude.

  • Lowering: llvm.abs.* (wrapping for INT_MIN) / llvm.fabs.f32 / llvm.fabs.f64; on an unsigned type, no instruction at all.
  • Tests: math_i32, math_intrinsics; i64, f32 and the unsigned widths (CLI only).

Math.min(a: T, b: T): T, Math.max(a: T, b: T): T take exactly two operands of one numeric type. On floats they give JavaScript's answer: a NaN operand gives NaN and -0 is below +0 (Math.min(0, -0) is -0).

  • Lowering: signed integers: llvm.smin/smax; unsigned: llvm.umin/umax. A native float build branches as clang compiles the C a programmer writes for it: fcmp uno to a quiet NaN, else fcmp olt/ogt and a select, and on fcmp oeq the operand whose sign bit is set (min) or clear (max), the three joined by a phi; x86 has no instruction for llvm.minimum, whose expansion measured 25–55% slower in bench/math. A wasm build keeps llvm.minimum/maximum.<f32|f64>, which are wasm's f64.min/f64.max, so no libm is linked for them.
  • Tests: math_i32, math_intrinsics, math_minmax_float, math_minmax_nan; reject_math_min_arity (`Math.min` expects exactly 2 arguments, got 1).

Math.clz32(x: i32 | u32): integer<0, 32> answers the leading zero bits of x's 32 bits, JavaScript's answer (an i32 and a u32 are the bits ToUint32 gives), so 0 answers 32. The answer is ranged (integer<Lo, Hi>, see Types), so 31 - Math.clz32(word & (0 - word)), the index of a word's lowest set bit, needs no overflow check. Any other operand type is refused rather than converted.

  • Lowering: llvm.ctlz.i32 with i1 false.
  • Tests: math_clz32; reject_math_clz32_type (`Math.clz32` expects an i32 or a u32, got f64 (convert with toI32 or toU32)).

Math.sign(x: T): T for any numeric T answers -1, 0 or 1 in x's own type (a ranged x answers its base type). A float -0 and NaN come back as JavaScript gives them.

  • Lowering: a float answers llvm.copysign(1.0, x) unless fcmp ueq x, 0.0, when it answers x itself; a signed integer is zext (x > 0) - zext (x < 0), an unsigned one zext (x != 0).
  • Tests: math_sign; reject_math_sign_type (`Math.sign` expects a number, got string).

Math.imul(a: i32 | u32, b: i32 | u32): i32 answers the low 32 bits of the product, read signed, with no overflow check, which is JavaScript's answer for any operands an i32 or a u32 holds (Math.imul(0x7fffffff, 2) is -2). Any other operand type is refused rather than converted, for Math.clz32's reason.

  • Lowering: one wrapping mul i32.
  • Tests: math_imul; reject_math_imul_f64 (`Math.imul` expects an i32 or a u32, got f64 (convert with toI32 or toU32)).

Math.random(): f64 answers 53 random bits in [0, 1), seeded from time and pid.

  • Lowering: xorshift64* in the runtime (nish_random).
  • Tests: math_random.

Math.PI: f64, Math.E: f64 are constants, also in i32 mode.

  • Tests: math_i32, math_intrinsics.

The f64-only functions reject i32/i64 arguments; number in i32 mode is an i32, so write Math.sqrt(toF64(n)) or use f64 mode. A numeric literal argument is typed f64 by context (Math.sqrt(2) works in both modes).

Numeric conversions

The reference for each conversion, with its parameter and its answer, is its doc comment in runtime/nish.d.ts, on the globals page (Number under NumberConstructor); the rules are here.

Function Takes Gives
toI32 any number i32
toI64 any number i64
toU8, toU16, toU32, toU64 any number that unsigned type
same width, other signedness any integer the other integer
toF32 any number f32
toF64 any number f64
f64ToBits f64 i64
f32ToBits f32 i32
bitsToF64 i64 f64
Number string | i32 | i64 | f64 | boolean f64
parseInt string i32
parseFloat string f64

toI32(x): i32 at narrower or equal width wraps. A float source (f32 or f64) saturates: NaN -> 0, and out of range clamps. On an i32 it is the identity.

  • Lowering: narrower or equal width: trunc (wraps); wider signed source: sext, wider unsigned source: zext; a float source: saturating llvm.fptosi.sat.i32.<f32|f64>; i32: identity.
  • Tests: conversions, u_conv_roundtrip; reject_toi32_string.

toI64(x): i64 saturates from an f64, and on an i64 it is the identity.

  • Lowering: signed source: sign-extend; unsigned source: zext; f64: saturating; i64: identity.
  • Tests: conversions, u_conv_widths.

toU8, toU16, toU32, toU64 widen a signed source by its sign, so toU64(-1: i32) is 2^64 - 1, as -1i32 as u64 is in Rust. Narrowing truncates. A float source saturates, clamped to 0 .. 2^bits-1, so a negative value is 0.

  • Lowering: widening: zext from an unsigned source, sext from a signed one; narrowing: trunc; a float source: saturating llvm.fptoui.sat, whose clamp is 0 .. 2^bits-1.
  • Tests: u_conv_widths, u_conv_f64, u_conv_roundtrip; reject_u_conversion_arg.

any integer -> an integer of the same width and other signedness keeps the bits as they are (toU32(i) on an i32, toI64(q) on a u64).

  • Lowering: no instruction at all: signedness is not part of the LLVM type and the bits do not move.
  • Tests: u_conv_same_width.

toF32(x): f32 rounds an f64 to nearest, and on an f32 it is the identity.

  • Lowering: signed integers: sitofp; unsigned integers: uitofp; f64: fptrunc (rounds to nearest); f32: identity.
  • Tests: f32_conversions, f32_constant.

toF64(x): f64 widens an f32 exactly, and on an f64 it is the identity.

  • Lowering: signed integers: sitofp; unsigned integers: uitofp; f32: fpext (exact); f64: identity.
  • Tests: conversions, u_conv_f64, f32_conversions.

f64ToBits(x: f64): i64 reinterprets the 64 bits and never converts the value: toI64(1.5) is 1, f64ToBits(1.5) is 0x3FF8000000000000. Under --deterministic a NaN gives 0x7FF8000000000000 (quietBitsF64).

  • Lowering: one bitcast, so a function that only reinterprets bits stays readnone.
  • Tests: conv_f64_bits; reject_f64tobits_type, reject_f64tobits_arity.

f32ToBits(x: f32): i32 is the same for an f32's 32 bits, signed as an i32: f32ToBits(1.5) is 0x3FC00000, and f32ToBits(-0) is -2147483648. Under --deterministic a NaN gives 0x7FC00000 (quietBitsF32).

  • Lowering: one bitcast float to i32; under Node, runtime/nish.mjs supplies it from a DataView.
  • Tests: f32_to_bits; reject_f32_to_bits_f64 (`f32ToBits` expects an argument of type f32, got f64).

bitsToF64(b: i64): f64 is the inverse. bitsToF64(f64ToBits(x)) === x for every x except NaN, which is not === to itself.

  • Tests: conv_f64_bits; reject_bitstof64_type.

Number(x: string | i32 | i64 | f64 | boolean): f64 parses a string with JavaScript's Number(s) rules (below). A boolean converts with true as 1, and an f64 is the identity. A numeric literal argument is typed f64 (Number(2.5) works in i32 mode). Under --deterministic a string goes to std/math.ts's numberExact instead, which reads a 0x integer the same on every target.

  • Lowering: i32/i64: sitofp; boolean: uitofp; f64: identity.
  • Tests: parse_numbers; reject_number_array (`Number` expects a string, number, or boolean, got i32[]).

parseInt(s: string): i32 reads base 10 only: ASCII whitespace, an optional sign, then digits; it stops at the first other character. Deviations from JavaScript: no NaN in an i32, so no digits give 0 (parseInt("abc"), parseInt("")); values beyond the i32 range saturate exactly like toI32 (parseInt("99999999999") is 2147483647); a 0x prefix is not hexadecimal (parseInt("0x10") is 0). It is always i32, also under --number-mode f64 (use Number there).

  • Lowering: strtoll.
  • Tests: parse_numbers; reject_parseint_number (`parseInt` expects a string, got i32).

parseFloat(s: string): f64 reads ASCII whitespace, then the longest decimal literal ([sign] digits [. digits] [e [sign] digits], . digits, or [sign] Infinity), correctly rounded. It answers NaN when there is none (parseFloat("abc"), "", "."); parseFloat("3.14xyz") is 3.14, "1e" is 1. Deviation: a 0x prefix is read as hexadecimal like strtod (parseFloat("0x1A") is 26; JavaScript stops at the x and gives 0). Under --deterministic a string goes to std/math.ts's parseFloatExact instead, which reads a 0x integer the same on every target.

  • Lowering: strtod.
  • Tests: parse_numbers.

The string parsers accept only ASCII whitespace ( \t\n\v\f\r), not the Unicode spaces JavaScript also trims. Number(s) trims it on both sides and then requires the whole string to be one literal: Number("") and Number(" ") are 0, Number(" 7.5 ") is 7.5, Number("12px"), Number("1e"), Number("Infinityx"), Number("nan") and Number("inf") are NaN, Number("0x1A") is 26 (as in JavaScript; hex floats such as 0x1p3 are also accepted by strtod, which JavaScript rejects), and the 0b/0o prefixes are NaN (JavaScript reads them). All three lower to nish_parse_number(s, mode) in the runtime (mode 0 parseFloat, 1 Number, 2 parseInt, the last followed by llvm.fptosi.sat.i32.f64); their effect is write (strtod/strtoll may set errno), so a function that parses is never readonly (wp7-runtime.md).

nish:unsafe: unchecked access and defined wrapping

Two things the language otherwise never does are each a decision about one site: reading or writing an element without the bounds check, and letting a signed + - * wrap. nish:unsafe holds the five functions that make them, two, readCString and readStruct, that read a C string or a C struct through a pointer nothing proves, and an eighth, cErrno, that reads the errno a C call left. A program reaches them only through an import, so every module that gives up a check or defines a wrap says so at its top and every site that does is a call a reader can find.

Reference: nish:unsafe.

Signature Lowering Effect Test
uncheckedGet(xs: T[], i: i32): T the GEP and load xs[i] lowers to, with no len load, no compare and no branch to the bounds panic read unsafe_get_set; reject_unsafe_arity, reject_unsafe_type
uncheckedSet(xs: T[], i: i32, v: T): void the GEP and store xs[i] = v lowers to, likewise unchecked; a statement write unsafe_get_set; reject_unsafe_arity, reject_unsafe_type, reject_unsafe_par_shared_write
wrappingAdd(a: T, b: T): T add, never nsw and never checked none unsafe_wrapping; reject_unsafe_arity, reject_unsafe_type
wrappingSub(a: T, b: T): T sub, never nsw and never checked none unsafe_wrapping; reject_unsafe_type
wrappingMul(a: T, b: T): T mul, never nsw and never checked none unsafe_wrapping; reject_unsafe_type
readCString(p: CPtr): string a NULL test that panics ffi-null, then a call to @nish.ffi.str, the scan-and-copy a string result of a declare function uses (readCString) read; allocates; may panic ffi_owned_string, panics_ffi_read_cstring_null; reject_unsafe_read_cstring, reject_ffi_owned_string_arena
readStruct(p: CPtr): T a NULL test that panics ffi-null, then nish_alloc_struct of the layout's size and one llvm.memcpy into it, T the declare class the context names (readStruct) read; allocates; may panic ffi_struct_read, panics_ffi_read_struct_null; reject_unsafe_read_struct_context, _null, _type_args
cErrno(): i32 a load i32 of @nish_errno_slot; naming it brackets every declare function call with a zero of errno and a save into the slot (cErrno) read ffi_errno, ffi_errno_threads; reject_ffi_errno_arg, reject_ffi_errno_import, reject_ffi_errno_wasm, reject_ffi_errno_global
  • The import is required. The seven are declared as globals in runtime/nish.d.ts, as every nish: export is, so tsc reads either spelling; the compiler refuses the global one with `uncheckedGet` is a `nish:unsafe` function and is called only through an import of it (NL2456, reject_unsafe_no_import). An import may rename (as) and cannot be shadowed, as every builtin import (above). A user function of the same name, with no import, is the user's function, as with every builtin.
  • uncheckedGet and uncheckedSet take an array of numbers: i32, i64, u8, u16, u32, u64, f32 or f64 elements, and nothing else is `uncheckedGet` takes an array of numbers (i32, i64, u8, u16, u32, u64, f32 or f64), got string[] (NL2455). A number is copied in and out and held by nothing, so the store needs none of the escape and inline-array reasoning xs[i] = v carries. The index is an i32, and an f64 one is `uncheckedGet` expects an argument of type i32, got f64; the value of a store is the element type. A readonly array may be read and not written (Cannot `uncheckedSet` through readonly u8[]). An index outside [0, xs.length) is undefined behaviour, which is the whole of the trade. The receiver is an argument, so an array field handed to either is a value and is not stored inline (Fixed-length array fields are stored inline); a store is a write of the receiver for every rule that asks, the parallel-body rule included (reject_unsafe_par_shared_write).
  • wrappingAdd, wrappingSub and wrappingMul take two operands of one type, i32 or i64, which is also the answer: the signed words are the ones whose operators are checked, and an unsigned operator already wraps. `wrappingMul` takes i32 or i64 operands (convert with toI32 or toI64), got u32 is NL2454, and two widths are `wrappingAdd` needs every operand of one type, got i32 and i64 (NL2400, as ctSelect's). A bare literal takes the other operand's type, wherever it stands. The answer is the two's-complement wrap at that width, defined for every pair (unsafe_wrapping). In a module constant's initialiser the call folds, wrapping, where 2147483647 + 1 is refused (Module constants).
  • Under Node runtime/nish.mjs answers the import, and the five are xs[i], xs[i] = v, (a + b) | 0, (a - b) | 0 and Math.imul on an i32, and BigInt.asIntN(64, …) on the BigInt an i64 is there; a program that stays in range prints the same as natively (unsafe_get_set and unsafe_wrapping are run both ways by tests/run.js). readCString and readStruct have no Node form: a CPtr comes only from a declare function, which Node cannot call, so a program that reads one is native only, as every FFI program is outside the differential oracle (docs/wp27-ffi.md §5).
  • Every import and every call is a recorded fact. --emit-checked prints, per module, an unsafe import <line>:<col> <name> from "nish:unsafe" line for each name imported and an unsafe call <line>:<col> <name> (node <id>) line for each site, so a report of where a program opts out of a check reads them rather than the source (tests/self/goldens/checked/, one file per module).
  • The import is the opt-in the no-panic scope takes. Under --deny-panics or "nish".noPanic, uncheckedGet and uncheckedSet are allowed, listed as unchecked sites with "allowed": true, because they cannot panic and the import has already said, at the top of the module, that an index out of range is the author's to rule out (The no-panic scope, deny_panics_unsafe). readCString and readStruct are not: each tests its pointer for NULL, an ffi-null site no proof removes (panics_ffi_read_cstring_null, tests/ffi-string.js; panics_ffi_read_struct_null, tests/ffi-struct.js), so --deny-panics refuses a call to either in its scope. The wrapping* three are not sites at all: they are defined for every pair.
  • To the bounds proof uncheckedGet and uncheckedSet are the load and the store xs[i] and xs[i] = v are, less the check: a call moves no length, so in uncheckedGet(a, at[0]) + b[at[1]] with at an array of two, at[1] stays proven after the call. Neither records a passed check, because neither has one: a checked xs[i] after either keeps its check, and a resize in the value stored still takes the length away (tests/cases/arr_bounds_unchecked_kept).
  • There is no flag that turns the checks off. --unchecked-indexing and --wrapping were removed before 1.0, and the CLI refuses each with exit 2 and a line naming its replacement (`--wrapping` is gone, tests/nish/cli.ts): an opt-out is written at the site, where --emit-checked and the no-panic scope can see it.

nish:simd: sixteen-byte vectors

A vector is sixteen bytes of lanes of one number type, held and passed as one value: an LLVM <N x T> in a register, a local, a field or an array slot. The module names six of them, and the calls that make one, read one out of an array, write one back and take one lane out or put one in. Nothing here names a target: the IR is insertelement, shufflevector, extractelement and a vector load or store, and LLVM picks the instructions (tests/run.js's simd section greps every simd_*.ll for llvm.x86. and llvm.aarch64.). The lane operations, the masks a compare answers and the shifts are in the next section.

The six are u8x16 (<16 x i8>), u16x8 (<8 x i16>), i32x4 and u32x4 (<4 x i32>), f32x4 (<4 x float>) and f64x2 (<2 x double>). Each has a constructor of one argument per lane, a splat (splatU8x16 … splatF64x2) and a load (loadU8x16 … loadF64x2) from an array of its lane type; the table below spells each family with i32x4.

Reference: nish:simd.

Function Takes Gives
i32x4 one i32 per lane i32x4
splatI32x4 one i32 i32x4
loadI32x4 readonly i32[], i32 i32x4
store i32x4, i32[], i32 void
extract i32x4, constant k one lane
replace i32x4, constant k, lane i32x4

i32x4(a, b, c, d), one argument per lane makes a vector of its arguments.

  • Lowering: one insertelement per lane into undef, which opt folds to a constant vector when every lane is one.
  • Tests: simd_lanes; reject_simd_arity, reject_simd_lane_type.

splatI32x4(x) makes a vector with x in every lane.

  • Lowering: one insertelement into lane 0 and a shufflevector whose mask is zeroinitializer.
  • Tests: simd_lanes.

loadI32x4(xs, i) reads a vector out of xs at i, after one bounds check for the whole vector, i < len and i + N <= len.

  • Lowering: the bounds check branches to nish_panic_index(i, len); then the element's address bitcast to <N x T>* and one load aligned to the lane, with the !alias.scope, !noalias and !tbaa of a scalar element access of the lane type. The check is left out where the bounds proof (src/bounds.ts, judgeVectorAccess) already knows every lane is in range: i non-negative under a guard such as i <= n - N or i + N - 1 < n on the array's length, or a literal index into an array of known length. A guard one lane short proves nothing, and the access keeps its check.
  • Tests: simd_access, simd_access_proven, simd_access_alias, simd_access_oob, simd_access_oob_first, simd_access_oob_negative, simd_access_oob_guard; reject_simd_lane_type.

store(v, xs, i) writes v back into xs at i after the same check. It is a statement.

  • Lowering: the same check, then one store.
  • Tests: simd_access, simd_store_oob; reject_simd_lane_type, reject_simd_arity.

extract(v, k) takes lane k out of v; k is a constant.

  • Lowering: extractelement at the constant k.
  • Tests: simd_lanes; reject_simd_lane_index.

replace(v, k, x) puts x in lane k, a constant, and answers a new vector: v is unchanged.

  • Lowering: insertelement at the constant k.

  • Tests: simd_lanes, simd_values; reject_simd_lane_index.

  • The import is required. The six types and every call are declared in runtime/nish.d.ts under declare module "nish:simd" only, and a module that spells one without the import is refused with `i32x4` is a `nish:simd` name and is used only through an import of it: add `import { i32x4 } from "nish:simd"` (NL2578, reject_simd_no_import). An import may rename (as) and cannot be shadowed, as every builtin import.

  • A load or store takes an array of the lane type, exactly: a u8x16 is loaded from a u8[] and nothing else, so the sixteen bytes it reads are the sixteen elements the index names, and an f64[] handed to loadF32x4 is `loadF32x4` takes an array whose elements are the vector's lane type, f32[], got f64[] (NL2579). A readonly array may be loaded from and not stored to (Cannot `store` through readonly u8[]). The index is an i32; the check is one compare pair for the whole vector, so a load whose last lane is past the end panics before it reads, naming a lane past the end (the first when it is out of range, else the last) and the length, as xs[i] does (simd_access_oob, run by tests/run.js natively and under Node). The access is aligned to the lane, not to sixteen: an array's elements are where the arena put them.

  • A lane index is an integer literal in range. extract(v, k) and replace(v, k, x) take k as a literal from 0 to one less than the lane count, because a constant lane is one instruction and needs no check at run time: a variable is `extract` takes a lane index that is an integer literal, from 0 to 3: i32x4 has 4 lanes (NL2580) and a literal past the end is `extract` lane index 4 is out of range for i32x4, whose lanes are 0 to 3 (NL2583, reject_simd_lane_index). The first argument must be a vector (`extract` takes a `nish:simd` vector as its first argument, got i32, NL2582), and a lane given to a constructor, a splat or replace is the lane type, as any argument is (reject_simd_lane_type).

  • A vector is a value. It is copied by assignment, passed and returned in registers, and held in a local, a class field or an array (simd_values). In an object or an array it is sixteen bytes aligned to sixteen, the layout clang gives __attribute__((vector_size(16))), which tests/layout/simd.c asserts against the IR's struct and reads lane by lane; an array of vectors steps by sixteen. The arena hands out objects aligned to eight, so a field access is align 8.

  • A vector has no operators, and no comparison. +, unary -, a truthiness test, a template hole and console.log of a whole vector or a mask are refused with the usual operand-type diagnostics (reject_simd_operators); the lane-wise forms are the functions of the next section. ===, !==, < and the rest, and the === inside an array's indexOf, are `===` does not compare `nish:simd` vectors or masks (got i32x4): compare lane by lane with `eq`, `lt` and the rest, which answer a mask, and ask it `all` or `any` (NL2581, reject_simd_compare): a lane-wise compare answers a mask, not a boolean, and under Node a vector is a typed array whose === is identity. A vector is not a Map key (reject_simd_map_key), not Secret (reject_simd_secret) and not nullable (reject_simd_nullable).

  • Under Node runtime/nish.mjs resolves the import to runtime/core.mjs, where a vector is a typed array whose element type does the lane's arithmetic: a u8 lane wraps through Uint8Array, an f32 lane rounds through Float32Array. --emit-ts writes the import as nish/runtime's. A load or a store checks the whole vector and panics with the native words. tests/run.js's simd section runs every simd_* program with a .out natively, under --profile wasi (where a sysroot is installed) and under Node, and holds the three to one output.

  • A vector does not cross the interop boundary yet. An exported function whose parameter or result holds a vector at any depth (the vector itself, an array of them, a Result with a vector arm, an array of a class that holds one) is left out of --emit-header, --emit-napi and the --emit-dts typings and wasm loader, with a comment naming the type, as every type the sidecars cannot spell is. A class that holds a vector anywhere (a field, an element, a Result arm, another class it points at) is opaque in the header (typedef struct Name Name;), as a Map or Secret holder is: C has no portable vector type to lay it out with. A bare pointer to such a class still crosses, as an opaque struct Name *.

nish:simd: lane operations and masks

Every operation on a vector is a free function of nish:simd, because a TypeScript program cannot overload +. The operand types pick the lowering, and each lowers to LLVM vector IR or a target-independent llvm.* intrinsic: no golden holds llvm.x86. or llvm.aarch64. (wp38-simd.md §3.2).

Reference: nish:simd.

Signature Lowering Test
add, sub, mul, div(a, b) on f32x4, f64x2 fadd, fsub, fmul, fdiv, no fast-math flags simd_ops_float; reject_simd_op_operands
neg, abs, sqrt(a) on f32x4, f64x2 fneg, llvm.fabs, llvm.sqrt simd_ops_float; reject_simd_op_operands
addWrapping, subWrapping, mulWrapping(a, b) on u8x16, u16x8, i32x4, u32x4 add, sub, mul with no nsw or nuw: each lane wraps simd_ops_int; reject_simd_op_operands
shl(v, n), shr(v, n), n a literal below the lane width shl; shr is lshr on an unsigned lane and ashr on i32x4 simd_shifts; reject_simd_shift_count
rotl(v, n), rotr(v, n) on the same four llvm.fshl, llvm.fshr with v as both halves simd_shifts, simd_chacha_quarter; reject_simd_shift_count
and, or, xor(a, b), not(a) on an integer vector or a mask and, or, xor; not is xor with all-ones simd_ops_int
min, max(a, b) on every vector llvm.umin/smin/umax/smax; floats llvm.minimum/llvm.maximum, NaN-propagating with -0 below +0, as Math.min simd_ops_int, simd_ops_float
eq, ne, lt, le, gt, ge(a, b) icmp (unsigned on u* lanes) or ordered fcmp (une for ne), answering the lane width's mask simd_ops_int, simd_ops_float; reject_simd_op_operands
select(m, a, b) select: lane k of a where lane k of m is set, else of b simd_ops_int; reject_simd_op_operands
any(m), all(m) llvm.vector.reduce.or, .and on <N x i1> simd_ops_int
bitmask(m), an i32 with bit k lane k bitcast <N x i1> to iN, zext simd_ops_int
sum(v) on u8x16, u16x8 (an i32, exact) and f32x4, f64x2 (the lane type, lanes added 0 to N-1) zext and llvm.vector.reduce.add; floats llvm.vector.reduce.fadd without reassoc simd_ops_int, simd_ops_float; reject_simd_sum
sumWrapping(v) on i32x4 (an i32) and u32x4 (a u32) llvm.vector.reduce.add, wrapping simd_ops_int; reject_simd_sum
shuffle(a, b, i0, …), one literal index per lane, from 0 to 2N-1 shufflevector simd_ops_int; reject_simd_shuffle
asU8x16(v) … asF64x2(v) bitcast: the same sixteen bytes read as other lanes simd_ops_int
  • Every operand is one vector type. Two types are `min` takes operands of one type, got i32x4 and u32x4; reinterpret the lanes with `asU8x16` and the rest, a scalar where a vector goes is `eq` takes a `nish:simd` vector, got i32 (NL2638), and a vector a family does not take names the family: `add` is float lane arithmetic and takes an `f32x4` or an `f64x2`, got i32x4: an integer lane can overflow, so its form is `addWrapping` (NL2623, reject_simd_op_operands). Integer lane arithmetic wraps, and its name says so: making a lane trap as scalar + does would cost the speed the type is for.
  • A shift count is a literal below the lane width. shl, shr, rotl and rotr take the count as an integer literal from 0 to 7, 15 or 31, so each is one instruction with a splat constant and nothing is shifted by a variable: a variable count is `shl` takes a count that is an integer literal below the lane width, from 0 to 31: a lane of u32x4 is 32 bits (NL2630), and the width or more is `rotl` count 32 is not below the lane width of u32x4, 32 bits: a shift by the width or more moves every bit out, and a rotate by it is a rotate by 0 (NL2631, reject_simd_shift_count). LLVM gives a shift by the width no meaning, and a rotate by it is a rotate by 0. shr is logical on u8x16, u16x8 and u32x4 and arithmetic (it copies the sign) on i32x4; a rotate is the same on either. They are what ChaCha20, SHA-256 and other add-rotate-xor code is built from (simd_chacha_quarter).
  • A mask is a compare's answer, held only as a value. eq and the rest on two vectors answer mask8x16, mask16x8, mask32x4 or mask64x2 by lane width, an <N x i1>; there is no constructor (`mask32x4` is a mask type and has no constructor: a mask is what a lane-wise compare (`eq`, `lt` and the rest) answers, NL2621). select takes the mask of its vectors' width (`select` between two i32x4 takes the mask their compare answers, mask32x4, got mask64x2, NL2629). A mask lives in a local, a parameter or the return of a function the module keeps, and is `mask32x4` cannot be a field: a `nish:simd` mask is a value held only in a local, a parameter or a return of a function this module keeps; store the vectors it selects, or its `bitmask` (NL2622) as a field, an array element, a Result arm, a type argument, a method's parameter or result, or an exported signature (reject_simd_mask_storage, reject_simd_mask_in_body): it has no size in memory that all three targets share.
  • A sum says whether it wraps. sum of u8x16 or u16x8 adds in an i32 and cannot overflow; sum of i32x4 or u32x4 can, and is `sum` of i32x4 can overflow its lane type, so its total is `sumWrapping`, which wraps as the lanes do (NL2636, reject_simd_sum). A float sum adds lane 0 first and lane N-1 last, so it is the same bits on every target.
  • A shuffle names every lane. shuffle(a, b, …) takes one literal per lane of the answer, each from 0 to 2N-1, where N and up are b's lanes; a wrong count, a variable or an index past the end is refused (NL2632 to NL2635, reject_simd_shuffle).
  • Under Node each operation is a function of runtime/core.mjs over the typed arrays: an integer lane wraps through its array type, a float lane rounds through Float32Array, a mask is an array of booleans, and a rotate is two shifts and an or. tests/run.js's simd section holds the native, WASI and Node outputs to one.

Constant time: ctSelect and ctEq

Crypto code must not branch on a secret or index memory by one, and LLVM promises neither: a select it can see through may become a branch, and a mask it can prove is a boolean may become a select. These two builtins are how a program selects and compares on a secret without giving the optimiser that choice (WP34 N6).

Function Takes Gives Effect
ctSelect T, T, T T none
ctEq T, T T none

ctSelect(mask: T, a: T, b: T): T answers (a & mask) | (b & ~mask): a when mask is all-ones, b when it is zero.

  • Tests: ct_select_u32, ct_u64; reject_ct_select_i32, reject_ct_select_i32_literal, reject_ct_select_f64, reject_ct_select_f32, reject_ct_select_bool_mask, reject_ct_select_mixed, reject_ct_select_arity, reject_ct_select_float_mask.

ctEq(a: T, b: T): T answers all-ones of T when a === b, zero otherwise.

  • Tests: ct_eq_u32, ct_u64; reject_ct_eq_i64, reject_ct_eq_u8, reject_ct_eq_f64, reject_ct_eq_f32, reject_ct_eq_bool, reject_ct_eq_ranged, reject_ct_eq_mixed, reject_ct_eq_arity, reject_ct_eq_literals, reject_ct_eq_value.

  • T is u32 or u64, one type for every operand. A mask is a bit pattern, and the unsigned words are the only types whose all-ones is not also a number with a sign: i32, i64, the narrow u8 and u16, a ranged integer (an i32 underneath, and named so), f32, f64 and boolean are each `ctEq` takes u32 or u64 operands (convert with toU32 or toU64), got i64 (NL2399). A boolean in particular is not a mask: ctEq makes the mask without the compare a boolean would need. Operands of two types are `ctSelect` needs every operand of one type, got u32 and u64 (NL2400), since a u32 mask over u64 words would leave half of each unselected. The type is judged at the operand it was read from before any other operand is measured against it, so a boolean mask is told the first message rather than the second, and ctSelect(m, 0x80000000, 0) on an i32 mask is NL2399 rather than "does not fit in i32" (reject_ct_select_i32_literal).

  • A bare numeric literal takes its type from the first operand that is not one, wherever it stands: ctEq(0, x) and ctSelect(m, a, 0) are both words of x's and m's type. A fraction is then refused as a fraction (Non-integer literal `0.5` where u32 is expected, reject_ct_select_float_mask), and with nothing but literals there is no width to take, so ctEq(1, 2) is an i32 and refused (reject_ct_eq_literals).

  • The mask convention is all-ones or zero, and it is not checked, because the check would be the branch. ctEq only ever answers one of the two. Any other mask is a defined bitwise blend, bit by bit as the formula says — ctSelect(0xf0f0f0f0, a, b) takes the high nibble of each byte from a — and the Node reading below computes the same blend (ct_select_u32).

  • They are globals, not a nish: module, by the rule Builtin modules sets: what is backed by the C runtime may be imported, and what lowers to instructions reads as language. These call nothing (reject_ct_nish_module). Like every function, neither is a value (reject_ct_eq_value).

  • The lowering is the formula behind an optimisation barrier. The barrier is an empty inline asm that hands a value back in the same register — call i32 asm "", "=r,0"(i32 %mask) readnone nounwind — which costs no instruction and which no pass can see into, the idiom BoringSSL and Rust's subtle crate use. ctSelect puts it on the mask before the blend; ctEq computes ((d | -d) >>> (w - 1)) - 1 from d = a ^ b and puts it on the answer, so a caller's ctSelect(ctEq(a, b), x, y) never shows LLVM a boolean. readnone nounwind is what clang writes on the same statement and is true of an empty string, so a function that only selects stays readnone. IR_COOKBOOK.md shows the IR. Without the barrier opt -O2 folds ctSelect(ctEq(a, b), x, y) into one select on a === b — tests/run.js checks both halves of that sentence, so it learns if LLVM ever sees through the asm or stops needing it.

  • What the barrier promises: no branch and no secret-indexed load in these primitives, checked where the promise lives, in the machine code. tests/run.js compiles every tests/cases/ct_asm_* fixture with clang -O2 -S for x86-64 and aarch64 and reads each function a fixture names by its symbol (tests/ct-asm.js). It refuses any conditional branch (j<cc>, b.<cond>, cbz, tbz and the rest), any call or jump out of the function, and any load or store whose address a secret reaches, following secrets through registers, flags and stack slots and erring towards refusal wherever its model of the machine is not exact. The fixtures are both builtins at both widths, a mask straight into a select, a 32-byte MAC compare over u8[] and a table read that loads every entry (ct_asm_primitives, ct_asm_mac); and ct_asm_refused holds an early-exit compare and a secret-indexed table lookup that the check must refuse, one for each kind, so a check that read nothing fails.

  • It also refuses an instruction whose latency follows a secret: an integer divide (div/idiv, udiv/sdiv) or a floating divide or square root whose operand a secret reaches (the latency kind, seeded by ct_asm_refused's secretDivide). It reads what clang -O2 emits for the baseline CPU of each target, and for x86-64 at x86-64-v3, the one level --cpu offers, and nothing else, so a build with another profile or -march is not what was read (docs/security/ct-verification.md, CT-2, CT-16). It reads only the functions a fixture names, so a module function that is not one — every loop in std/crypto, for instance — rests on review; that record lists them.

  • What it does not promise: anything about the caller's own code. if (ctEq(a, b) !== 0) branches on a secret because the program wrote a branch; table[secret] indexes by one; === on a secret is a compare LLVM may branch on; a loop whose trip count is secret leaks it; and / and % take time that depends on their operands on many processors. The fixtures show the shapes that stay constant time, and a new one belongs beside them. On wasm32 the builtins compile with the same barrier, but the engine compiles the module again, so the .s this check could read is not what runs and nothing is promised there. Weekly, and never from npm test, .github/workflows/ct-timing.yml runs tests/ct-timing.js on an x86-64 and an aarch64 runner — a dudect-style Welch's t-test of every function the fixtures name, a fixed secret against a random one, filed as an issue when |t| passes 4.5 — which measures whether their running time on that hardware moves with the secret, and cannot promise more: a difference below a shared runner's noise, a secret index into a table that fits in the L1 cache, or another processor goes unseen.

The TypeScript reading (wp33-round-trip.md §2): runtime/nish.mjs supplies both as pure functions of their operands, so the answers agree and only the timing is not claimed. Over u32 they are class A: a u32 is a number there, the shim puts each answer back in range with >>> 0 (JavaScript's & answers a signed 32-bit integer), and ct_select_u32 and ct_eq_u32 run unmodified under the prelude and print what the native build prints. Over u64 the prelude takes BigInts and answers BigInts, and the ct_prelude check in tests/run.js holds its answers to ct_u64's native output line for line. A u64 program is still class C under Node, as all 64-bit arithmetic is (RUN_UNDER_NODE.md): a bare u64 literal is a number there, and mixing one with a BigInt throws a TypeError rather than comparing the two and answering zero.

Wiping a secret: secureZero

A key, a secret scalar or another secret intermediate has to be cleared once it is used, and a store the program never reads again is exactly what the optimiser removes: key.fill(0) just before a key goes out of scope is a dead store at -O2, and a memset in the runtime is one too once -flto inlines it into that caller. This builtin is the store that stays (#385).

Signature Semantics Effect Test
secureZero(bytes: u8[]): void every byte of bytes set to zero, in stores no optimisation removes; an empty array is a call that clears nothing; a statement write wipe_bytes; reject_wipe_i32, reject_wipe_string, reject_wipe_readonly, reject_wipe_value, reject_wipe_arity, reject_wipe_region
  • A u8[] only, as crypto.getRandomValues takes, because the byte is the unit a secret is written in: anything else is `secureZero` fills a `u8[]`, got i32[] (NL2402), a string included, which is immutable and has nothing to clear (reject_wipe_i32, reject_wipe_string). The call writes its argument, so a readonly u8[] is Cannot `secureZero` through readonly u8[] (declare it u8[] to write through it) (NL2264, reject_wipe_readonly), and a function that wipes a parameter does not get readonly on it (wipe_bytes). In a scope's region it is refused where fill is (NL2394, reject_wipe_region).
  • A statement: there is nothing to answer, so `secureZero` returns void and can only be used as a statement (NL2105, reject_wipe_value), and it takes exactly one argument (NL2061, reject_wipe_arity).
  • The lowering is one call, call void @nish_wipe(%struct.nish_array* %key), declared nounwind willreturn and nocapture without readonly and with no memory attribute, so LLVM may not treat it as a store it can drop. nish_wipe in runtime/runtime.c writes each byte through a volatile pointer, which the C standard makes behaviour no optimisation may remove even once LTO has inlined the body, and is noinline besides, so the call stays visible in a disassembly. IR_COOKBOOK.md shows the IR.
  • It is pinned under -O2 -flto, where it matters. The runtime unit test builds a probe the way --link builds, -O2 -flto over the runtime's sources: a key written to a block, used, wiped and freed, then read back through a pointer the optimiser cannot follow. With no wipe the key is still there, and with a memset wipe in another translation unit it is still there too, dropped as a store before free; with nish_wipe nothing is left. The same test fails when nish_wipe is written as a memset.
  • What it does not promise: the copies the program made. It clears the bytes of the array it is handed. A copy taken before (slice, a push into another array, an element read into a local, a register spilled to the stack) is the program's to clear, and nothing clears what the operating system swapped out.
  • It is a global, and not in a nish: module. It is backed by the C runtime, so by the rule Builtin modules sets it may be imported, but no module exports it yet. It works under the wasi profile, which links runtime.c; a freestanding wasm module has no nish_wipe: it links runtime-wasm.c, which supplies the arena, arrays, the panics and floor/ceil/trunc but no wipe, and, when it uses strings, runtime-text.c, which has the strings and the digits but not the wipe.

The TypeScript reading (wp33-round-trip.md §2) is class A: runtime/nish.mjs supplies it as a loop that sets every element to zero, and wipe_bytes prints the same lines under the prelude as natively. What Node cannot promise is the native build's stronger claim, because a collector that moved the array may have left the old bytes behind.

Who owns a path: lstatOwnerModeSync, geteuid and isExecutableSync

What a driver asks before it trusts a directory or a program it found on its own: who owns this path, may anyone else write to it, and will the shell run it (#386; security/cli.md, CLI-7 and CLI-9).

Function Takes Gives Effect
lstatOwnerModeSync string i64 write
geteuid nothing i64 write
isExecutableSync string boolean write

lstatOwnerModeSync(path: string): i64 does one lstat of path. It answers the owner's uid in the high 32 bits and st_mode (the file type and the permission bits) in the low 32, or -1 when the path does not resolve or holds a NUL. A symbolic link answers for itself, not for what it names.

  • Tests: owner_checks; reject_owner_arity, reject_owner_i32, reject_owner_wasm.

geteuid(): i64 answers the effective user id, the one to compare an owner with.

  • Tests: owner_checks; reject_owner_euid_arity, reject_owner_euid_wasm.

isExecutableSync(path: string): boolean is access(path, X_OK): whether the real user may run path. It answers false for a path that does not resolve or holds a NUL.

  • Tests: owner_checks; reject_owner_type, reject_owner_exec_wasm.

  • The owner and the mode are one answer. Both come out of one lstat, so they describe the same file however the path changes in between; two calls could describe two. Take them apart with (om >> 32) & 0xffffffff (the owner) and om & 0xfff (the permission bits) or om & 0xf000 (the type: 0x8000 a regular file, 0x4000 a directory, 0xa000 a symbolic link). The mask is not optional: a uid is unsigned and one of 2^31 or more fills bit 63, which >> on an i64 copies down, so om >> 32 alone answers a negative owner for a file such a user owns and never equals their geteuid(). >>> would say the same natively, but a BigInt has no >>>, so the mask is the spelling that reads the same under Node (owner_checks). A uid fills 32 bits, so the answer is an i64 in both number modes, as geteuid's is, and an i32 is refused rather than truncated (Cannot initialize i32 variable `om` with i64, reject_owner_i32).

  • Not following a link is the point. A directory that is a symbolic link to one this user owns is not a directory this user owns, and the test of a path that has to be trusted is of the path as written. That a link answers for itself is pinned by RT-9 in tests/runtime-test.c, since a program cannot make one.

  • isExecutableSync is the shell's question, asked for the real user as access asks it: a file with no execute bit is not executable, root included, and a readable file is not the same answer.

  • Refused on a wasm32 build, as statMtimeSync is: there is no file system and no user there, so each is `lstatOwnerModeSync` reaches the operating system, and a wasm32 build has none to reach (reject_owner_wasm, reject_owner_euid_wasm, reject_owner_exec_wasm). Each takes exactly its arguments (NL2061, reject_owner_arity, reject_owner_euid_arity), and a path is a string (reject_owner_type).

  • The lowering is one call each, call i64 @nish_lstat_owner_mode(i8* %p), call i64 @nish_euid() and call zeroext i1 @nish_is_executable(i8* %p), declared nounwind willreturn and ordered with every write, because the file system is not memory LLVM tracks and a failure stores errno (IR_COOKBOOK.md).

  • Globals, and not in a nish: module yet, as secureZero is.

The TypeScript reading (wp33-round-trip.md §2) is class A: runtime/nish.mjs supplies fs.lstatSync's uid and mode packed the same way as a BigInt, process.geteuid(), and fs.accessSync with X_OK, and owner_checks prints the same lines under the prelude as natively.

Secrets: nish:secret

Key material is a Secret<T>: an opaque value that the checker keeps from leaving the program by any road but one, and that is zeroed, with a store no optimiser may remove, before the function that made it lets go of it.

Reference: nish:secret.

Export Meaning Test
Secret<T> an opaque class holding an array or record of integers secret_wipe, secret_flow; reject_secret_shape
secret(v) v, which nothing else holds, wrapped; a local is moved secret_flow, secret_inferred; reject_secret_moved, reject_secret_unmovable, reject_secret_alias_*, reject_secret_inferred_*
expose(s, f) f(value), under the rules below secret_wipe; reject_secret_expose_*
exposeWith(s, arg, f) f(value, arg), under the same rules secret_wipe; reject_secret_expose_writes_arg
wipe(x) zero a Secret, or an array or record of integers expose's function holds secret_wipe, secret_wipe_o2, secret_wipe_o2_inferred; reject_secret_wipe_shape, reject_secret_wipe_nullable

nish:secret is the one builtin module with source behind it: std/secret.ts, loaded like a standard-library module, imported only by the builtin name (`nish/secret` is the builtin module `nish:secret`, NL2453, reject_secret_std_spelling) and copied, as std/collections.ts is, into each module that uses it, so it writes no .ll of its own. runtime/nish.d.ts declares it for tsc, with a private constructor and field so that tsc refuses new Secret and s.value too. Under Node (runtime/nish.mjs) a Secret is a plain wrapper and wipe zero-fills it.

A Secret local needs no annotation. const k = secret(v) gives k the type secret answers, Secret<T> for the T that v is, and every rule below holds for it as for const k: Secret<T> = secret(v): the payload's shape, the move and the second reference, each judged where v is written (secret_inferred binds an array literal, new Array, a call, a moved local, a ternary, a record and a class unannotated; reject_secret_inferred_literal, _alias, _call, _field and _moved_class are refused unannotated as annotated). The same holds for any generic an imported module declares: const b = box(7) holds the Box<i32> that box answers, whose layout comes with the call (tests/link/generic_import_inferred), with its methods even where the module already held Box<i32> as a type argument (tests/link/generic_import_inferred_held), and with those of every struct its fields reach, through an array, a nullable, a Result or a nested instance, in either order (tests/link/generic_import_inferred_closure).

What a Secret may hold. One run of integers, so that wipe reaches every byte with one store: `Secret<string>` is not key material (NL2428, reject_secret_shape). wipe takes a Secret, never a nullable one (narrow it first), or an array or record of integers, and anything else is `wipe` zeroes a `Secret`, or an array or record of integers, and `string` is neither (NL2438, reject_secret_wipe_shape, reject_secret_wipe_nullable).

Where a Secret may not go. Each place is refused at the point it is written, before the general rule it would also break, so the one diagnostic names the secret:

  • a template literal hole (NL2431, reject_secret_template);
  • an argument of any builtin — console.log, a file write, nish:net, a process, an environment read, crypto.getRandomValues — named or imported (`console.log` cannot be handed a `Secret`, NL2432, reject_secret_console, reject_secret_file_write, reject_secret_net_write);
  • === and !== against anything but another Secret or null (NL2433, reject_secret_equality): k === null is how a nullable one narrows;
  • the condition of an if, a loop or a ternary (NL2434, reject_secret_condition);
  • an index, as the receiver or as the index (NL2435, reject_secret_index, reject_secret_as_index);
  • a member read, written or called anywhere but std/secret.ts (NL2436, reject_secret_member), and new Secret (NL2437, reject_secret_new);
  • an array element, a Result payload, or a type argument of any class, Map and Set included (NL2429, reject_secret_array, reject_secret_result, reject_secret_generic_class). A Secret lives in a local, a parameter, or a holder's field (A Secret field), and Secret<T> | null is how a function hands back one it may not have. Any other field — of an interface, of a generic class, not readonly, or not exactly Secret<T> — is NL2430 (reject_secret_field, reject_secret_field_interface, _generic_class, _not_readonly, _nullable).

A Secret leaves the function that made it returned or wiped, on every path. This is Result rule 1's analysis in its two places — the statement that drops a value, and the body that never handles one — with the second walked path by path rather than counted, because "wiped on one arm of an if" is not "wiped" (reject_secret_one_branch):

  • A Secret a call makes has an owner: a const or let, a return, or wipe. Anywhere else — a statement, a call's argument — it is `Secret<u8[]>` is made here and nobody owns it (NL2439, reject_secret_discarded, reject_secret_temporary).

  • Every return, break, continue and orReturn() that leaves a Secret the function made neither wiped nor returned is `k` holds a `Secret<u8[]>` that this exit leaves neither wiped nor returned (NL2440, reject_secret_unwiped_return, reject_secret_break, reject_secret_or_return), and a block that can reach its end with one is NL2441 (reject_secret_scope_end). A branch that proves the local null (if (k === null)) holds none, a panic or process.exit ends the process and the key with it, and a loop is walked to its fixpoint, so a wipe inside a loop that may run no pass does not count (secret_flow).

  • A Secret local may not be assigned while it still holds one (NL2442, reject_secret_overwrite), read once any path has wiped it (NL2443, reject_secret_after_wipe), or copied: const j = k moves k when the function made it, and k is not read again (NL2446, reject_secret_moved_out), but a parameter, an element of an array literal or an arm of a ternary is a second name for one key (NL2444, reject_secret_copy_param).

  • secret(v) takes a value nothing else holds, and moves it, because a second name for the bytes would read the key outside the Secret, and zeros once wipe has run. A local handed in is not read again (NL2445, reject_secret_moved, whose message says the plain array, and reject_secret_inferred_moved_class and reject_secret_moved_interface, where it names the class or interface the plain value is), and neither is the local in either arm of secret(c ? a : b). A parameter, a field or an element, whose owner keeps the plain bytes, is refused (NL2447, reject_secret_unmovable). Everything else is judged where it is written:

    • an array or object literal, null, and a ternary of fresh values are fresh, and so is a new whose constructor keeps no this;
    • a call is fresh when its callee answers a fresh value, judged over the callee's own body by this same rule, through every depth of calls (a recursive one included); expose and exposeWith answer what the function they run answers, and r.orReturn(), r.expect(m) and r.unwrapOr(d) what r carries in its Ok (d too, which may not be a local here, since the move does not reach it). A method of a value, rows.pop(), may answer what the value holds;
    • a local is fresh when it is the sole reference to its value: every value it is given is fresh (another local is not, because that local still names it), it is not a for...of binding, which names an element, and no other use of it leaves a second name. Reading an integer element or field, .length, an array method on it, ===, iterating over it, wipe, and handing it to a builtin, or to a function or method the whole-program facts say keeps nothing of it, are all such uses. An initialiser, an assignment's value, a field or element store, a literal's element, a ternary arm, either side of ??, the argument of Ok or Err, an assignment to it used as a value (alias = raw = [1], xs.push(raw = [1]); one whose value is dropped, as a statement or in a for header, is not), a later declarator of the same statement, a return, an argument of a method of another value (rows.push(raw), m.set(k, raw)), a read by an arrow that captures it (a data-parallel body or a spawned task), and a function or constructor that keeps it each leave one.

    Each is `secret` takes a value nothing else holds, and `raw` has a second reference: `const alias = raw` (line 6), naming the reference and its line, or the callee and its parameter that keep it (NL2512, reject_secret_alias_const, reject_secret_alias_let, reject_secret_alias_reassigned, reject_secret_alias_assign_chain, _assign_init, _assign_push, _assign_field, _assign_kept, _assign_helper, _assign_parents, _assign_constructor, _assign_source, reject_secret_alias_declarators, reject_secret_alias_field, reject_secret_alias_element, reject_secret_alias_param, reject_secret_alias_return, reject_secret_alias_mutual, reject_secret_alias_ternary, reject_secret_alias_nullish, reject_secret_alias_ok, reject_secret_alias_err, reject_secret_alias_unwrap_or, reject_secret_alias_unwrap_helper, reject_secret_alias_unwrap_local, reject_secret_alias_constructor, reject_secret_alias_receiver, reject_secret_alias_pop, reject_secret_alias_for_of, and reject_secret_alias_capture_map, _from, _map_into, _reduce and _spawn, and reject_secret_alias_spawn_arg and _spawn_with for a task's argument, which the scope keeps until the task has run). Pass a fresh value instead, or a copy: const copy: u8[] = new Array<u8>(raw.length); copy.set(raw), then secret(copy). A declarator that reads raw after secret(raw) in the same statement is a moved read, NL2445 (reject_secret_alias_declarator_moved). The rule refuses some programs that are safe, each with the reference named: a data-parallel arrow that reads the local before the move, a Result local copied into another (reject_secret_alias_result_copy), and a let given another local even after the move (reject_secret_alias_reassigned_after).

  • A Secret parameter is borrowed. The callee may read it through expose or hand it on, and owes nothing: the caller wipes what it made. It neither returns it, which would give the caller a second name for a key it already owns (NL2546, reject_secret_field_launder_function and its siblings, among them reject_secret_field_launder_local for a caller's own local), nor wipes it, which would leave the caller reading zeroes and wiping it again (NL2548, reject_secret_field_wipe_param and its siblings, among them reject_secret_field_wipe_param_local). The cases are listed under A Secret field.

    This narrows 0.18.0. Up to 0.18.0 a callee could also return or wipe a borrowed Secret; from the release after it, both are refused, for every Secret parameter and not only a holder's field. A function that returned its parameter is called for its effect and the caller goes on using the key it passed; a function that wiped its parameter leaves the wipe to the owner, the function or holder that made the key.

A Secret field

A key that lives as long as an object does — a connection's traffic secret, a signer's private scalar — is kept in a readonly field of exactly Secret<T>, in a class with no type parameters. Such a class is a holder. The checker proves three things of it: the field is set once, from a fresh value; it is read only where the key cannot get out; and the holder's wipe() ends it.

class Session {
  readonly key: Secret<u8[]>;
  constructor() {
    this.key = secret(new Array<u8>(32));   // once, from a fresh value
  }
  rekey(by: u8): void {
    exposeWith(this.key, by, refill);       // rewritten in place, never replaced
  }
  wipe(): void {
    wipe(this.key);                         // every field, on every path
  }
}
  • The releaser. A holder declares wipe(): void, taking no parameters (`Holder` holds the `Secret` field `k`, so it declares `wipe(): void`, NL2540, reject_secret_field_no_wipe, _wipe_signature). Its body calls wipe(this.f) itself for every Secret field, on every path: the field is walked as an owned local would be, so a return before it, one arm of an if, a loop that may run no pass, and a helper method that wipes it are each `this.k` can leave this `wipe()` unwiped (NL2541, reject_secret_field_wipe_missing, _wipe_return, _wipe_branch, _wipe_loop, _wipe_helper, and _wipe_new, where a constructor new reaches zeroes the key in place). That method is the only place a field ends: wipe(h.f) anywhere else — a function, another method, a constructor, or a function the constructor hands this to or stores it for — is `k` is wiped outside its holder's `wipe()` (NL2547, reject_secret_field_wipe_outside, _wipe_through, _wipe_method, _wipe_ctor, _this_fin, _this_store); call the holder's wipe().
  • Construction. The field is set in the constructor, by this.f = secret(v) and nothing else, and v is judged by the sole-reference rule above (NL2512, reject_secret_field_alias_raw). A parameter, a call that answers a Secret, or a local holding one is `this.k` is set from something other than `secret(...)` (NL2542, reject_secret_field_param, _call, _local); a second store while the first may still be there, a later loop pass included, is NL2543 (reject_secret_field_twice, _ctor_loop). The field is readonly, so a store anywhere else — a method, a function, a compound +=, or a second name for this — is the readonly refusal, NL2181 (reject_secret_field_store_method, _store_outside, _compound, _this_escape). The store is a statement of its own: const x = (this.k = secret(v)) binds the key a second time, and is NL2544 (reject_secret_field_store_value). A holder is made only by new: an object literal is NL2545 (reject_secret_field_object_literal), and typed JSON neither encodes nor decodes one (NL2485, reject_secret_field_json, _json_decode).
  • Reads. h.f is borrowed, as a Secret parameter is: it is an argument of a call or a new — expose, exposeWith, or a function that takes a Secret, and wipe only in the holder's own wipe() (NL2547) — or an operand of === or !== against another Secret (a field is never null, so h.k === null is a type error, NL2024). Bound to a local, assigned, returned, chosen by a ternary or ??, or read through ?. or a chain of fields, it would be a second owner of the key, and is `k` is a `Secret` field, which is read only as a call's argument or an `===`/`!==` operand (NL2544, reject_secret_field_bind, _assign_local, _return, _ternary, _nullish, _optional_chain, and _nested for o.h.k; _chain is the chained store this.k = this.j = secret(v)). What a borrow receives stays borrowed: a function, an arrow or a generic id<T> that takes a Secret and returns it would hand the caller a second name for the holder's key, and is `s` is a `Secret` parameter, which its caller owns and wipes: returned (NL2546, reject_secret_field_launder_function, _launder_arrow, _launder_generic, _launder_method, and _launder_local for a caller's own local); one that wipes it is NL2548 (reject_secret_field_wipe_param, _wipe_param_new, _wipe_param_method, and _wipe_param_local for a caller's own local). A ternary that picks one of two borrowed parameters to return is refused before it gets there, as a copy (NL2444, reject_secret_field_launder_ternary). Every place a Secret may not go is closed to the field too: reject_secret_field_console (NL2432), _template (NL2431), _equality (NL2433), _condition (NL2434), _index (NL2435), _member (NL2436), _array, _array_literal and _result (NL2429), _new_arg (NL2439: a Secret made for a constructor that keeps none, beside an accepted new Box(h.k) that borrows the field), and _secret_arg and _holder_payload (NL2428: neither a Secret nor a holder is key material).
  • The holder is a value. It may be aliased and kept in an array, a Map, another object's field or a Result. The rules that follow are the general ones, applied to a holder as to any class: a data-parallel body may not capture one (NL2504, reject_secret_field_capture) nor may a task (NL2509, _task_wipe), the scope's own thread may not wipe one a task of it may read before the join — through wipe(), a helper or a constructor (_scope_wipe, _scope_helper, _scope_new) — and using does not take one (NL2387, reject_secret_field_using).

secret_field runs all of it, and tests/run.js holds the wipe to -O2 ("nish:secret: a holder's wipe survives opt -O2"): secret_field_wipe_o2's wipe() is a volatile llvm.memset of the key after opt -O2, where its twin secret_field_wipe_o2_plain, zeroing the same bytes with fill(0), has an ordinary one. An exported holder appears in an --emit-header sidecar as an opaque struct, a typedef with no fields: with its layout a host could allocate one and skip the constructor that sets the key, so it reaches C only as a pointer. A Secret an exported function takes or returns is likewise a pointer to an opaque struct, as a Map is, and --emit-dts and --emit-napi bridge no function that takes or returns a holder or a Secret ("nish:secret: --emit-header keeps a Secret and its holder opaque, and --emit-dts / --emit-napi bridge neither").

What the rule leaves out, so a review need not look for it here:

  • A holder nobody wipes. The checker proves that wipe() ends every key, not that anybody calls it: a holder dropped, or released with its arena, without wipe() leaves its key in memory. Holders are values that cross functions, aliases and containers, and their owner is not tracked.
  • Aliases. wipe() through one alias zeroes the key for every alias, and an expose through another reads zeros afterwards: a wrong answer, never a leak.
  • Plain temporaries. What an exposed function copies out of the key is a plain value, and is wiped only if that function wipes it.
  • Per-packet and per-record keys. These cannot be a Secret under today's expose: sealing writes the output buffer (NL2450) and the AES kernels are C (NL2448).

expose and exposeWith. The function either runs is a function parameter: a top-level function named at the call, or an arrow written there. It is judged once the whole-program facts are in, at the call, and what it returns is the only thing that leaves:

  • It reaches no I/O — no builtin that touches a file, a directory, a process, the environment, a socket, a stream, the clock or entropy — and no C, through any depth of calls (`leak` reaches `writeFileSync` through `store`, NL2448, reject_secret_expose_io, reject_secret_expose_io_deep, reject_secret_expose_c). A panic or exit is allowed only with a literal, since a computed message or status is output (reject_secret_expose_panic). A failed runtime check — an index, a slice, a division — still stops the program with its message, which carries the operand it failed on; an index that depends on a key is already a cache-timing defect, and is for the constant-time checks of WP34 N6 to find.
  • It keeps nothing of the value: returning it, storing it or passing it where it is kept is NL2449 (reject_secret_expose_keeps). exposeWith's function writes nothing through the argument it is handed either (NL2450, reject_secret_expose_writes_arg), which would be a road out its result type does not name.
  • Its result is a declared type that holds no Secret (NL2451, reject_secret_expose_secret_result), and an arrow writes that type down (NL2452, reject_secret_expose_undeclared).

The wipe. Each wipe instance is one llvm.memset with its volatile flag set, over an array's whole capacity — a pop leaves the old element in the buffer — or a record's sizeof. LLVM never removes, merges or narrows a volatile memory intrinsic, so the store survives every optimisation, inlining included: tests/run.js compiles secret_wipe_o2, whose wiped bytes nothing reads again, beside a twin that zeroes them with fill(0), and after opt -O2 the twin holds no store and the wipe holds one volatile zero per byte. secret_wipe_o2_inferred does the same for a Secret bound with no annotation, beside secret_wipe_o2_inferred_plain ("nish:secret: an inferred Secret's wipe survives opt -O2 (a volatile memset of its key), where the twin's fill(0) is deleted"). Two things it does not reach, and says so: a buffer a push outgrew, which is no longer the array's, and a string, which is immutable and is not key material.

wipe and secureZero both make a store no optimisation removes, and differ in what they take and where the store is written. secureZero is a global over a bare u8[], one call into the C runtime; wipe is the Secret's own, takes any integer width or a record of integers, and is an intrinsic emitted into the program, so it needs no runtime function and works wherever the program's IR does. Key material belongs in a Secret, and secureZero is for bytes that never were one.

Compact classes

class R implements Compact (import { Compact } from "nish:compact") stores every field typed exactly R or R | null — a link, in R itself or in any other class — as a 32-bit offset into the one address range the arena reserves, where a plain class stores a 64-bit pointer. A list or tree node of an i32 and two links is 12 bytes instead of 24, so twice as many fit in a cache line.

import { Compact } from "nish:compact"

class Cell implements Compact {
  value: i32
  next: Cell | null
  constructor(value: i32, next: Cell | null) {
    this.value = value
    this.next = next
  }
}
  • Only a link moves. A local, a parameter, a return, an array element, a Map key or value, and a field whose type is a wider union all stay full pointers. A load of a link widens it back to a pointer and a store narrows it; what the program reads and prints is the same as without the marker (compact_list).
  • The class's own layout follows its fields. A compact class whose fields are all 4 bytes or narrower is 4-aligned and rounded to 4, so Cell is 8 bytes; one with a string or an f64 keeps 8. Its objects are allocated in the range and never on the stack.
  • Where a pointer is already 4 bytes, nothing moves. On wasm32 (--profile wasi, --target wasm32), in a page module built for the host with --emit-dts (it runs on the wasm runtime, whose root trace reads each link as a pointer: compact_root_page), and under NISH_ASAN, the marker compiles to the IR the class has without it.
  • The marker changes nothing under Node. Type stripping removes the implements Compact clause, and the nish:compact import resolves to the frozen values runtime/core.mjs exports, so the program runs unmodified there.
  • A generic field instantiated as exactly a compact class is narrowed. Box<Cell>.inner, with Cell compact, holds a compact link, so an interface view of the field and its implementer agree on the layout.
  • A compact class is refused where C, Node or the runtime would read its layout as full pointers, each with its own code: in a build with --emit-header, --emit-napi or --emit-napi-async, at the class (NL4021, compact_reject_header), or at an export whose signature holds one (NL4020, compact_reject_export); as a declare class (NL2795, compact_reject_declare), or held by one (NL2796, compact_reject_declare_field); and holding a Secret, whose wipe reads it through a full pointer (NL2797, compact_reject_secret); and as a generic class, whose instantiations keep full-pointer links (NL2798, compact_reject_generic).

Wide is accepted and changes nothing yet; it is reserved for a class to refuse a compact layout the compiler would pick on its own.

Reference: nish:compact.

Regions: nish:region

A region is memory a program frees when it says so: an arena of its own, reached through a handle. A long-running program — a cache, a server's per-connection state — keeps each thing that lives and dies together in one region and gives the whole of it back with one free, where the thread's arena would hold it until the program ends.

Reference: nish:region.

Export Meaning
Region<T> a handle to a region whose root is a T: a 64-bit number, not a pointer
region(f), regionWith(arg, f) a new region, whose root is what f answers; f's allocations land in it
child(f), childWith(arg, f) a new region whose parent is the region entered now; called only in the f an enter or enterWith runs
enter(r, f), enterWith(r, arg, f) f(root) (or f(root, arg)), allocating in the region
free(r) gives the region's memory back; r and every copy of it are dead. Freeing a parent frees its children first, deepest first; freeing a child unlinks it from its parent

f is an arrow or a named function, and an arrow captures nothing (NL2335), so the ways into a region are its root, arg and module state. The checker follows each (src/region-rules.ts), with two kinds of value: region memory (the root, and whatever f allocates) and outer memory (arg, module state, what a callee answers from outside). A number, a boolean, an enum and a Region are copied, never pointed at, so they cross freely (tests/cases/region_cache).

Refused Code
region memory stored into outer memory: a field or an element, including this stored by a constructor f reaches (h.l = this). Module state cannot hold region memory at all, since a module binding is a number, boolean or string constant or module storage, whose elements are pointer-free, and a top-level let is refused first (reject_region_ctor_module_state) NL2609 (reject_region_to_outer, reject_region_ctor_this_arg, reject_region_store_string, reject_region_arrow_keep, reject_region_arrow_generic; a spawned task's region into what it shares, reject_async_spawn_store_region)
outer memory that is not a number, boolean, enum or string literal stored into region memory: a field or element store, a constructor argument (new Game(g.host)) or a field of a literal NL2610 (reject_region_outer_string_ctor, reject_region_outer_string_literal, a spawned task's shared object kept by its own memory, reject_async_spawn_keep_holder, reject_region_arrow_link, reject_region_arrow_result, reject_region_arrow_compose, reject_region_arrow_twice, reject_region_fixpoint_chain)
enter's or enterWith's f answering anything but a number, a boolean, an enum or a Region: an array, an object or a string NL2611 (reject_region_enter_result, reject_region_enter_string)
scope(), spawn, a parallel body or a channel inside f, directly or through a callee NL2612 (reject_region_threads)
a value whose type holds a Region at any depth crossing to another thread: a task's argument or answer, a parallel body's capture, a value sent on a channel NL2613 (reject_region_task_arg_field, reject_region_task_answer, reject_region_channel_map, reject_region_parallel_capture)
region's or regionWith's f answering anything but memory it allocated: region(() => moduleValue), regionWith(g, (a) => a) NL2614 (reject_region_root_module, reject_region_root_arg)
outer storage growing inside f, directly or through a callee: push on an outer array (arrays have no unshift or splice; the rule names them so it holds if they come), Map.set or Set.add on an outer Map or Set NL2615 (reject_region_grow_push, reject_region_grow_map, reject_region_grow_set, each with a _callee twin, reject_region_arrow_grow; a spawned task growing what it shares, reject_async_spawn_grow)
a call inside f the region rules have no facts for. No program reaches it today, since every function, instantiation, lifted arrow and declare function has whole-program facts; it stays so that such a callee is never let through unexamined (tests/wordings/unreachable.txt) NL2616
Arena.mark or Arena.used inside f, or a callee reaching one: inside a region they would read the region's own arena, not the thread's NL2617 (reject_region_arena_mark through region, reject_region_arena_used_callee through enter and a callee)
new Region<T>() NL2618 (reject_region_new)
child or childWith outside the f an enter or enterWith runs, or in a function that something other than an enter also calls: with no region entered there is no parent NL2702 (reject_region_child_outside, reject_region_child_callee; a helper making the child, called from main, reject_region_child_helper_main, or from the function enter runs, reject_region_child_helper_f)
region, regionWith, child, childWith, enter, enterWith or free in a parallel body or a spawned function, directly or through a callee NL2351 (reject_region_enter_spawned, reject_region_parallel_callee, reject_region_child_parallel, reject_region_child_spawned, reject_region_child_with_parallel, reject_region_child_plain_spawned)
using r = region(f), in any declarator: a block does not free a region NL2387 (reject_region_using_first, reject_region_using_second)
  • Arena.mark and Arena.used are refused inside f (NL2617). Inside a region they read the region's arena, not the thread's, so a mark taken there names nothing the thread's arena knows once f returns (docs/security/runtime.md, RT-45).
  • An arrow handed to a function runs inside f, and is judged as f's own code. Its body is walked with what it captures, its parameters take what the call is handed, what any arrow of the same call answers (the callee may feed one arrow's answer to another, or to itself) or memory the callee allocated, and what it answers is part of what the call answers, so pick(() => a.node) stored in the root is NL2610 as a.node is (reject_region_arrow_keep, _grow, _link, _result, _generic, _compose, _compose_temp, _twice, _twice_temp). The walk runs to a fixpoint however long a chain of assignments is, never a fixed number of rounds (reject_region_fixpoint_chain).
  • The rules are on types for threads, on values inside f. NL2613 follows the type, so a handle hidden in a field of a task's argument is caught where it crosses, whatever route brought it there. A handle stored in region memory, or answered by enter, stays on its thread and is allowed.
  • using a = arena() inside f is allowed, and so is enter of a second region: only numbers come out of it.
  • A child region may point into its parent's memory that existed before the current enter: what the root enter handed in reaches, and, in the child's f, what childWith's arg reaches when the root reaches it (tests/cases/region_child_keep, region_child_chain, region_child_map). The parent outlives the child, since freeing the parent frees the child, and nothing can release that memory while the parent lives: a scope opened during an enter rewinds only to a mark taken after the enter began, so it releases only what this enter allocated. That memory new in this enter is what the child may not keep (NL2610, reject_region_child_new_memory). The other rules hold the parent as outer memory: the child's memory stored into the parent's is NL2609 (reject_region_child_to_parent), growing a parent array, Map or Set in the child's f is NL2615 (reject_region_child_grow), and the parent's memory as the child's root is NL2614 (reject_region_child_root_parent, and reject_region_child_root_listed, where it is read back out of a list the child's f made). The child's f may keep the parent's memory in the root it allocated, by a field or element store or a push onto that root's own array (tests/cases/region_child_builder_list). Only f's own body may: a function it calls that keeps what it is handed in the memory it answers makes that answer NL2614, under regionWith as under childWith (reject_region_helper_keeps_arg, reject_region_child_helper_keeps_arg). A plain region() made inside enter stays independent of the region entered, and keeps the rules above.
  • What the root reaches counts as older than this enter only while f stores nothing into memory the root may reach. Once f stores into memory that may be older than this enter — a field, push, or a callee that writes into what it is handed, such as Map.set — anywhere in its body, a value read off the root may be one it allocated, so childWith handing it is NL2610 (reject_region_child_root_readback, reject_region_child_root_push). A store into memory f made counts only when f also puts the root's reach (other than a string) into memory it made, as [root] or new Holder(root) does (tests/cases/region_child_fresh_list, tests/cases/region_child_local_map); then a store through it may put new memory in the root's reach (reject_region_child_alias_field, reject_region_child_alias_element). The check is not ordered: a store after the childWith counts too, and one such store makes every childWith in f hand outer memory rather than the parent's. To collect root nodes in a list, collect them in the child's f, as above. A call's result handed to childWith is memory this enter may have allocated, even when the callee only reads the root, so a child may not keep it.
  • In an enter of a handle that may be a child's, the root's reach may be its ancestors' memory as well as the child's own. A child made there may keep it, but it may not grow (NL2615), keep memory other than an ancestor's (NL2610 for outer memory, reject_region_child_handle_in; NL2609 for memory new in this enter, reject_region_child_handle_new), nor be stored into outer memory (NL2609, reject_region_child_handle_out). Nor may such a root grow: a handle that reaches enter through a parameter is one, even when every caller hands a plain region (NL2615, reject_region_child_handle_param). A handle may be a child's when its type is one child or childWith answers somewhere in the program, unless the handle is a call of region or regionWith or a variable bound only ever to one: a parameter, a field or an element of that type may hold a child's (tests/cases/region_child_plain).
  • Every call checks its handle at run time. enter, enterWith and free stop the program on a handle that is 0, freed already, or another thread's; enter of a region already entered and free of an entered one (or of a parent with an entered child) stop it too, so freeing twice is a panic and never a double free. A child's handle is dead once its parent is freed (tests/link/region_child_stale), and child with no region entered stops the program. Each call is a region site under --deny-panics (reject_deny_panics_region), and no proof removes it. Past 16,777,216 live regions region stops the program as well.
  • A program run under Node as it is written (runtime/nish.mjs) gets a region as an entry in a table, free drops it, and the same handle checks throw. --emit-ts output is different: the runtime it imports (nish/runtime) has no regions yet, so it refuses the module (NL4015, reject_emit_ts_region).

process

The reference for each member of process, with its parameters, its answer and its capability, is its doc comment in runtime/nish.d.ts; the rules are here.

Member Takes Gives Effect
process.exit i32 void write, noreturn
process.argv (property) string[] read
process.platform (property) string none
process.arch (property) string none

process.exit(code: i32): void ends the program with code. The stdio buffers of linked C code are flushed; the arena is abandoned. It is used in statement position, and it is a terminator.

  • Lowering: nish_exit -> libc exit(code).
  • Tests: process_exit, reject_exit_unreachable.

process.argv: string[] is the command line. process.argv[0] is the program path (C's argv[0], one index earlier than Node, whose argv[0] is the node binary and argv[1] the script), and the rest are the arguments as UTF-8 byte strings. It is read-only.

  • Tests: argv_echo (run with argv_echo.argv), link/argv_import; reject_argv_assign, reject_argv_push (`process.argv` is read-only).

process.platform: string is the operating system the program is running on, spelled as Node spells it: "linux", "darwin", or "unknown" for anything this compiler has no target triple for (every WASI build included).

  • Tests: io_host; reject_platform_assign (Cannot assign to `process.platform`).

process.arch: string is the architecture, likewise: "x64", "arm64", or "unknown".

  • Tests: io_host; reject_arch_call (Unknown builtin `process.arch`).

process.platform and process.arch are what --target host asks the machine (WP14 §7a): the pair maps to x86_64/aarch64 plus -unknown-linux-gnu/-apple-darwin, and anything else means this compiler has no triple for the host and says so. Each is one call to the runtime, which answers the address of a string in its own constant data — settled when runtime.c was compiled, so a cross build reports the target, and nothing is allocated or loaded, which is why the declarations are readnone and a function built only from these stays pure. They carry no rule about the entry point, unlike process.argv below: there is nothing for a wrapper to build, so a wasm or N-API library may read them too. Neither can be assigned to.

process.argv is an ordinary string[] value (length, a[i], for...of, passing it to functions, aliasing it with const args = process.argv) that the @main wrapper builds once with nish_argv_init(argc, argv) before the program runs; every module of the program may read it, and each read is one load of the runtime global @nish_argv (a memory read: the function is at most readonly). The array lives outside the arena, so no arena scope's release invalidates it. Storing into it (process.argv[i] = s, op=, ++/--, push) is rejected. A program without export const main, as for a wasm or N-API library, has nothing to build it from and rejects every use with `process.argv` requires a `main` entry point (reject_argv_no_main, tests/link/argv_no_main). Under the wasi build profile the WASI host supplies the arguments (examples/wasi-host.mjs).

Reference: nish:process.

File I/O

Reference: nish:fs.

Function Takes Gives Effect
readFileSync string string write
readFileSyncOrNull string string | null write
readFileBytesSync string u8[] | null write
writeFileSync string, string void write
appendFileSync string, string void write
writeFileBytesSync string, readonly u8[] i32 write
renameSync string, string i32 write
unlinkSync string i32 write
writeFileSyncMode string, string, i32 i32 write
createFileExclusiveSync string, string, i32 i32 write
mkdirSyncMode string, i32 i32 write
fileOpen string i32 write
fileReadAt i32, u8[], i32, i32, i64 i32 write
fileSize i32 i64 write
fileClose i32 i32 write

readFileSync(path: string): string reads the whole file as one arena string. A failure prints nish: cannot read <path> to stderr and exits 1.

  • Tests: io_files; reject_readfile_number (`readFileSync` expects an argument of type string, got i32).

readFileSyncOrNull(path: string): string | null is the same read, with null where the other exits (WP14 B3): null for exactly the set of paths readFileSync prints nish: cannot read <path> for, a directory included. The result is narrowed with if (text !== null) like any other nullable.

  • Tests: io_streams; reject_readfile_or_null_unchecked.

readFileBytesSync(path: string): u8[] | null answers the file's bytes, as they are on disk, or null for exactly the paths readFileSyncOrNull answers null for; there is no exiting twin. The result is an ordinary arena u8[] with length === capacity, narrowed with if (b !== null). A wasm32 build has no readFileBytesSync, as it has no readFileSync. The TypeScript reading is class A (WP34 N2).

  • Tests: bytes_read, bytes_read_import, mem_read_bytes_scope; reject_bytes_read_unchecked, reject_bytes_read_arity (NL2061), reject_bytes_read_type (NL2268).

writeFileSync(path: string, data: string): void creates or truncates the file (0644) and writes; it is used in statement position. A symbolic link as the last component of path is refused (nish: cannot write <path>), where Node follows it.

  • Tests: io_files.

appendFileSync(path: string, data: string): void creates or appends and writes; it is used in statement position. A symbolic link as the last component is refused, as for writeFileSync.

  • Tests: io_files.

writeFileBytesSync(path: string, bytes: readonly u8[]): i32 creates or truncates the file (0644) and writes the array's bytes as they are. A symbolic link as the last component is refused (-40 ELOOP). Where writeFileSync exits, this answers bytes.length or a negative errno, never a panic.

  • Tests: io_write_file_bytes, io_write_bytes; reject_write_file_bytes_arity, reject_write_file_bytes_type (NL2268), reject_write_file_bytes_wasm.

renameSync(from: string, to: string): i32 is rename(2): from replaces to in one step, so a reader sees the old file or the new one and never neither. It answers 0, or a negative errno; -18 EXDEV across file systems, since nothing is copied. Never a panic.

  • Tests: io_rename, caps_fs_replace; reject_rename_arity, reject_rename_type, reject_rename_wasm.

unlinkSync(path: string): i32 is unlink(2): it removes the name, and a symbolic link itself rather than what it names. A directory is refused (-21 EISDIR on Linux, -1 EPERM where the kernel answers so), with no fallback to rmdir. It answers 0, or a negative errno. Never a panic.

  • Tests: io_unlink, caps_fs_replace; reject_unlink_arity, reject_unlink_type, reject_unlink_wasm.

writeFileSyncMode(path: string, data: string, mode: i32): i32 is writeFileSync with the file left at exactly mode. An fchmod(2) sets it before the first byte is written, so the umask cannot widen it and a wider existing file is narrowed before it holds the new contents. A descriptor opened before is not revoked. The file is written in place: a failure leaves it empty or partial, and a file this process may write but does not own is emptied before -1 EPERM, so a secret that replaces another goes through a temporary and renameSync. A symbolic link as the last component is refused (-40 ELOOP), its target untouched. A mode outside 0..0o777 is -22 (the setuid, setgid and sticky bits are refused, not applied). It answers the length written, or a negative errno. Never a panic.

  • Tests: io_write_file_mode, caps_fs_mode, fs_mode (modes read back by stat, umask 022 and 077); reject_write_file_mode_arity, reject_write_file_mode_type, reject_write_file_mode_wasm.

createFileExclusiveSync(path: string, data: string, mode: i32): i32 is O_CREAT|O_EXCL at exactly mode, set as for writeFileSyncMode. It answers -17 EEXIST when any name is already there, a dangling symbolic link included, which is never followed. Exactly one of any number of racing processes wins, so it is a lock file; a winner whose write fails removes the file before it answers. It answers the length written, or a negative errno. Never a panic.

  • Tests: io_create_exclusive, fs_mode (eight children race, one wins; a failed write leaves no file); reject_create_exclusive_arity, reject_create_exclusive_type, reject_create_exclusive_mode_type, reject_create_exclusive_wasm.

mkdirSyncMode(path: string, mode: i32): i32 is mkdir(2) of one level at mode, with the process umask applied. An existing name is -17 EEXIST, its mode never changed; a mode outside 0..0o777 (the sticky, setuid and setgid bits included) is -22. It answers 0, or a negative errno. Never a panic.

  • Tests: io_mkdir_mode, caps_fs_mode, fs_mode; reject_mkdir_mode_arity, reject_mkdir_mode_type, reject_mkdir_mode_mode_type, reject_mkdir_mode_wasm.

A file a window at a time

readFileBytesSync answers the whole file; these four read it a window at a time into a u8[] the program owns, so memory follows the window rather than the file, and a file past 2^31 − 1 bytes is readable (WP44, wp44-streaming-files.md). No call allocates. Every answer that is not a count is the negated Linux errno, on every platform, WASI and Node included: -2 not found, -6 a Unix-domain socket (ENXIO), -9 a bad descriptor, -13 permission denied, -20 a component is not a directory, -21 a directory, -22 not a regular file or a refused position, -23/-24 descriptors exhausted, -29 not seekable, -36 name too long, -40 too many symbolic links, and -5 for any other failure, WASI's ENOTCAPABLE among them. They are exported by nish:fs and global.

fileOpen(path: string): i32 opens a regular file for reading and answers its descriptor (>= 0), or a negative errno. It opens with O_CLOEXEC, so a spawned child does not inherit it, and O_NONBLOCK, so a FIFO with no writer cannot hang it. Anything that is not a regular file is closed again and refused: a directory is -21, a FIFO or a device -22, and a Unix-domain socket -6, as open(2) answers for one. An empty path, or one holding a NUL, is -2 before the file system is asked. Symbolic links are followed.

  • Tests: file_read_at, file_open_errors.

fileReadAt(fd: i32, buf: u8[], off: i32, len: i32, pos: i64): i32 reads at most len bytes of the file from byte pos into buf[off, off + len), with pread(2): there is no cursor, so one descriptor serves any number of readers. It answers the count, 0 at or past the end of the file, or a negative errno. A short read is allowed before the end: loop until 0, or until the size planned for is read. len of 0 answers 0. A pos below 0, or a pos + len past 2^63 − 1, is -22 before the kernel is asked. The window is checked as netRead's is, and buf is written, so a readonly u8[] is refused (Cannot `fileReadAt` through readonly u8[]) and another element type is NL2402, as netRead's are.

  • Tests: file_read_at, file_read_at_eof, file_read_at_pos, file_read_at_bounds, file_size_large, mem_file_stream_flat; reject_file_read_readonly, reject_file_read_element.

fileSize(fd: i32): i64 answers the file's size in bytes now, or a negative errno. The size can change: a file may shrink or grow between fileSize and the reads, so a reader that gets 0 before the size it planned for has a file that shrank, and a server that has sent Content-Length must close the connection there.

  • Tests: file_read_at, file_size_large.

fileClose(fd: i32): i32 closes the descriptor and answers 0 or a negative errno; either way the descriptor is gone. It is never retried. A call on a closed descriptor is -9, as is one on a negative descriptor (under Node too, where fs would throw a range error), but nothing checks a use after close: the number may already name another file the program opened since (recorded open, docs/security/runtime.md, RT-50).

  • Tests: file_read_at_pos.

All four are fs.read (capabilities; caps_file_read, reject_file_read_denied) and write effects, so none may be called in a scope() task or a parallel body (reject_file_fd_type for the descriptor's i32, reject_file_read_task). Under Node they read through fs.openSync, fs.readSync, fs.fstatSync and fs.closeSync, with the size a bigint. Under --profile wasi they read through WASI's path_open, fd_pread and fd_filestat_get, from the directories the host preopens. A freestanding wasm32 build has no file system to read, and a call is refused at check time: `fileOpen` reads a file, and a freestanding wasm32 build has no file system to read (build with `--profile wasi`) (NL2718; reject_file_open_wasm). The --profile wasm runtime defines the four answering -38 (ENOSYS), so a link that names them never fails.

The ones that answer an i32 answer the platform's errno values, untranslated, and the same numbers under Node; the ones shown here are Linux's, so compare against a negative result. A wasm32 build has none of these, and a WASI build answers -ENOSYS from the three that take a mode, as WASI has no file modes to set.

Bytes out, and the string route. A native string is {len, bytes}, never validated as UTF-8, and write, writeError, writeFileSync and appendFileSync send its len bytes as they are, so a string built by String.fromBytes from any bytes, all 256 values included, leaves exactly as it came in (tests/cases/io_write_bytes, compared with io_write_bytes.bin byte for byte). That route is native-only. Under the prelude (runtime/nish.mjs) String.fromBytes makes each byte one UTF-16 unit, which leaves as UTF-8, so a byte above 127 becomes two; under --emit-ts the bytes decode as UTF-8 and a byte that is not UTF-8 becomes U+FFFD. writeBytes and writeFileBytesSync send a u8[] as it is on every host, and are the route for bytes that are not text.

Paths are relative to the working directory. readFileSync, readFileSyncOrNull and readFileBytesSync treat a file longer than 2^31 − 1 bytes as one they cannot read, in both number modes, which matches Node. Every path, name and argument a builtin hands the OS that holds a NUL byte is refused as a missing path is — null, false, -1 or cannot …, whichever that builtin answers — where Node refuses it by throwing (docs/security/runtime.md, RT-1, RT-3, RT-4). These are globals, not import { readFileSync } from "fs" (bare imports are rejected), and so are write, writeError and panic above. A user function of the same name wins, as it does for every identifier builtin.

Directories and subprocesses

Reference: nish:fs, nish:process.

Function Takes Gives Effect
mkdirSync string boolean write
isDirectorySync string boolean write
spawnSync string[] number write
spawnSyncTo string[], string, string number write
spawnPiped string[], readonly u8[], u8[], u8[] i32 write
readdirSync string string[] | null write
realpathSync string string | null write

mkdirSync(path: string): boolean creates one directory, mode 0777 & ~umask, not recursive, exactly like Node's fs.mkdirSync(p) with no options. It answers true when a directory exists at path once the call returns, whoever made it; false for every other outcome, a plain file at path included.

  • Tests: io_mkdir; reject_mkdir_arity, reject_mkdir_type.

isDirectorySync(path: string): boolean does one stat: true when a directory exists at path as the call runs, false for everything else. It is the question -o <dir> asks of a path, and the stat half of mkdirSync, which calls it (WP14 §7a).

  • Tests: io_is_directory; reject_is_directory_arity, reject_is_directory_type.

spawnSync(argv: string[]): number runs argv[0], searched on PATH, and waits for it. It answers its exit status, or 128 + n when signal n killed it (the shell's convention). It answers -1 for an empty vector, a child that cannot be started or waited for, and always on WASI, which has no processes. The child inherits this process's environment, streams and working directory.

  • Tests: io_spawn; reject_spawn_arity, reject_spawn_element_type.

spawnSyncTo(argv: string[], stdoutPath: string, stderrPath: string): number is the same run, with each stream sent to a file created or truncated at 0644. An empty path leaves that stream inherited, and a symbolic link as a path's last component is refused, which answers -1. The two paths must not name one file; capture them apart and concatenate.

  • Tests: io_spawn_to, io_spawn_to_inherit; reject_spawn_to_arity, reject_spawn_to_type.

spawnPiped(argv: string[], stdin: readonly u8[], stdout: u8[], stderr: u8[]): i32 is the same run, with every stream a pipe: the child reads stdin and then end of file, and its stdout and stderr are appended to stdout and stderr. It answers the status, 128 + n, or -1 as for spawnSync (and for a failed pipe or poll). A stream past 16 MiB (NISH_PIPED_CAP) kills the child with SIGKILL and answers -2. The child holds no descriptor of the program but its three streams, and the program's SIGPIPE from a child that stopped reading is absorbed. The two output arrays are grown by the runtime, so they are push receivers to the arena analysis, and readonly is refused for them, as is a typed-array spelling (Int32Array, ...), whose length is fixed.

  • Tests: io_spawn_piped, mem_spawn_piped_scope; reject_spawn_piped_arity, reject_spawn_piped_element_type, reject_spawn_piped_stdin_type, reject_spawn_piped_output, reject_spawn_piped_typed_output, reject_spawn_piped_wasm, reject_using_arena_spawn_piped.

spawnPipedWithin(argv: string[], stdin: readonly u8[], stdout: u8[], stderr: u8[], timeoutMs: i32): i32 is spawnPiped's run with a deadline timeoutMs milliseconds after the call, on the monotonic clock; a negative timeoutMs is none, and the run is spawnPiped's. It gives every answer spawnPiped gives, and -3 when the deadline passes first: the child's pid is killed with SIGKILL and reaped, and the bytes read before then stay in stdout and stderr. The deadline covers the whole run, the wait for the child after its streams close included, so a child that closes its output and keeps running is -3 too. Only the child's own pid is killed, never a process group: a grandchild still holding an output pipe keeps the run going until the deadline, which turns an exit 0 into -3, and is left running. The arguments are checked as spawnPiped's, the fifth as an i32.

  • Tests: io_spawn_piped_within; reject_spawn_piped_within_arity, reject_spawn_piped_within_element_type, reject_spawn_piped_within_timeout_type, reject_spawn_piped_within_output, reject_spawn_piped_within_typed_output, reject_spawn_piped_within_wasm, reject_using_arena_spawn_piped_within.

readdirSync(path: string): string[] | null answers the directory's entries, sorted ascending by bytes, without . and ... It answers null when the directory cannot be read at all, a different answer from the empty array of an empty directory. A WASI build always answers null: fd_readdir lists a preopened directory rather than a path, which is a different contract and not a port of this one.

  • Tests: io_readdir, io_readdir_null; reject_readdir_arity, reject_readdir_type, reject_readdir_unchecked.

realpathSync(path: string): string | null answers path resolved to an absolute path with no symbolic link, ., .. or repeated separator, or null when it does not resolve. POSIX realpath needs every component but the last to exist, so "does not resolve" is one answer rather than several, and the answer the caller asked for rather than an error. The result is copied into the arena and narrowed with if (p !== null) like any other nullable (WP19 §5a item 4).

  • Tests: io_realpath; reject_realpath_arity, reject_realpath_type.

All five answer a value where they could have exited, for the reason readFileSyncOrNull answers null (WP14 B3): the language has no exceptions, so a driver has to be able to turn the failure into its own diagnostic. They exist so that a compiler written in Nish can create its own -o dir/, tell -o out meaning a directory from -o out meaning a file, and shell out for --link — mkdirSync and spawnSync are the calls wp14-selfhost.md §3a D4 named as the price of a self-hosted link step, and isDirectorySync is the one §7a named as the price of the last spelling of -o that, at the time, only the TypeScript compiler handled.

spawnSync and spawnSyncTo are the two builtins whose pointer argument the runtime keeps: the spawn path copies each element's bytes pointer into a vector that outlives the call, so the array escapes, a parameter passed to either is never nocapture, and an array literal handed to one is never stack-allocated (io_spawn pins all three). Nothing that can reach either is willreturn either, because the child may never exit. The argument is checked down to its element type: an i32[] is an array but not a command line (reject_spawn_element_type). Arguments are passed as bytes, so an embedded NUL truncates one.

spawnSyncTo is a second builtin rather than two more parameters on the first because the language had no default parameters when it was added — and the alternative shape, answering the output as a string, was refused for two reasons: the runtime would have to buffer a child that can print more than memory holds, and a harness comparing output against a golden wants the bytes on disk anyway, to diff and to keep after a failure.

spawnPiped is the run whose streams stay in memory, for a child that takes a secret: a password manager's or keychain's CLI reads it on stdin, and stdin is the one place ps, /proc/<pid>/environ and the file system do not show. Its shape is four arguments with the outputs as out-parameters, the way readBytesInto fills a caller's array. The rejected shape answered a record ({ status, stdout, stderr }): no builtin returns a class, and there is no way to declare one the checker, both emitters and the TypeScript prelude would share, so the record would have been the first and its own construct. The out-parameters cost a caller two const out: u8[] = [] lines; they also let a loop reuse one pair of arrays. The runtime writes stdin and drains both outputs in one poll loop, so a child that fills one pipe while the parent waits on another cannot deadlock it, and the 16 MiB cap is the answer to the reason spawnSyncTo gave for refusing to buffer: a child that prints more than memory holds is stopped, with a code no exit status can be, rather than followed (docs/security/runtime.md, RT-27 to RT-31).

spawnPipedWithin is the same run for a child that may never answer: a credential helper waiting on a prompt nobody sees, or a tool hung on the network. It is a fifth argument on a second builtin rather than an optional one on spawnPiped, so spawnPiped's IR and every program that calls it are unchanged; -3 is a code of its own because a caller tells a slow child from a failed one (the deadline was the program's choice, so it may retry with a longer one). The deadline is a point in time rather than a budget per wait: the poll loop's waits and the final wait for the child all end at it, so the whole call returns within timeoutMs plus the time a kill and a reap take. It kills the pid and not a process group, because putting the child in a group of its own would take it out of the terminal's job control, and the program cannot know the child would not mind (docs/security/runtime.md, RT-49).

spawnWithFds numbers the descriptors from 3 rather than taking a target for each, because 3 onward is where a child that is handed descriptors looks for them (systemd's socket activation, LISTEN_FDS), and a target list would let a program aim one at 0 to 2 and lose the child's streams. The runtime duplicates each source above the targets before the child's dup2 reads it, so an entry that is its own target or another's is never overwritten first (docs/security/runtime.md, RT-40).

readdirSync sorts, which neither readdir(3) nor Node's fs.readdirSync does, and that is a decision about this language rather than a convenience: every caller of an unsorted listing would have to sort it before reading it, and a run whose order is the file system's is a run that differs between two machines. The sort is by bytes, like every other string comparison here, so it agrees with Node's .sort() for ASCII names and can differ for names that are not (docs/wp13-differential.md).

The environment

Reference: nish:process.

Function Takes Gives Effect
getenv string string | null write

getenv(name: string): string | null answers the value of environment variable name as the call runs, or null when it is unset; a variable set to nothing (CC=) answers a zero-length string. The bytes are copied into the arena, so the result outlives any later change to the environment. It is narrowed with if (cc !== null) like any other nullable (WP19 R1).

  • Tests: io_getenv; reject_getenv_arity, reject_getenv_type, reject_getenv_unchecked.

It is a call and not process.env.CC, and that is a language decision rather than a spelling: process.env.NAME is member access on a key chosen at runtime, which Phase 0 forbids outright (wp19-stage0-retirement.md §4). The two process.* values the language does have — process.argv, and process.platform / process.arch below — are fixed names with no key.

null and "" are different answers, and a driver acts on the difference: an unset CC means "use the default", and CC= means someone set it to nothing. That is why the result is string | null and not a string that is empty when nothing is there.

There is no setenv: a program can read its environment and pass an environment on to a child through spawnSync, which inherits this process's, but it cannot change its own. Nothing in the compiler needs to, and a setter would make getenv's "as the call runs" a much sharper caveat than it is. Because of that, a test that depends on a variable is given one by the harness — tests/cases/<name>.env, one NAME=value per line, read by both tests/run.js and the differential runner so the native binary and the Node rewrite see the same environment.

Two reads of one variable in a function are two calls: the declaration is not readonly, because a setenv from linked C — this program cannot call one, but a library linked beside it can — may change the answer between them, so LLVM may not fold the second read into the first.

The value is what getenv(3) answers, so it is bytes, and every offset into it is a byte offset.

The clock

Reference: nish:process.

Function Takes Gives Effect
monotonicNanos nothing i64 write
wallClockNanos nothing i64 write

monotonicNanos(): i64 reads CLOCK_MONOTONIC in nanoseconds. The origin is arbitrary, so only the difference between two reads means anything.

  • Tests: io_monotonic; reject_monotonic_arity.

wallClockNanos(): i64 reads CLOCK_REALTIME in nanoseconds since 1970-01-01 UTC, as an i64. It can go backwards when the system time is set, so a duration is monotonicNanos's job. It is not refused on wasm, as the monotonic clock is.

  • Tests: wall_clock; reject_wall_clock_arity.

It is an i64 in both number modes and not a number, because a number is an i32 by default and would overflow inside a tenth of a second, and an f64 loses nanoseconds after about 104 days of uptime.

It is the monotonic clock and not the wall clock, and the type cannot say so, so the name does. An elapsed time measured from a wall clock has to survive that clock being corrected mid-run, and it cannot: the difference can come out negative. The wall clock is Date.now(), below, and it is the only part of Date there is: a program that formats a date needs a calendar, a time zone database and a locale, none of which the language has. The ISO calendar is a library instead: nish/temporal is TC39 Temporal's dates, times, instants and durations, with no zone but UTC yet (nish/temporal).

The effect is write rather than none for the reason getenv is not readnone: the clock is not memory LLVM is tracking, so two reads with work between them must not fold into one. A readnone clock would make every measured interval exactly zero, which io_monotonic is the test for.

The host: the wall clock, entropy, file times and signals

Four facts a program that runs for days asks of the machine (WP34 N3): what time it is, bytes nobody can predict, whether a file was renewed, and whether it has been asked to stop. All four are C in runtime/runtime-host.c, a translation unit of its own with its own .text* ceiling, which scripts/build.sh compiles beside runtime.c as it does the others.

Reference: nish:fs, nish:process, and the globals page for Date.now (under DateConstructor) and crypto.

Function Takes Gives Effect
Date.now nothing f64 write
crypto.getRandomValues u8[] void write
statMtimeSync string f64 write
signalFd nothing i32 write
readSignal i32 i32 write

Date.now(): f64 answers the wall clock, CLOCK_REALTIME, in whole milliseconds since 1970-01-01 UTC, as JavaScript answers it. The milliseconds are counted as integers and then converted, so the value is exact and a time before 1970 floors. It is an f64 in both number modes, because an i32 of milliseconds overflowed in January 1970. It can go backwards when the clock is corrected, so an interval is monotonicNanos's job, not this one's.

  • Tests: os_now, os_now_i32; reject_os_date_now_arity.

crypto.getRandomValues(bytes: u8[]): void fills every byte of bytes from the kernel's CSPRNG. It takes a u8[] only, where the Web API takes any integer typed array: the byte is the unit a key or a nonce is written in, and a wider element would make the answer's byte order a fact about the machine. It is a statement, where the Web API also answers its argument, which the caller already holds. An empty array is filled with nothing. It fills at most 65,536 bytes a call, the Web API's limit: more is a panic, crypto.getRandomValues: <n> bytes asked for, and one call fills at most 65536, exit 1. A source that fails is a panic too; nothing weaker is ever substituted.

  • Lowering: getrandom(2) on Linux, looping over EINTR and short reads, and getentropy(3) in 256-byte chunks on macOS.
  • Tests: os_random, os_random_limit; reject_os_random_i32, reject_os_random_u32, reject_os_random_string, reject_os_random_readonly, reject_os_random_value, reject_os_random_arity, reject_os_crypto_member, reject_os_crypto_value, reject_thread_region_random.

statMtimeSync(path: string): f64 answers the modification time in milliseconds since the epoch, with the sub-millisecond fraction kept. It is computed as Node's mtimeMs is (seconds × 1e3 plus nanoseconds ÷ 1e6, in doubles), so the two print the same digits. It answers NaN when the path cannot be stat'd, because an f64 has no null (Nullable types); test it with m !== m. stat follows a symbolic link, and a directory has a time of its own. Also exported by nish:fs.

  • Tests: os_mtime, os_mtime_import; reject_os_mtime_type, reject_os_mtime_arity, reject_os_mtime_nullable.

signalFd(): i32 answers a descriptor that becomes readable when SIGTERM or SIGINT arrives: the read end of a pipe a sigaction handler writes one byte to per signal, so the signals' default action never runs. The handler runs in whichever thread the signal lands on, so a scope() task or a parallel worker running when signalFd() is called cannot take the signal and end the process, and nothing is blocked for a spawned child to inherit. It overrides a disposition the process started with, as process.on("SIGTERM") does under Node. It is made once; -1 when it cannot be made. Also exported by nish:process.

  • Tests: os_signal, os_signal_import; reject_os_signal_fd_arity.

readSignal(fd: i32): i32 blocks until one of the two signals arrives and answers 15 or 2; -1 for an fd that is not signalFd()'s and for a read that fails. The argument is an i32 in both number modes, and an f64, an i64 or a string is refused rather than converted. Also exported by nish:process.

  • Lowering: not willreturn.
  • Tests: os_signal, os_signal_import; reject_os_signal_f64, reject_os_signal_i64, reject_os_signal_string, reject_os_signal_arity.

Date has one member. new Date(), Date(), Date.parse, Date.UTC, any other member, and Date or Date.now used as a value are all refused under one rule: `new Date()` is refused: Nish has no `Date` object; the one `Date` member is `Date.now()`, the wall clock in milliseconds (NL2401; reject_os_date_new, reject_os_date_call, reject_os_date_parse, reject_os_date_utc, reject_os_date_value, reject_os_date_bare). A local named Date shadows the global as any local does, so calling it is the ordinary Unknown function `Date` (reject_os_date_shadowed). crypto has one member the same way: `crypto.randomUUID` is refused: the one `crypto` member is `crypto.getRandomValues(bytes)` (NL2403). The array getRandomValues fills is written, so a readonly u8[] is refused as a store through it is, a parameter it fills is never readonly in the IR, and inside a scope's region it is refused on an array a task may read, as fill is. Anything but a u8[] is `crypto.getRandomValues` fills a `u8[]`, got i32[] (NL2402).

A descriptor, not a handler. A handler would be a callback the runtime calls, which a function reference is not, and would run between any two instructions of the program. A descriptor is what a server's loop can wait on beside its sockets (WP34 N5 will add that wait); until then readSignal is the blocking read. Underneath it is a handler rather than a Linux signalfd, because a signalfd hears only a signal blocked in every thread, and a mask reaches only the thread that sets it. A child that spawnSync or spawnSyncTo starts after signalFd() can still be stopped by either signal, because exec resets a caught signal to its default (os_signal). Signals other than SIGTERM and SIGINT are not in the language.

On wasm. A wasm32 target and the wasi profile have no host for these to reach, and runtime-host.c is empty there, so all five are refused by the checker: `Date.now` reaches the operating system, and a wasm32 build has none to reach (NL2404; reject_os_now_wasm, reject_os_random_wasm, reject_os_mtime_wasm, reject_os_signal_fd_wasm, reject_os_read_signal_wasm).

The TypeScript reading (wp33-round-trip.md §2). Date.now is class A: Node's is the same floored CLOCK_REALTIME, and the os_ block of tests/run.js holds both readings between two reads of Node's own clock. statMtimeSync is class A under runtime/nish.mjs, which answers fs.statSync(path, { throwIfNoEntry: false })?.mtimeMs, or NaN: the os_ block sets a file's mtime to a fraction of a millisecond and requires the native build and the prelude both to print Node's mtimeMs for it. crypto.getRandomValues is class A under the prelude and class C without it: Node's own takes only a typed array and a u8[] is a plain Array there, so the prelude replaces the method on Node's crypto with one that fills a plain array too and panics past 65,536 with the native words; os_random and os_random_limit run unmodified under it. signalFd and readSignal are class C, with no synchronous reading at all: Node delivers a signal only to its event loop, which a blocking readSignal never returns to, so the prelude's versions throw rather than answer. The row in wp33-round-trip.md §3.5 gives the translation.

Children and the machine: spawn, kill, waitPid, availableParallelism, totalMemory

What a supervisor asks of the host (WP34 N12): start a child without waiting for it, stop it, reap it when it ends, and size itself to the machine it runs on. All five are C in runtime/runtime-os.c, beside spawnSync, and each is exported by nish:process as well as being a global.

Reference: nish:process.

Function Takes Gives Effect
spawn string[] i32 write
spawnWithFds string[], readonly i32[] i32 write
kill i32, i32 i32 write
waitPid i32, i32 i32[] write
availableParallelism nothing i32 none
totalMemory nothing i64 none

spawn(argv: string[]): i32 starts argv[0] exactly as spawnSync does and answers the child's pid at once, without waiting, or a negative errno (-22 for an empty vector or an argument holding a NUL). Test pid > 0 before handing the answer on: a failure passed to waitPid reads as "any child". A child never reaped stays a zombie until the program exits.

  • Tests: os_spawn; reject_os_spawn_type, reject_os_spawn_wasm.

spawnWithFds(argv: string[], fds: readonly i32[]): i32 is spawn, with the child holding fds[i] as descriptor 3 + i, close-on-exec cleared there. No other descriptor of this process reaches it but one opened without close-on-exec, which nothing the runtime opens is. More than 64 descriptors, or a negative one, is -22 before anything is opened or started. The child shares each descriptor's open file, offset and status flags, and this process keeps its own. It answers its pid, or a negative errno.

  • Tests: spawn_fds_child, caps_process_spawn_fds; reject_spawn_fds_arity, reject_spawn_fds_element, reject_spawn_fds_argv, reject_spawn_fds_wasm.

kill(pid: i32, sig: i32): i32 sends sig to pid and answers 0, or a negative errno. It takes only 0, 2 (SIGINT), 9 (SIGKILL) and 15 (SIGTERM), whose numbers are the same on Linux and Darwin; any other sig is -22, and so is a pid of 0 or below, which kill(2) reads as more than one process. A positive pid may be any process of this user, not only a child (RT-19).

  • Tests: os_kill; reject_os_kill_f64, reject_os_kill_wasm.

waitPid(pid: i32, flags: i32): i32[] reaps a child (pid a child's or -1 for any child; flag 1 is WNOHANG) and answers a fresh [pid, status], the status as spawnSync's; a failure puts the negative errno first. After signalFd() a blocking wait never sees EINTR, because that handler is installed with SA_RESTART and the wait restarts. It is an array because the language has no tuple.

  • Tests: os_wait, os_spawn, os_kill; reject_os_wait_arity, reject_os_wait_wasm.

availableParallelism(): i32 answers the CPUs this program may use, at least 1: the smallest of the CPUs online, its affinity mask or cpuset, a cgroup v2 CPU quota rounded up, and a positive NISH_NUM_THREADS, which can lower the count and never raise it. It needs no --threads.

  • Tests: os_cpu; reject_os_cpu_arity, reject_os_cpu_wasm.

totalMemory(): i64 answers bytes of physical memory, or -1 where the host does not say. It is the machine's, not a cgroup's limit.

  • Lowering: _SC_PHYS_PAGES times the page size on Linux, hw.memsize on Darwin.
  • Tests: os_memory; reject_os_memory_wasm.

Capabilities. spawn, kill and waitPid carry process.spawn, the label spawnSync carries: each reaches other processes — kill any process of this user, as kill(1) does, not only a child spawn started — so none takes a label of its own (caps_process_wait). availableParallelism and totalMemory carry none: how many cores and bytes the machine has is a fact about the host, as process.platform is, read to size a program rather than to reach anything (caps_host).

In a task. spawn, kill and waitPid write state outside the program, so a scope's task or a parallel body that calls one — under any import alias — is refused as any shared write is (reject_os_spawn_task, reject_os_kill_task, reject_os_wait_task, reject_os_kill_task_alias, reject_os_kill_parallel). availableParallelism and totalMemory write nothing and may be called there (tests/link/thread_host_facts_task).

On wasm. A wasm32 target has no processes and no host to ask, so all five are refused by the checker with the N3 rule's words: `spawn` reaches the operating system, and a wasm32 build has none to reach (NL2404; reject_os_spawn_wasm, reject_os_kill_wasm, reject_os_wait_wasm, reject_os_cpu_wasm, reject_os_memory_wasm).

The TypeScript reading. kill, availableParallelism and totalMemory are class A under runtime/nish.mjs: kill checks the signal and the pid as the native one does and turns Node's error code back into its errno; availableParallelism is os.availableParallelism() lowered by NISH_NUM_THREADS (Node reads no cgroup quota, so inside a container with one the two can differ); and totalMemory is os.totalmem(), a bigint as every i64 is there. spawn and waitPid are class C: Node reaps its children itself, on the event loop, so a synchronous wait has nothing to find, and the prelude's versions throw, naming themselves, rather than answer (the os_ block of tests/run.js).

Scheduling: setNice, setIdlePriority, setCpuAffinity

How much of the CPU the program asks for, and where: a batch job or a background indexer steps aside for the interactive work on the machine, and a prefork worker keeps to one core. Each is C in runtime/runtime-os.c, and each is exported by nish:process as well as being a global.

Reference: nish:process.

Function Takes Gives Effect
setNice i32 i32 write
setIdlePriority nothing i32 write
setCpuAffinity i32 i32 write

setNice(n: i32): i32 sets the nice value with setpriority(PRIO_PROCESS, 0, n) and answers 0, or a negative errno. It is lower-only, even as root: outside 0..19 is -22 and below the current value is -13, both with no system call, so the priority never rises. Children started afterwards inherit it.

  • Tests: os_nice; reject_os_nice_wasm, reject_os_nice_task.

setIdlePriority(): i32 moves the program to SCHED_IDLE with sched_setscheduler(0, SCHED_IDLE, …) and answers 0 or a negative errno. It is Linux only: Darwin answers -95 (Linux's EOPNOTSUPP, as nish:net spells it). An unprivileged process can never leave SCHED_IDLE, and an idle process can starve on a saturated machine, so a caller keeps a normal-priority fallback for work that must finish.

  • Tests: os_nice; reject_os_idle_wasm.

setCpuAffinity(cpu: i32): i32 pins the calling process to the one CPU cpu with sched_setaffinity and answers 0, or a negative errno. It is narrow-only: a cpu outside the CPUs the process may already run on (taskset, a cgroup) is -22 with no change. availableParallelism() answers 1 afterwards. It is Linux only: -95 on Darwin and under Node.

  • Tests: link/net_prefork_pin; reject_os_affinity_wasm, reject_os_affinity_task.

Per thread. Linux keeps both the nice value and the policy per thread: a call sets the calling thread's and every thread started after it. So both belong at the top of main, before a scope() or a parallel loop starts the pool, and a task or a parallel body that calls one is refused as any shared write is (reject_os_nice_task). Neither ever crashes the program: a failure is only the answer.

Capabilities. Both carry process.spawn, the label kill carries: neither reaches another process, but the priority is the one the program's children inherit, and a step down cannot always be taken back.

On wasm. A wasm32 target schedules nothing of the host's, so both are refused by the checker with the N3 rule's words (NL2404; reject_os_nice_wasm, reject_os_idle_wasm).

The TypeScript reading. Both are class C under runtime/nish.mjs: Node has os.setPriority but no SCHED_IDLE, and a reading of one without the other would differ from the native build on the same host, so both throw, naming themselves.

nish:net: addresses, non-blocking TCP and UDP, and the readiness loop

Sockets for a program that owns its loop (WP34 N5): a server listens, accepts and reads and writes bytes, a client connects, or either binds and trades datagrams, and every socket is non-blocking, so a call that would wait answers "would block" instead, and the program decides what to do next. There is no callback, and an async function cannot wait on a socket yet (its tasks run on an in-process queue): the program is the loop, and it waits in pollWait for whichever of its descriptors is ready. Every socket is also close-on-exec, so a child that spawnSync starts inherits none of them. All twenty-six functions are C in runtime/runtime-net.c, a translation unit of its own with its own .text* ceiling, which scripts/build.sh compiles beside runtime.c as it does the others; section GC leaves every one of them out of a program that calls none (the net_ block of tests/run.js links one and counts the symbols). Each is exported by nish:net and is a global too, as every nish: export is.

Reference: nish:net.

Function Takes Gives Effect
netAddress u8[], string, i32 i32 write
netLocalPort i32 i32 write
tcpListen string, i32, i32 i32 write
tcpListenFlags string, i32, i32, i32 i32 write
tcpAccept i32, u8[] i32 write
netRead i32, u8[], i32, i32 i32 write
netWrite i32, readonly u8[], i32, i32 i32 write
netWrite2 i32, two ranges i32 write
netShutdown i32, i32 i32 write
netClose i32 i32 write
netSetOption i32, i32, i32 i32 write
netMss i32 i32 write
unixListen string, i32, i32 i32 write
unixListenMode string, i32, i32, i32 i32 write
unixConnect string i32 write
unixAccept i32 i32 write
unixSocketPair i32, i32[] i32 write
sendFd i32, i32, readonly u8[], i32, i32 i32 write
recvFd i32, u8[], i32, i32, i32[] i32 write
peerCredentials i32, i32[] i32 write
netRingState none i32 write

netAddress(out: u8[], host: string, port: i32): i32 writes the address form of a numeric host and port into out. It answers 0, or -22, leaving out as it was. No nish:net call looks a name up; nish/net/dns resolves it first.

  • Tests: net_tcp_calls, net_tcp_import; reject_net_unknown_export.

netLocalPort(fd: i32): i32 answers the port the socket is bound to: the one the kernel chose for a listen on port 0.

  • Tests: net_tcp_calls, net_tcp_echo.

tcpListen(host: string, port: i32, backlog: i32): i32 answers a listening socket on a numeric host with SO_REUSEADDR. "::" hears both families, and on a kernel without IPv6 it is IPv4's 0.0.0.0. It answers -98 when the port is in use.

  • Tests: net_tcp_calls, net_tcp_echo, net_dual_stack; reject_net_arity, reject_net_wasm.

tcpListenFlags(host: string, port: i32, backlog: i32, flags: i32): i32 answers tcpListen's socket, with flags. 1 is SO_REUSEPORT, which differs by platform as the table below shows. 2 is TCP_NODELAY, which the runtime sets on every accepted connection, on every platform, closing one on which it cannot. 4, with 1, steers each connection to the listener of the CPU that received it, Linux only (-95 elsewhere, -22 without 1). 8 (NET_LISTEN_RING) accepts into io_uring slots on Linux, as slot handles say, and does nothing elsewhere. Any other bit is -22 before a socket is made.

  • Tests: net_tcp_reuseport, net_tcp_nodelay, net_ring_calls, link/net_tcp_reuseport_spread, link/net_tcp_nodelay_shared; reject_net_reuseport_wasm, reject_net_nodelay_wasm.

tcpAccept(fd: i32, peer: u8[]): i32 answers the next waiting connection, non-blocking and close-on-exec, with the peer's address in peer. It answers -11 when none is waiting, and a peer shorter than 18 bytes is -22 before a connection is taken.

  • Lowering: not willreturn: on a blocking socket the program inherited it waits.
  • Tests: net_tcp_calls, net_tcp_echo; reject_net_accept_readonly.

netRead(fd: i32, buf: u8[], off: i32, len: i32): i32 reads at most len bytes into buf[off, off + len). It answers the count, 0 at the end of the stream, or -11 when nothing is waiting. The range is checked.

  • Tests: net_tcp_echo, net_tcp_bounds; reject_net_readonly, reject_net_element, reject_net_import_wasm.

netWrite(fd: i32, buf: readonly u8[], off: i32, len: i32): i32 writes at most len bytes of buf[off, off + len). It answers the count, which may be short. A peer that has gone is -32, never SIGPIPE, so a closed connection cannot end the process. The range is checked.

  • Tests: net_tcp_echo, net_tcp_bounds; reject_net_write_element.

netWrite2(fd: i32, a: readonly u8[], aOff: i32, aLen: i32, b: readonly u8[], bOff: i32, bLen: i32): i32 writes two ranges in one system call, answered as netWrite is, -32 and never SIGPIPE included. Both ranges are checked, a's first. As for every call here, a wasm32 build refuses it and Node throws.

  • Lowering: sendmsg with two iovecs.
  • Tests: tests/link/net_write2, net_write2_bounds; reject_net_write2_element, reject_net_write2_element_a, reject_net_write2_wasm.

netShutdown(fd: i32, how: i32): i32 shuts the read side (0), the write side (1) or both (2).

  • Tests: net_tcp_calls.

netClose(fd: i32): i32 closes the descriptor, which is gone after the call whatever the answer.

  • Lowering: not willreturn, because a socket the program inherited with a linger time can wait.
  • Tests: net_tcp_calls, net_tcp_echo; reject_net_fd_type.

netSetOption(fd: i32, option: i32, value: i32): i32 sets one socket option from a whitelist. Any other option is -22, so there is no way to pass a raw level and name through. NET_TCP_QUICKACK is Linux's, -95 elsewhere.

  • Tests: net_set_option; reject_net_set_option_wasm.

netMss(fd: i32): i32 answers the maximum segment size a connected TCP socket sends, or a plain -1, never an errno, for every other descriptor and every call on wasm. It never panics.

  • Lowering: Linux's TCP_INFO tcpi_snd_mss, or TCP_MAXSEG where that is not above 0 and on Darwin.
  • Tests: net_mss; reject_net_mss_arity, reject_net_mss_type; net_mss in tests/run.js for wasm32 and WASI.

unixListen(path: string, backlog: i32, flags: i32): i32 answers a listening Unix-domain stream socket at path, whose file only its owner may connect to (0600). Its paths and refusals are under that heading.

  • Tests: net_unix_calls, net_unix_paths, net_unix_fds; reject_net_unix_arity, reject_net_unix_wasm.

unixListenMode(path: string, backlog: i32, flags: i32, mode: i32): i32 answers unixListen's socket with the file's mode chosen, 0 to 0o777, set before the listen so no connect gets in under a wider one. An abstract name is -22.

  • Tests: net_unix_paths, net_unix_calls.

unixConnect(path: string): i32 answers a new socket, non-blocking and close-on-exec, connected to the listener at path. A full backlog is -11 on Linux and -111 on Darwin.

  • Tests: net_unix_calls, net_unix_paths; reject_net_unix_path_type.

unixAccept(fd: i32): i32 answers the next waiting connection, non-blocking and close-on-exec; no peer address is written.

  • Tests: net_unix_calls.

unixSocketPair(kind: i32, fds: i32[]): i32 writes two connected sockets into fds, non-blocking and close-on-exec, of kind 1 (NET_UNIX_STREAM) or 2 (NET_UNIX_SEQPACKET). Kind 2 is -95 on Darwin.

  • Tests: net_unix_calls, net_unix_fds; reject_net_unix_pair_element, reject_net_unix_pair_readonly.

sendFd(sock: i32, fd: i32, buf: readonly u8[], off: i32, len: i32): i32 sends bytes over the Unix socket sock with one copy of fd riding on them (SCM_RIGHTS). A len of 0 is -22, a socket of another family -95, and a gone peer -32, never SIGPIPE. The range is checked.

  • Tests: net_unix_scm, net_unix_scm_bounds; reject_net_unix_scm_arity, reject_net_unix_scm_wasm.

recvFd(sock: i32, buf: u8[], off: i32, len: i32, fd: i32[]): i32 receives bytes from the Unix socket sock, with the first descriptor the message carried in fd[0], or -1; every further one is closed before the answer. The one kept is close-on-exec and not made non-blocking. The range is checked.

  • Tests: net_unix_scm, net_unix_scm_bounds; reject_net_unix_scm_element, reject_net_unix_scm_readonly, reject_net_unix_scm_buf_readonly, reject_net_unix_scm_wasm.

peerCredentials(fd: i32, out: i32[]): i32 writes the pid, uid and gid of the process at the other end of a connected Unix socket, as the kernel recorded them when the connection was made, not who holds the other end now. A socket with no peer is -107 on every platform. On Darwin the pid is -1 where the SDK has no LOCAL_PEERPID or the kernel refuses it.

  • Tests: net_unix_scm, caps_net_peer; reject_net_unix_peer_arity, reject_net_unix_peer_readonly, reject_net_unix_peer_element, reject_net_unix_scm_wasm.

netRingState(): i32 answers whether NET_LISTEN_RING listeners accept into rings: 1 when they do, 0 until a listener has found rings or a refusal, or the negative errno the kernel refused rings with (-1 when a seccomp filter refuses io_uring_setup, -38 for a kernel without it), and then they serve descriptors. The first listener that finds rings or a refusal decides for the process; one whose rings fail for want of descriptors or memory serves descriptors itself, and a later listener asks again. It is -95 on Darwin.

  • Tests: net_ring_calls, link/net_ring_fallback; reject_net_ring_arity, reject_net_ring_wasm.

Slot handles

A connection tcpAccept takes from a NET_LISTEN_RING listener on Linux is a slot handle: a number at or above 2^20 that names a slot in one of the process's io_uring rings, not a descriptor, so a process holds connections past its RLIMIT_NOFILE. Its peer is written as zeros.

  • netRead, netWrite, netWrite2, netClose, netLocalPort, pollAdd, pollModify, pollRemove and pollWait take a slot handle as they take a descriptor, with the same readiness contract: -11 until pollWait reports it, then progress. A slot is in one loop at a time: a pollAdd to a second loop answers -17 until the first loop removes it or is closed.

  • netShutdown shuts a slot down through the ring, behind the bytes already written: the peer reads them and then the end of the stream. After a shut writing side (how 1 or 2) netWrite answers -32; after a shut reading side (how 0 or 2) netRead answers what is queued and then 0. A how past 2 answers -22.

  • netMss answers -1: the ring has no route to TCP_MAXSEG.

  • netSetOption answers -95: the slot's socket belongs to the ring. Every other call that takes a descriptor (sendFd, peerCredentials, FFI) answers -9, and so does every call on a handle after its netClose.

  • tcpAccept answers -23 when every slot its listener may hold is taken: up to 64 rings, each of as many slots as the soft descriptor limit, at most 16,384. A slot freed by netClose takes a waiting connection again.

  • A ring listener whose accept finds memory short (-12, -105) answers that errno from tcpAccept once, as a descriptor does, pauses accepting for 10 ms, doubling to 640 ms while it recurs, and then accepts again. Waiting connections stay in the listener's backlog meanwhile. Any other errno but -11, -4, -103 or -71 means the listener is broken: it stops accepting and tcpAccept answers that errno from then on. Close the listener and listen again.

  • A slot holds at most 256 KiB of bytes the kernel has not yet sent: a netWrite past that copies what fits, and a full slot answers -11 until the peer reads. Its received bytes queue up to the same bound before the runtime stops receiving for it.

  • A slot whose connection has failed, a peer's reset or a failed send, answers netWrite and netWrite2 with the errno, -104 for a reset. netRead answers it once the queued bytes are read, and on every read after, where a descriptor answers it once and then 0.

  • Where the kernel refuses rings, the listener serves descriptors and netRingState answers why; a program written against the readiness contract runs the same either way.

  • Tests: link/net_ring_beyond_limit, link/net_ring_full, link/net_ring_sink, link/net_ring_flood, link/net_ring_halfclose, link/net_ring_starve, link/net_ring_reset, link/net_ring_transient, link/net_ring_fallback.

What tcpListenFlags's flag 1 does on each platform:

Platform Flag 1
Linux Listeners that share the port each get new connections, spread by a hash of the connection's 4-tuple; only listeners of the same user may share
Darwin Listeners share the port, but one of them takes the connections: the flag buys a restart without a gap rather than a spread; flag 4 is -95
wasm32, --profile wasi Refused at compile time with NL2404, as every nish:net call is

Errors are numbers. Every call answers an i32: >= 0 is success (a descriptor, a count, 0), and a negative number is the failure's errno, negated. The codes a loop branches on are Linux's numbers on every platform: -11 would block, -95 unsupported, -32 the peer has gone, -104 the connection was reset, -98 the address is in use, -111 the connection was refused, -110 it timed out, -88 not a socket and -22 a bad argument; the Darwin runtime translates its own. Any other failure is the host's -errno (-9 for a descriptor that is not open, on both). -95 is what tcpAccept answers for a datagram socket, on both (net_udp_calls). A number rather than a Result, because no builtin answers one and a failure here is routine, and because a number allocates nothing. It follows the shape signalFd set, where a negative i32 is a failure, and adds the errno to it: signalFd and readSignal answer a plain -1 for any failure (and readFileBytesSync answers null), while a nish:net call says which failure it was, so a loop tells "would block" from "the peer has gone" without asking again. No nish:net call touches the arena (net_tcp_echo serves two hundred messages in a loop whose pass scope gives back what each pass builds, and prints flat when that moved the arena no further than one message did).

Socket options

netSetOption's option is one of five ids nish:net exports, each an i32 and each a global too. They are the runtime's own numbers, not the kernel's, so a program reads the same on Linux and Darwin, and the runtime maps each to its level and name; there is no raw passthrough, because a level, a name and a length the program spells is an untyped call into the kernel.

Id Value Sets value
NET_TCP_NODELAY 1 IPPROTO_TCP, TCP_NODELAY 1 sends a small write at once, 0 lets Nagle's algorithm batch it
NET_TCP_QUICKACK 2 IPPROTO_TCP, TCP_QUICKACK, Linux only; -95 on Darwin 1 acknowledges at once, 0 lets the kernel delay the ACK. Linux does not keep the mode: it leaves it on its own, so a server that wants one sets it again after every response
NET_SO_KEEPALIVE 3 SOL_SOCKET, SO_KEEPALIVE 1 probes an idle connection
NET_SO_SNDBUF 4 SOL_SOCKET, SO_SNDBUF bytes; Linux doubles the value it stores
NET_SO_RCVBUF 5 SOL_SOCKET, SO_RCVBUF bytes; Linux doubles the value it stores

An id is read as a value, as platform is: it folds to its number in the IR, cannot be assigned, and is not a module constant, so a module constant cannot be initialised from one. A local of the same name shadows the global spelling.

netMss's -1 is one answer, not an errno, because a program does the same thing for every reason there is no MSS: it uses a size of its own. That is also why it compiles on wasm when netSetOption is refused there: a socket option a program sets is an effect a wasm build cannot have, while a size it reads has an honest answer everywhere, "none".

The address form

An address is 18 bytes of a u8[]: the 16 bytes of an IPv6 address, with an IPv4 address written as the mapped ::ffff:a.b.c.d, then the port, big-endian, so 127.0.0.1:8080 is ten zeros, 255 255, 127 0 0 1, 31 144 (net_tcp_calls). One form for both families, in memory the caller owns, so a server that accepts a million connections allocates nothing to learn who they are from.

Both families on one socket. net_dual_stack listens on "::", and the net_ block of tests/run.js connects to it from Node over 127.0.0.1 and then over ::1: the first peer arrives as ::ffff:127.0.0.1 and the second as ::1, each with Node's own port in the last two bytes, and both are echoed. Its UDP socket, bound to "::" too, sends itself a datagram with each ECN mark to its IPv4 address and to its IPv6 one, and reads every mark back from meta[1] over both. That needs a host with IPv6: where ::1 cannot be listened on, "::" is the IPv4 fallback above, and the check says so and passes rather than counting a skip. CI's ubuntu-latest has IPv6 and runs it.

The client half

Function Takes Gives Effect
tcpConnect readonly u8[] i32 write
connectResult i32 i32 write

tcpConnect(addr: readonly u8[]): i32 makes a new TCP socket, non-blocking and close-on-exec, that starts connecting to the address in addr, and answers its descriptor at once, while the connection is still being made; the loop reports it writable when that has finished. Any other failure closes the socket and is answered instead of it.

  • Tests: net_tcp_connect, net_tcp_connect_import; reject_net_connect_element, reject_net_connect_arity.

connectResult(fd: i32): i32 answers how the connection went, asked once fd is writable: 0, or its failure from SO_ERROR, such as -111 for a port nobody listens on. The kernel clears the failure as it is read.

  • Tests: net_tcp_connect, net_tcp_connect_import; reject_net_connect_wasm.

A connection is made the way everything else here waits: in the loop. Linux answers EINPROGRESS even over loopback, so a program asks for writability and not for an answer from the call. Neither call waits, so both are willreturn, and neither allocates. net_tcp_connect is a Nish client and a Nish server in one loop: two exchanges of 1 and 200 messages come back byte for byte with the arena where it stood, and a port that was bound and closed again answers -111 and then 0. The net_ block of tests/run.js runs it under a timeout, so a lost wake fails the run rather than hangs it.

const addr: u8[] = new Array<u8>(18);
netAddress(addr, "127.0.0.1", 8080);
const fd = tcpConnect(addr);          // the descriptor, still connecting
pollAdd(loop, fd, 2, 1);              // writable once it has finished
// ... pollWait until token 1 comes back ...
if (connectResult(fd) === 0) {
  pollModify(loop, fd, 1, 1);         // connected: now wait to read
}

UDP

Function Takes Gives Effect
udpBind string, i32, i32 i32 write
udpSendTo i32, a range, to, segment, ecn i32 write
udpRecvFrom i32, a range, from, meta i32 write

udpBind(host: string, port: i32, flags: i32): i32 answers a datagram socket, non-blocking and close-on-exec, bound as tcpListen's is, "::" and its IPv4 fallback included. Flag 1 is SO_REUSEPORT, flag 2 is UDP_GRO, and flag 4, with flag 1 and a group size N from 1 to 255 in bits 8 to 15, steers QUIC: a classic-BPF reuseport program hands each datagram to the port's socket number (byte 0 of the Destination Connection ID) mod N, in bind order (docs/h3-multicore.md). A refused attach closes the socket and answers the errno. Flag 4 without 1, with 2 or with N 0, N without 4, and any other bit are -22; flags 2 and 4 are -95 on Darwin.

  • Tests: net_udp_calls, net_udp_echo, net_udp_offload, net_dual_stack, net_quic_prefork; reject_net_udp_wasm, reject_net_udp_parallel.

udpSendTo(fd: i32, buf: readonly u8[], off: i32, len: i32, to: readonly u8[], segment: i32, ecn: i32): i32 sends buf[off, off + len) to to, cut by the kernel into datagrams of segment bytes when that is above 0 (GSO), with ecn as the IP header's two ECN bits. The range is checked.

  • Lowering: one sendmsg. Not willreturn: on a blocking socket the program inherited it waits.
  • Tests: net_udp_calls, net_udp_echo, net_udp_offload, net_udp_bounds; reject_net_udp_send_element, reject_net_udp_arity, reject_net_udp_task.

udpRecvFrom(fd: i32, buf: u8[], off: i32, len: i32, from: u8[], meta: i32[]): i32 receives the next datagram into buf[off, off + len), cut to len, with its sender in from, its GRO segment size in meta[0] and its ECN bits in meta[1]. It answers -11 when none is waiting. The range is checked.

  • Lowering: one recvmsg. Not willreturn, as udpSendTo.
  • Tests: net_udp_calls, net_udp_echo, net_udp_offload, net_udp_bounds; reject_net_udp_readonly, reject_net_udp_meta_readonly, reject_net_udp_meta_element, reject_net_udp_element, reject_net_udp_region.

A datagram carries no stream, so an echo is udpRecvFrom and then udpSendTo back to from, which is already an address; the net_ block of tests/run.js drives net_udp_echo from Node's dgram, two hundred datagrams in a round, and requires every byte back and the arena flat. An empty datagram is a datagram: udpRecvFrom answers 0 for it, which is how net_udp_echo ends a round. A socket that is IPv4 — bound to an IPv4 host, or to "::" on a kernel without IPv6 — sends to a mapped ::ffff:a.b.c.d address as IPv4 (net_udp_calls).

Segmentation offload. On Linux, one udpSendTo of 4,800 bytes with segment = 1200 leaves as four datagrams of 1,200, which a Node dgram socket receives one by one. A socket bound with flag 2 receives what one such send left as one datagram of up to 64 KB with meta[0] == 1200, while a socket without it receives the four apart, each with meta[0] == 0 (net_udp_offload, run by the net_ block). The kernel coalesces only datagrams of one flow, and only those that arrive after the option is set, which is why it is a flag of udpBind; on loopback it coalesces what left as one GSO send, not datagrams sent one by one. The ECN bits a send asks for arrive in the receiver's meta[1] (net_udp_offload). Measured on loopback, Nish sender to Nish receiver, 1,200-byte datagrams: 459,000 a second at one per call, and 1,760,000 a second sent with segment = 1200 over 49,200-byte sends.

Darwin has none of the three. There udpSendTo with a segment or an ecn above 0, and udpBind with flag 2, answer -95, and meta is always 0, 0. A meta[1] of 0 is also what a datagram that really is not ECN-capable reads as, so a receive on Darwin cannot say ECN is unsupported: a program learns that from the -95 a send with an ecn above 0 answers, and on Darwin it reads every arrival as Not-ECT. CI's runtime-net (macos-latest) job runs that branch: the net_ sections there take the -95 a GSO send and a GRO bind answer, and fall back to one datagram a send (net_loopback, net_relay), and net_udp_offload requires the -95 of GSO, the GRO bind and each ECN mark, with nothing sent by them while a plain send still arrives. On Linux no UDP call answers -95, and the code is pinned there by tcpAccept on a datagram socket instead, above.

The readiness loop

Function Takes Gives Effect
pollCreate nothing i32 write
pollAdd i32, i32, i32, i32 i32 write
pollModify i32, i32, i32, i32 i32 write
pollRemove i32, i32 i32 write
pollWait i32, i32[], i32 i32 write

pollCreate(): i32 answers a loop: a descriptor, close-on-exec, that watches others.

  • Lowering: epoll on Linux, kqueue on Darwin.
  • Tests: net_loop_calls, net_loop_two; reject_net_poll_wasm.

pollAdd(loop: i32, fd: i32, events: i32, token: i32): i32 watches fd for events (1 readable, 2 writable) and reports it under token, an i32 of the program's choosing. It answers -17 for a descriptor the loop already watches.

  • Tests: net_loop_calls, net_loop_two; reject_net_poll_arity, reject_net_poll_parallel.

pollModify(loop: i32, fd: i32, events: i32, token: i32): i32 sets new events or a new token for a watched descriptor. It answers -2 for one the loop does not watch.

  • Tests: net_loop_calls, net_loop_two.

pollRemove(loop: i32, fd: i32): i32 stops watching fd. It answers -2 for one the loop does not watch. Closing a descriptor stops the watching too.

  • Tests: net_loop_calls, net_loop_two.

pollWait(loop: i32, ready: i32[], timeoutMs: i32): i32 waits until a watched descriptor is ready or timeoutMs passes, forever for a negative one, and writes a token and events pair per ready descriptor into ready, 4 added for a hang-up or an error. It answers the count, at most 64, or 0 on a timeout.

  • Lowering: not willreturn, because it can wait forever.
  • Tests: net_loop_calls, net_loop_two; reject_net_poll_readonly, reject_net_poll_element, reject_net_poll_task, reject_net_poll_region.

Level-triggered. A descriptor that is still ready when the program waits again is reported again, so a loop that handles one pair of several, or leaves a datagram unread, loses nothing: net_loop_calls sends a datagram and waits twice, and both waits report it. The same is why the 64 a call reports at most is a bound on a stack array rather than on the program: the rest are still ready at the next wait. A UDP socket is always writable, and on Linux one whose both sides are shut is readable, writable and hung up, 7 (net_loop_calls). Nothing allocates; ready is the caller's.

A signal wakes the loop. The descriptor signalFd() answers is a descriptor like any other, readable when SIGTERM or SIGINT arrives, so a server adds it to its loop and shuts down when its token comes back: readSignal then answers the number without waiting. A signal that interrupts the wait itself makes pollWait answer 0, as a timeout does, rather than wait on past the deadline it was given; the loop comes round and the next wait finds the signal descriptor ready (tests/runtime-test.c). Once the process has opened a NET_LISTEN_RING listener with rings, for the rest of its life, closed or not, the rings' own work interrupts the wait too, so an interrupted wait goes on to its deadline and only the signal descriptor reports a signal. net_loop_two watches two UDP sockets and signalFd() in one loop, and the net_ block of tests/run.js drives it from Node: it writes to one socket and waits for the line that says that socket woke the loop before it writes to the other, in both orders, and then sends SIGTERM, which wakes the loop through the signal descriptor; the program reads 15, prints flat because the arena stood where it did at the top of every pass, and exits 0.

const loop = pollCreate();
pollAdd(loop, server, 1, 1);
pollAdd(loop, signalFd(), 1, 2);
const ready: i32[] = new Array<i32>(32);
let running = true;
while (running) {
  const n = pollWait(loop, ready, -1);
  for (let k = 0; k < n; k++) {
    if (ready[2 * k] === 2) {
      running = false;             // SIGTERM or SIGINT
    } else {
      serve(server);               // until it answers -11
    }
  }
}

Darwin is kqueue, with a filter per direction rather than a set of events, and the runtime builds epoll's answers from the filters: pollAdd of a descriptor already watched is -17, pollModify and pollRemove of one not watched are -2, a descriptor added watching nothing holds a disabled read filter so the loop still knows it, and a descriptor both readable and writable is one pair, 3, folded from its two filters' records. Three differences stay. The pollAdd that answers -17 leaves the descriptor, both ways, under the token it gave rather than the old one. A descriptor watching nothing is never reported: epoll still reports its hang-up or error as 4, but a disabled filter reports nothing. And a peer's half-close is 1 on Linux but 5, 1|4, on Darwin: kqueue marks the read filter with end-of-file as soon as the FIN arrives, and the runtime reads end-of-file as 4, where epoll sets its hang-up bit only once both directions are shut. CI's runtime-net (macos-latest) job runs net_loop_calls and net_loop_two on a mac, as it runs every net_ section.

Unix-domain sockets

A Unix-domain socket is named by a path rather than a host and port, and reaches only processes on the same machine. Once made, it is a descriptor like any other: netRead, netWrite, netShutdown, netClose and the readiness loop take it as they take a TCP one, and a peer that has gone is -32 here too. There is no address form: unixAccept writes no peer.

  • The path. It is bytes up to the first NUL, so a NUL inside it is -22, as an empty path is. It must fit sun_path with its NUL: 107 bytes on Linux, 103 on Darwin, and a longer one is -36 before a socket exists, rather than the kernel's cut-short name binding a file the program never named. A relative path is the working directory's, and the limit is on the string as written, not the absolute path it resolves to.
  • Abstract names. A path that starts with @ is Linux's abstract namespace: the rest, at most 107 bytes, is the name, nothing is made on the file system and nothing is left behind when the socket closes. A bare @ is -22. Darwin has none, and there every @ path is -95. A file whose name starts with @ is spelled ./@name, which is a relative path like any other.
  • The file's mode. A pathname listener's file is 0600 for unixListen, so only its owner may connect, and unixListenMode's mode otherwise. The runtime sets it between the bind and the listen, while every connect is still refused, so the umask's mode is never one a client gets in under. A connect needs write permission on the file, and search permission on every directory above it, which is the real boundary: put the socket in a directory only the users who may connect can reach.
  • A path that exists. A bind never replaces a file. A path that exists is -98, whatever is there, and flag 1, NET_UNIX_UNLINK_STALE, clears only a socket file nothing listens on, the one a server that died left: the runtime connects to it, and only a refusal makes it unlink the file and bind again. On Linux a live listener, even one whose backlog is full, stays -98; a regular file, a directory or a link is -17 and left alone. On Darwin a connect to a listener whose backlog is full is refused just as one to a dead file is, so the probe cannot tell them apart: there a socket file in the way is -95 and left alone, and the flag never unlinks one; remove a file the program knows is its own with unlinkSync. The probe is a connection the live listener sees close at once. Between the probe and the unlink another process can bind the path, which is why the directory must be the program's own (docs/security/runtime.md, RT-34).
  • Cleanup. Closing a listener does not unlink its file: unlinkSync it when the server stops, or let the next start's flag 1 clear it. A unixListen that fails after its bind closes the socket and unlinks the file it made, so no failure leaves a descriptor or a file behind (net_unix_fds counts the descriptors around every failure, a chmod made to fail after the bind among them, and checks that none of the paths it named is left on disk).
  • Pairs. unixSocketPair makes two sockets already connected to each other. NET_UNIX_STREAM is a byte stream, as a connection is; NET_UNIX_SEQPACKET keeps each write one message, so a read returns one message, never two run together (net_unix_calls). Darwin has no Unix seqpacket, and kind 2 there is -95.
  • Descriptors and credentials. The descriptor recvFd gets for one sendFd sent is a new descriptor for the same open file, with its offset and status flags shared, as dup gives. recvFd keeps the first descriptor of a message and closes every other one before it answers, so a peer cannot leave this process holding descriptors it never asked for (docs/security/runtime.md, RT-37). The one it keeps is close-on-exec, so a later spawn does not hand it on, but it is not made non-blocking: netRead on a blocking one waits, so set what the program needs on it before it joins a loop. peerCredentials names the process at the other end as the kernel recorded it at the connect; a peer that has changed its uid since, or passed the socket on, still reads as it was then (RT-39). On a TCP socket, or any socket not AF_UNIX, the three are -95, and on a descriptor that is not a socket -88, both before anything is sent or asked (net_unix_scm).
Constant Value Meaning
NET_UNIX_UNLINK_STALE 1 unixListen's flag: clear a socket file nothing listens on
NET_UNIX_STREAM 1 unixSocketPair's byte-stream kind
NET_UNIX_SEQPACKET 2 unixSocketPair's message kind; -95 on Darwin

NET_UNIX_UNLINK_STALE and NET_UNIX_STREAM are both 1, as NET_TCP_NODELAY is: the constants are plain i32s, so the checker cannot tell one passed where another belongs. Name the one the call takes.

On Darwin the calls are the same C under #if, with seven differences: no abstract names (-95), no seqpacket pair (-95), flag 1 meeting a socket file is -95 rather than a probe, a connect to a full backlog is -111 rather than -11, fcntl after accept and socketpair where Linux passes SOCK_NONBLOCK | SOCK_CLOEXEC, fcntl after recvmsg where Linux passes MSG_CMSG_CLOEXEC, so a child another thread spawns between the two inherits the descriptor recvFd keeps (RT-38), and peerCredentials reading getpeereid and LOCAL_PEERPID, its pid -1 where that refuses, where Linux reads SO_PEERCRED. A listener has no peer: Linux answers its own process's credentials there, and what Darwin answers is not yet seen on a mac. CI does not run them on a mac yet.

Bounds

netRead, netWrite, netWrite2 (each of its two ranges), udpSendTo, udpRecvFrom and fileReadAt check 0 <= off <= off + len <= buf.length in the compiled code, before the call, the way dst.set(src, at) checks the range it writes, and a range outside it panics with the same words, slice out of range: [4, 9) of length 8, exit 1 (net_tcp_bounds, net_udp_bounds, net_write2_bounds, file_read_at_bounds). A negative off or len fails it. A peer, out, to or from array shorter than 18 bytes, or a meta shorter than 2, is not a panic but -22, from the runtime, because the length is the argument's property rather than the program's arithmetic. So is a ready shorter than 2, and pollWait never writes past ready.length, so it has no range to check.

Types, threads and wasm

  • Bytes only. Each buffer is a u8[], as crypto.getRandomValues's is: a socket carries bytes, and a wider element would make the byte order on the wire a fact about the machine. Anything else is `netRead` fills a `u8[]`, got i32[] (NL2402; reject_net_element) for a buffer the call writes, and `netWrite` sends from a `u8[]`, got string (NL2405; reject_net_write_element, reject_net_udp_send_element) for the one it reads. udpRecvFrom's meta holds numbers rather than bytes, a segment size up to 65,535 among them, so it is an i32[], and anything else is `udpRecvFrom` fills an `i32[]`, got u8[] (NL2406; reject_net_udp_meta_element), and so is pollWait's ready, whose tokens are the program's own i32s (`pollWait` fills an `i32[]`, got u8[], reject_net_poll_element), and so is unixSocketPair's fds (`unixSocketPair` fills an `i32[]`, got u8[], reject_net_unix_pair_element), and so are recvFd's fd and peerCredentials's out (reject_net_unix_scm_element, reject_net_unix_peer_element). A path is a string, and anything else is refused (reject_net_unix_path_type). A descriptor, an offset, a length, a port, a backlog, how, the flags, a segment size, the ECN bits, a loop, the events, a token, a timeout, a socket option and its value, a pair kind and a mode are i32 in both number modes, and an f64 is refused rather than converted (reject_net_fd_type).
  • What is written. netAddress's out, tcpAccept's peer, netRead's buf, udpRecvFrom's buf, from and meta, pollWait's ready, unixSocketPair's fds, recvFd's buf and fd and peerCredentials's out are written, so a readonly array there is refused as a store through it is (reject_net_readonly, reject_net_accept_readonly, reject_net_udp_readonly, reject_net_udp_meta_readonly, reject_net_poll_readonly, reject_net_unix_pair_readonly, reject_net_unix_scm_readonly, reject_net_unix_scm_buf_readonly, reject_net_unix_peer_readonly), a parameter one of them fills is never readonly in the IR, and netWrite's buf, netWrite2's a and b, udpSendTo's buf and to and sendFd's buf may be readonly (net_udp_calls, net_write2_bounds).
  • Threads. Every call changes the kernel's state of a socket, which is a write its caller can see, and a loop is a socket's state as much as its bytes, so none may be made in a scope() task (reject_net_task, reject_net_udp_task, reject_net_poll_task) or a parallel body (reject_net_parallel, reject_net_udp_parallel, reject_net_poll_parallel), and a call that writes an array is refused in a scope's region on one a task may read, as fill is (reject_net_region, reject_net_udp_region for meta and reject_net_poll_region for ready).
  • On wasm. A wasm32 target and the wasi profile have no sockets for these to reach, and runtime-net.c is empty there, so every call is refused by the checker with the host builtins' words: `tcpListen` reaches the operating system, and a wasm32 build has none to reach (NL2404; reject_net_wasm, reject_net_import_wasm, reject_net_udp_wasm, reject_net_poll_wasm, reject_net_reuseport_wasm, reject_net_set_option_wasm, reject_net_unix_wasm). The freestanding --profile wasm link, which takes IR compiled for the host, has no runtime-net.c either, so a program that calls one fails there at the link, on the undefined nish_ symbol. The one exception is netMss, which compiles under a wasm32 target and the wasi profile to the constant -1, its descriptor still evaluated, with no runtime call (net_mss in tests/run.js).
  • Not yet. No nish:net call resolves a name (reject_net_unknown_export): the stub resolver is the library module nish/net/dns, which drives these calls on the program's own loop. There is no sendmmsg or recvmmsg, no edge-triggered mode and no timer but pollWait's timeout. A program that does not want a loop can still wait for a connection, a byte or a datagram by calling again on -11, as net_tcp_echo and net_udp_echo do.

The TypeScript reading (wp33-round-trip.md §2) is class C, as signalFd's is: Node's sockets are asynchronous only, and a socket becomes ready only to Node's event loop, which a program that owns its loop never returns to. runtime/nish.d.ts declares nish:net and the globals, so npm run check types these programs, and under runtime/nish.mjs each function throws `netAddress` has no synchronous reading under Node: a socket is ready only to the event loop, which a program that owns its loop never returns to (docs/wp33-round-trip.md) rather than answer -11 forever (the net_ block of tests/run.js, for netAddress, tcpConnect, udpBind and pollCreate). The option ids are plain numbers, so they read the same under Node as natively, and netSetOption throws rather than set one (net_set_option in tests/run.js). netMss throws too rather than answer -1: Node's net.Socket has no MSS to read, and a -1 would be a guess that the program keeps (net_mss in tests/run.js). The same holds for the Unix-domain calls: the NET_UNIX_ constants read natively, and each of the five calls throws (net_unix in tests/run.js). In a browser there are no sockets at all. A loop has no reading either: the one a Node program has is Node's own, which it returns to between callbacks, and pollWait is a program that never returns to it.

nish:term: terminal input

What a program needs to run a full-screen terminal interface: read a key the moment it is pressed, wait for one with a timeout that doubles as a frame clock, and ask the terminal whether it is one, how big it is, and to stop echoing and line-editing. None of it needs the net capability that a pollAdd(0) and netRead(0, …) would carry; the six functions carry tty instead. They are C in runtime/runtime-term.c, a translation unit of its own with its own .text* ceiling, which section GC leaves out of a program that calls none of them. Each is exported by nish:term and is a global too, as every nish: export is. Linux and Darwin only, and WASI as below.

Reference: nish:term.

Function Takes Gives Effect
readStdin u8[], i32, i32 i32 write
waitStdin i32 boolean write
isTerminal i32 boolean write
terminalSize nothing i32[] write
setRawMode boolean i32 write
setTerminalRestore string i32 write

readStdin(buf: u8[], off: i32, len: i32): i32 reads at most len bytes of stdin into buf[off, off + len), without waiting. It answers the count, 0 at the end of the stream, or -11 when nothing is waiting. The range is checked, and the buffer is a writable u8[].

  • Tests: term_calls, term_import; reject_term_read_arity, reject_term_read_i32_buffer, reject_term_read_readonly.

waitStdin(timeoutMs: i32): boolean waits until stdin is readable, its end and an error included, or timeoutMs passes, for ever for a negative one. It answers false when the time ran out or a signal ended the wait early, but not for a Ctrl-Z and fg once setRawMode(true) has caught SIGTSTP, after which the wait goes on for what is left. Before raw mode Linux goes on with what is left and Darwin ends the wait.

  • Lowering: not willreturn.
  • Tests: term_calls, term_import, term_pty; reject_term_wait_string.

isTerminal(fd: i32): boolean answers whether the descriptor is a terminal.

  • Lowering: isatty.
  • Tests: term_calls, term_pty; reject_term_is_terminal_arity.

terminalSize(): i32[] answers a fresh [columns, rows] of the terminal on stdout, or on stderr when stdout is not one; [-25, 0] for "not a terminal".

  • Tests: term_calls, term_pty, mem_terminal_size_scope; reject_term_size_arity.

setRawMode(on: boolean): i32 puts stdin's terminal in raw mode (true) or back as it was (false). It answers 0, or -25 when stdin is not a terminal. setRawMode(false) is always safe.

  • Tests: term_calls, term_pty; reject_term_raw_number, reject_term_unknown_export.

setTerminalRestore(bytes: string): i32 keeps a copy of bytes for the runtime to write to the terminal on stdout as the program ends, replacing what it kept before. It answers 0, -25 when stdout is not a terminal and -7 (E2BIG) past NISH_TERM_RESTORE_MAX, 4096 bytes, both keeping what was there. When they go out is under Raw mode.

  • Tests: term_restore; reject_term_restore_arity, reject_term_restore_number, reject_term_restore_wasm.

Errors are numbers, in nish:net's shape: >= 0 is success and a negative number the failure's errno, negated. The two a program branches on are the same number on every platform: -11, nothing is waiting, and -25, not a terminal. Any other is the host's own.

stdin is never made non-blocking. O_NONBLOCK belongs to the open file description, which a shell shares with every program it starts, so a program that set it and then crashed would leave the shell's own stdin non-blocking. readStdin asks select with a timeout of 0 before it reads instead, and so does waitStdin: Darwin's poll does not support devices, and answers a terminal at once whether a key is waiting or not.

The loop. waitStdin is the frame clock: a game waits up to one frame for a key, reads what came, advances the world and draws, so the wait costs nothing when a key arrives early and the frame rate holds when none does.

import { readStdin, setRawMode, waitStdin } from "nish:term";

export const main = (): number => {
  setRawMode(true);
  const keys: u8[] = new Array<u8>(16);
  let running = true;
  while (running) {
    if (waitStdin(16)) {
      const n = readStdin(keys, 0, keys.length);
      if (n === 0 || (n < 0 && n !== -11)) {
        running = false; // stdin ended, or reading it failed
      }
      for (let i = 0; i < n && i < keys.length; i++) {
        if (keys[i] === 113) {
          running = false; // q
        }
      }
    }
    // advance and draw one frame
  }
  setRawMode(false);
  return 0;
};

Raw mode

Raw mode is cfmakeraw's input side without its output side. Echo, line editing, CR-to-NL translation, flow control (Ctrl-S, Ctrl-Q) and parity stripping are off, bytes are 8 bits wide, and a read answers as soon as one byte is there, so a key arrives as its bytes — an arrow as the three of ESC [ A. Two things stay as they were:

  • ISIG stays on. Ctrl-C is still SIGINT and ends the program, Ctrl-\ is still SIGQUIT and Ctrl-Z still stops it; none of them arrives as a byte (3, 28, 26). A program that wants Ctrl-C as a key asks for SIGINT through signalFd and handles it in its loop. Keeping it means a program that is stuck in a loop can always be ended from the keyboard.
  • OPOST stays on, so "\n" still moves to the start of the next line and output needs no "\r\n".

The terminal is always put back. Each setRawMode(true) that enters raw mode saves the termios that was there, and it comes back on setRawMode(false); when main returns or process.exit is called (an atexit handler); on every panic, a failed check included, whose path ends in _exit and so is given its own hook; and on every signal whose default action ends the program, SIGKILL aside: SIGINT, SIGTERM, SIGQUIT and SIGHUP, SIGPIPE from a reader that went away, SIGALRM, SIGUSR1, SIGUSR2, SIGPROF, SIGVTALRM, SIGXCPU, SIGXFSZ and SIGSYS, and the faults a program dies of, SIGILL, SIGTRAP, SIGABRT, SIGFPE, SIGBUS and SIGSEGV (a stack overflow, or a fault in C code a program links). For each a handler is installed only when the program left it at its default action: it restores the terminal, puts the default back and raises the signal again, so the program still dies of it and its parent sees that (exit status 143 for SIGTERM in term_pty). The handlers run on a 64 KiB alternate stack of the thread that entered raw mode, or on the one that thread already had, so a stack overflow there restores the terminal too. The mode is marked raw once the switch is made, so a signal that ends the program while TCSAFLUSH drains the output, or while SIGTTOU holds it stopped in the background, leaves the terminal as it was. A program that took SIGTERM or SIGINT through signalFd keeps it, and puts the terminal back itself, or by returning. SIGKILL leaves raw mode behind, and so does a stack overflow on a thread other than the one that called setRawMode(true); stty sane undoes either. SIGTSTP, Ctrl-Z, is caught the same way when it is at its default, because no shell can be counted on to: bash puts its own termios back while a job is stopped but does not give the job its raw mode back on fg, and a shell without that bookkeeping puts back nothing. The handler restores the terminal, stops the program, and once fg or bg continues it puts the handler back and enters raw mode again, so the terminal is as it was while the program is stopped and raw after fg; no setRawMode on another thread runs between the two. A program continued in the background stops again, by SIGTTOU, until fg. A waitStdin the stop interrupts goes on waiting until its timeout, measured from when it began. Any other blocking call goes on where the system restarts it after a handler (SA_RESTART), and ends early where it never does, as a pollWait does after any signal. A stop by SIGSTOP, which no program can catch, leaves the terminal raw at a shell that puts nothing back. Job control also stops a program started in the background (app &) when it calls setRawMode(true), by SIGTTOU, as it stops any program that sets the terminal, and the program enters raw mode on fg. Two threads that call setRawMode at once take turns, so the termios saved is the one from before either.

Restore bytes go out on the same paths. The bytes setTerminalRestore kept are written to stdout once, with write alone, before the termios comes back, when main returns or process.exit is called, on every panic and on each signal above whose default ends the program, raw mode or not: setting them installs the same handlers. Ctrl-Z and setRawMode(false) do not end the program, so they write nothing. A program sets them once it has switched the terminal ("\u001b[?1049h" written, say) and again with each change, last the set that undoes them all; a signal that lands while it does writes the old string or the new one, whole. They are written at most once, however many of those paths race. Output is not buffered, so all the program printed reaches the terminal before them. term_restore under script(1) checks that what follows its last line is exactly its bytes, once, after it returns, panics and is killed by SIGTERM and by SIGSEGV, nothing after it cleared them, and that stty -g reads as before.

The term block of tests/run.js runs term_pty under script(1), which gives it a pseudo-terminal: it is a terminal on both descriptors with a size, enters raw mode twice and leaves twice, each 0, and after it returns in raw mode, panics in raw mode, and is killed by SIGTERM, and then by SIGSEGV, while it waits in raw mode, stty -g reads what it read before. Where script is missing or makes no terminal, that check is a counted skip that says which.

Where there is no terminal

Host readStdin waitStdin isTerminal terminalSize setRawMode setTerminalRestore
Linux, Darwin as above as above isatty the ioctl as above as above
A pipe or a file as above as above false [-25, 0] -25 for true, 0 for false -25, 0 for "", -7 past the cap
--profile wasi waits for its bytes as above false [-25, 0] -25 for true, 0 for false -25, keeping nothing; 0 for "", -7 past the cap
--target wasm32, freestanding refused refused refused refused refused refused
Node (prelude) throws throws tty.isatty Node's columns and rows throws throws
A browser (nish/runtime/browser) throws throws false [-25, 0] throws throws

WASI has read and poll on stdin and no terminal interface. There readStdin reads without asking poll first, because Node's WASI answers a zero-timeout poll of stdin with "nothing waiting" while bytes are, so it waits for its bytes: call waitStdin first, as the loop above does. For the same reason a waitStdin there waits at least 10 ms: Node's WASI counts a shorter wait from before it looks at stdin, and on a busy machine answers "nothing waiting" before it has looked. The checker accepts the six there, unlike nish:net's, and refuses them on a freestanding --target wasm32 build, whose runtime has no stdin (reject_term_wasm). Under Node, stdin is read only by the event loop, which a program that owns its loop never returns to, and Node's own raw mode turns Ctrl-C into a byte, so the four that would need either throw, each naming itself. The term and term_wasi blocks of tests/run.js feed term_calls and term_import over a pipe — four bytes after a delay, a pipe held open and never written, and a thousand bytes — natively and under WASI, and expect the same lines from both.

Arrays and strings as receivers

The reference for each member, with its parameters and its answer, is its doc comment on Array, ReadonlyArray or String in runtime/nish.d.ts; the rules are here.

Member On Gives
length arrays number
push(v) arrays number
pop() arrays T
indexOf(v) arrays number
join(sep?) arrays string
set(src, offset?) arrays void
fill(value, start?, end?) arrays void
sort(cmp) arrays T[]
map(f) arrays U[]
filter(pred) arrays T[]
some(pred), every(pred) arrays boolean
findIndex(pred) arrays number
forEach(f) arrays void
slice(start?, end?) arrays T[]
length strings number
charCodeAt(i) strings number
substring(start, end?) strings string
slice(start, end?) strings string
indexOf(sub, from?) strings number
startsWith(sub), endsWith(sub) strings boolean
String.fromCharCode(c) strings string
String.fromBytes(bytes, start, end) strings string

a.length is a number; read-only.

  • Tests: arr_length, reject_arr_length_assign.

a.push(v: T): number appends and returns the new length. An array never grows past 2^31 − 1 elements, in either number mode: the push that would fails as an allocation does, nish: out of memory and exit 1 (tests/link/cg_sec_push_limit).

  • Lowering: grows capacity by doubling (4 from 0) through nish_array_grow.
  • Tests: arr_push; reject_arr_push_type (Cannot push string onto i32[]).

a.pop(): T removes and returns the last element. An empty array panics and exits 1 (index out of range: 0 >= 0) — there is no undefined to return; behind a test that the array holds an element, such as if (a.length > 0), the check is proven away (panics_pop). The capacity is untouched, so the next push reuses the storage.

  • Tests: arr_pop_index; reject_arr_pop_arity.

a.indexOf(v: T): number answers the first index whose element is === v, or -1: strings by content, classes / interfaces / arrays by identity, floats with fcmp oeq (a NaN element is never found, as in JavaScript).

  • Lowering: a scan in the emitted code.
  • Tests: arr_pop_index; reject_arr_index_of_type.

a.join(sep: string = ","): string is for string[] only (`join` requires string[], got i32[], reject_arr_join_elem). Element conversion would allocate per element, which is the quadratic shape join exists to replace (wp14-selfhost.md §3). Like a concatenation, its result is held to 2^31 − 1 bytes: a longer one fails as an allocation does, nish: out of memory and exit 1, before anything is allocated (CG-3).

  • Lowering: one pass summing the lengths, one allocation, one memcpy per part.
  • Tests: arr_join, cg_sec_join_limit; reject_arr_join_elem.

a.set(src: T[], offset: number = 0): void copies every element of src into a from offset on, with TypedArray.prototype.set's meaning. Arrays of numbers only, of any width and ranged integers included, and src has a's own element type. A statement. See Bulk writes.

  • Tests: bytes_set_disjoint, bytes_set_self, bytes_set_end, bytes_set_oob, bytes_set_overlap, bytes_set_float_oob, bytes_set_u64_oob; reject_bytes_set_string, reject_bytes_set_type, reject_bytes_set_arity, reject_bytes_set_readonly, reject_bytes_set_value.

a.fill(value: T, start: number = 0, end: number = a.length): void stores value into every slot of [start, end), each end relative and clamped as TypedArray.prototype.fill's are. Arrays of numbers only. A statement. See Bulk writes.

  • Tests: bytes_fill, bytes_fill_zero, bytes_offset_float, bytes_offset_u64; reject_bytes_fill_record, reject_bytes_fill_arity, reject_bytes_fill_value, reject_bytes_fill_index, reject_bytes_fill_value_position, reject_bytes_fill_readonly.

a.sort(cmp: (a: T, b: T) => number): T[] sorts a in place, stably, and answers a itself, as Array.prototype.sort does: the order is element for element Node's for any consistent comparator. cmp answers a negative number, zero or a positive number, and only "positive or not" is read, so -0 and NaN count as zero.

The comparator is required: a.sort() compares as strings in JavaScript and is refused (`sort` needs a comparator: write `xs.sort((a, b) => a - b)`, NL2620); a second argument is refused too (`sort` takes one comparator, NL2619). A readonly T[] is refused (Cannot `sort` through readonly), and so is process.argv (`process.argv` is read-only, as push is). a - b panics on i32 elements a full range apart ([2147483647, -1]); (a, b) => (a > b ? 1 : a < b ? -1 : 0) cannot overflow.

The comparator is a compile-time function argument (Function parameters): a top-level function, or an arrow, which may read the const locals and parameters of the function it is written in; an arrow reading a let is refused (reject_arr_sort_capture_let). See Sorting.

  • Lowering: compiled to std/sort.ts's sortWith, a merge sort over runs of 32 sorted by insertion, instantiated once per comparator with the comparator called directly; it is copied into the module that sorts.
  • Tests: arr_sort_cmp, arr_sort_cmp_f64, arr_sort_cmp_record, arr_sort_edge, arr_sort_capture, arr_sort_overflow; reject_arr_sort_no_comparator, reject_arr_sort_arity, reject_arr_sort_readonly, reject_arr_sort_argv, reject_arr_sort_capture_let.

a.map(f: (x: T[, i: number]) => U): U[] answers a new array of f(a[i]) (or f(a[i], i)) for each element, in order, as Array.prototype.map gives. The callback takes one or two parameters; a third, JavaScript's array argument, is refused (The callback of `map` takes 3 parameters, and Nish passes at most two …, NL2698), and so is a second argument to map itself (JavaScript's thisArg). It is a compile-time function argument under the comparator's rule. See Callback methods and slice.

  • Lowering: compiled to std/array.ts's map or mapIndexed, called directly.
  • Tests: arr_map, arr_map_f64, arr_map_template; reject_arr_map_third_param, reject_arr_map_this_arg, reject_arr_map_capture_let.

a.filter(pred: (x: T[, i: number]) => boolean): T[] answers a new array of the elements pred keeps, in order. pred answers boolean (reject_arr_filter_not_boolean); the callback rules are map's.

  • Tests: arr_filter; reject_arr_filter_not_boolean.

a.some(pred), a.every(pred): boolean answer whether some (every) element satisfies pred, stopping at the first that decides; an empty array answers false (true).

  • Tests: arr_some_every.

a.findIndex(pred): number answers the first index pred accepts, or -1. There is no find: TypeScript types its answer T | undefined and Nish has no undefined, so xs.find(f) is refused and points here (`find` is not supported: TypeScript types its answer `T | undefined` …, NL2697).

  • Tests: arr_find_index; reject_arr_find.

a.slice(start: number = 0, end: number = a.length): T[] answers a copy of a[start..end), with s.slice's rule: 0 <= start <= end <= a.length, or the program panics with slice out of range: [start, end) of length len and exits 1, where JavaScript counts a negative index from the end and clamps the rest. It takes zero to two arguments (NL2699), each a number, where s.slice takes any integer type. It works on a readonly T[], and the copy is a mutable T[].

  • Tests: arr_slice, arr_slice_bounds; reject_arr_slice_args.

s.length is a number, the UTF-8 byte length.

  • Tests: str_length.

s.charCodeAt(i: number): number answers the byte at i, bounds-checked against s.length exactly as a[i] is — out of range panics and exits 1, where JavaScript answers NaN, which number cannot hold.

  • Lowering: no call: a load i8.
  • Tests: str_bytes; reject_str_char_code_arity.

s.substring(start: number[, end: number]): string answers the bytes of [start, end), end defaulting to s.length. Both ends are clamped into [0, s.length] and then swapped into order, as in JavaScript, so s.substring(5, 0) is s.substring(0, 5) and a negative offset is 0.

  • Lowering: one allocation and one memcpy (nish_str_new). The clamp is two llvm.smin / llvm.smax calls per bound and is not emitted for a bound the compiler can place in [0, s.length], because a clamp cannot move a value that is already inside its range: the proof is the same one that removes a bounds check (Element access), and a literal 0 is proven for every string, since none has a negative length. Each bound is proven against the facts that hold where that bound is evaluated — the arguments run left to right and the receiver's length is read before either of them, so an assignment in argument 1 cannot prove argument 0 and an argument that rebinds the receiver takes the proof with it (perf_clamp_order, perf_clamp_rebind). A bound it could not prove keeps the clamp and is reported by the performance warning below.
  • Tests: str_bytes, perf_clamp_quiet, perf_clamp_order, perf_clamp_rebind; reject_str_substring_arity.

s.slice(start: number[, end: number]): string answers the same bytes, without the clamp: start and end must satisfy 0 <= start <= end <= s.length and anything else panics with slice out of range: [start, end) of length len and exits 1, where JavaScript would count a negative offset from the end and answer "" for a reversed pair. end defaults to s.length, an empty range is "", and an in-range non-negative pair gives exactly what JavaScript's String.prototype.slice gives.

  • Lowering: two icmp ule against a cold panic block, one allocation and one memcpy. Measured 1.18x over substring on a lexer-shaped scan (wp15-performance.md §4).
  • Tests: str_slice, str_slice_panic; reject_str_slice_arity, reject_str_slice_type.

s.indexOf(sub: string[, from: number]): number answers the first byte offset at or after from at which sub occurs, or -1; from defaults to 0.

from is clamped into [0, s.length] first, as in JavaScript: a negative start searches from 0, and a start past the end finds nothing, except that the empty needle answers the clamped start, so s.indexOf("") is 0, s.indexOf("", 3) is 3 and s.indexOf("", 99) is s.length. A float start converts as JavaScript's ToIntegerOrInfinity does — a fraction truncates and NaN is 0 — and an unsigned one is clamped from above only, so a u64 past 2^63 is past the end.

Under Node the program is the same call to String.prototype.indexOf, whose answers differ only where every offset does, on text that is not ASCII (NL8001, under "Portability warnings").

  • Lowering: one nish_str_index_of or nish_str_index_of_from call and no allocation: the search is the runtime's, so it is the libc's vectorised one rather than a probe per offset (WP15 §7c), and the clamp is a llvm.smin / llvm.smax pair in the caller, which a literal 0 does without. A loop that walks every match, at = s.indexOf(sub, at + sub.length), therefore copies nothing.
  • Tests: str_search, str_index_of_from, str_index_of_from_port; reject_str_index_of_type, reject_str_index_of_arity, reject_str_index_of_from_type.

s.startsWith(sub: string): boolean answers whether sub's bytes are a prefix.

  • Lowering: nish_str_at.
  • Tests: str_search.

s.endsWith(sub: string): boolean answers whether they are a suffix; a sub longer than s is false.

  • Tests: str_search.

String.fromCharCode(c: number): string answers the one-byte string of c & 0xFF, the inverse of charCodeAt. A value above 127 makes a byte that is not valid UTF-8 on its own; nothing validates it.

  • Tests: str_search.

String.fromBytes(bytes: u8[], start: number, end: number): string answers the bytes of bytes[start..end) as one string, where a loop of String.fromCharCode and a join allocates once per byte. bytes is a u8[] reached any way an array is (a local, a field, a call, a ternary, a literal), readonly or not, since the call only reads it; anything else, a wider element or a u8[] | null included, is `String.fromBytes` expects an argument of type u8[], got i32[] (NL2268). Both ends are required (NL2060) and are numbers (NL2058).

The range is s.slice's, 0 <= start <= end <= bytes.length, checked once after all three arguments are evaluated, so an argument that grows or shrinks the array is checked against what it left; anything else panics with slice out of range: [start, end) of length len and exits 1.

As with fromCharCode, nothing validates the bytes as UTF-8: a multi-byte character copied whole is that character, and a range that cuts one is bytes that are not a valid string. Natively the string's bytes are the array's, so write or writeFileSync sends any bytes back out unchanged (File I/O, io_write_bytes). Under Node, where String.fromBytes is the prelude's (runtime/nish.mjs), each byte is one UTF-16 unit, as fromCharCode makes it, so ASCII agrees and a byte above 127 does not (a C row, wp33-round-trip.md §3.2); under --emit-ts the bytes decode as UTF-8, a sequence that is not UTF-8 to U+FFFD. Bytes that are not text go out through writeBytes and writeFileBytesSync, which agree on every host. src/ does not call it until the release that ships it (the rolling freeze).

  • Lowering: one range check, one allocation and one memcpy (nish_str_new).
  • Tests: str_from_bytes, str_from_bytes_ways, str_from_bytes_order, str_from_bytes_lifetime, str_from_bytes_arrow, str_from_bytes_oob; reject_str_from_bytes_type, reject_str_from_bytes_elems, reject_str_from_bytes_start, reject_str_from_bytes_index, reject_str_from_bytes_arity, reject_str_from_bytes_arena, reject_deny_panics_from_bytes.

Every offset above is a byte offset, like s.length: a lexer walks bytes, and a code-point index would need a decode per access (codePointLength in nish/text counts code points natively, and under Node for any string that is not built from bytes). Cutting a multi-byte character in half is therefore possible and produces bytes that are not a valid string (docs/wp13-differential.md notes what Node then does with them).

A numeric literal argument to push, indexOf or fill takes the element type, so wide.push(3) on an i64[] is an i64 three (Numeric literals).

Sorting with a comparator

xs.sort(cmp) sorts with a function the program passes. The checker rewrites the call to sortWith(xs, cmp) from std/sort.ts, which the program does not import: a .sort( call is what loads it. Because cmp is a compile-time function parameter, each comparator gets its own instantiation whose inner loop calls it directly, with no function pointer and no boxing of its answer. The sort is stable, so for a comparator that is a consistent total preorder there is exactly one answer and it is Node's; under Node and --emit-ts the call is the platform's own Array.prototype.sort. A 10,000 element array of records with many ties (arr_sort_cmp_f64) prints the same checksum natively and under Node.

The sort costs one scratch array of n elements, allocated once at full size (new Array<T>(n) for any T, which sortWith alone may write: the copy after it fills every slot before one is read) from the function's arena and released before sortWith returns, so a sort leaves Arena.used() where it was and a loop that sorts keeps its per-pass scope (arr_sort_edge). The scratch is not on the stack: its length is the array's, known only at run time. Insertion sorts runs of 32 from the scratch into the array, then bottom-up passes merge pairs of runs back and forth between the two. A pair already in order (the last of the left run not after the first of the right) is copied without a comparison per element, so a merge pass over sorted input costs one comparison per pair. Every move is a whole slot: an array of interface records, which is stored inline, moves a record as one copy of its bytes (arr_sort_cmp_record), and the sort never holds an element across a write, which would read a record half overwritten. Every index inside the sort is proven in range by a guard beside it, so the sort keeps no bounds check, and at -O2 the comparator is inlined into the loops that call it.

Where JavaScript leaves the order to the implementation, so does Nish: an array holding NaN under (a, b) => a - b, or a comparator that is not a consistent order (one answering 1 for both (a, b) and (b, a)), comes out a permutation of its elements, but not necessarily Node's permutation. A program meant to agree with Node gives NaN a place of its own in the comparator.

The idiom (a, b) => a - b is checked arithmetic on i32: two elements more than 2^31 - 1 apart, as in [2147483647, -1], panic with attempt to subtract with overflow. On values that may span the whole range, compare instead of subtracting: (a, b) => (a > b ? 1 : a < b ? -1 : 0).

What differs from JavaScript is refused rather than reinterpreted: no comparator (JavaScript would compare the elements' strings), a readonly receiver or process.argv, and a comparator that reads a let of the function it is written in (reject_arr_sort_capture_let). A comparator is an arrow argument like any other, so it may read that function's const locals and parameters (arr_sort_capture), under the rule in Function parameters.

Callback methods and slice

xs.map(f), xs.filter(f), xs.some(f), xs.every(f), xs.findIndex(f), xs.forEach(f) and xs.slice(...) are lowered as sort is: the checker rewrites the call to a template of std/array.ts, which a call of one of those names loads, and the callback is a compile-time function parameter, so each callback gets an instantiation of its own whose loop calls it directly. xs.map(f) and map(xs, f) with map imported from nish/array are the same call and compile to one instance (arr_map_template). The template is picked by the callback's arity: map for (x) => ..., mapIndexed for (x, i) => ..., and slice, sliceFrom or sliceAll by the argument count.

xs.forEach(f) answers void, so its answer cannot be bound (reject_foreach_answer), and its callback is written to answer nothing: a concise body such as (x) => console.log(x) or (x) => ys.push(x) is a statement, and what it computes is dropped. It compiles to a counted loop over the array with the callback inlined into its body, as for...of does, but the loop stops at the length the array had when the call began, as Node does, where for...of reads the length on every pass. With a callback that does not resize the array, opt -O2 leaves the same instruction count for both: 18 on the walk function measured in IR_COOKBOOK.md. An arrow argument never assigns a name of the function it is written in (NL2586), and a forEach callback is no exception, so a running total is a for...of loop, whose body may assign it; the refusal says so (reject_foreach_capture_sum): for (const x of xs) { sum += x }.

The callback takes the element, or the element and its index, as TypeScript allows. JavaScript passes a third argument, the array itself; Nish refuses a callback that declares one (NL2698), since the array is the receiver the call already names. A callback that declares no parameter, xs.map(() => 0), is refused too, though TypeScript accepts it: the template is picked by the callback's arity, and there is none for zero, so write (_x) => 0. The callback rule is the comparator's: a top-level function, or an arrow that reads only the const locals and parameters of the function it is written in (reject_arr_map_capture_let).

Each method walks the array from index 0 and calls the callback once per element, in order, up to the length the array had when the call began, as Array.prototype does; some, every and findIndex stop at the first element that decides. A callback that shrinks the array ends the walk at the new length, where JavaScript goes on and skips the missing indices. So a forEach callback that pushes onto its receiver does not visit what it pushed, one that stores ahead of the walk is handed the stored value, and one that pops stops the walk early, as Node does (arr_foreach). map and filter answer a new array, so writing into it does not touch the receiver; an element that is an interface record is stored inline and copied, so filter(...)[0].id = 10 leaves the receiver's record as it was, as it would under Node (arr_filter).

There is no find. TypeScript types xs.find(f) as T | undefined, and Nish has no undefined: a program testing r !== null would pass tsc and mean something else under Node. The call is refused with a message that points to findIndex (NL2697, reject_arr_find): const i = xs.findIndex(f); if (i >= 0) { ... xs[i] ... }.

slice panics where JavaScript clamps. The bounds rule is string slice's: 0 <= start <= end <= xs.length, and anything else exits 1 with slice out of range: [start, end) of length n (arr_slice_bounds). A negative index, which JavaScript counts from the end, and an end past the length, which it clamps, are both outside. A number bound under --number-mode f64 is made an integer before the check, as llvm.fptosi.sat does: a fraction is truncated toward zero (xs.slice(0.9, 2.5) is xs.slice(0, 2)), NaN is 0, and ±Infinity or a magnitude of 2^63 or more saturates to the i64 limits, so it fails the check and the message prints the limit (slice out of range: [-9223372036854775808, 9223372036854775807) of length 3; arr_map_f64, arr_slice_saturate). The bounds are number only, where string slice takes any integer type: an i64 or u32 bound is refused (Argument 2 of `slice`: expected i32, got i64; reject_arr_slice_i64_bound), so convert it first: with toI32(n) in the default i32 mode, and with Number(n) under --number-mode f64, where the message reads expected f64. Neither conversion works in the other mode. An array's slice is not a slice panic site: its check is std/array.ts's own panic(...), so a call to it is a call site that reaches panic, which no proof at the call removes, and --deny-panics refuses it (reject_deny_panics_arr_slice). Under Node and --emit-ts the call is the runtime's sliceArray, which applies the same check with the same words, so the two agree on every input; map, filter, some, every, findIndex and forEach are the platform's own Array.prototype methods there, and an answer that holds records inline is copied with the runtime's copyRecords, as the native push copies.

Bulk writes: set and fill

The reference for set and fill, with their parameters and their answers, is their doc comments on Array in runtime/nish.d.ts; the rules are here.

A packet is copied, cleared and filled through u8[] without a loop per byte (WP34 N2). Both methods take their meaning from TypedArray.prototype, and both are admitted on an array of numbers only — u8[] above all, but any width, f64[] and a ranged integer<Lo, Hi>[] included — because an element there is a fixed-size value, so the lowering is one copy or one store of bytes. On any other array each is `set` copies bytes, so it needs an array whose elements are numbers (`u8[]`, `i32[]`, `f64[]`, ...), got string[] (NL2395, reject_bytes_set_string, reject_bytes_fill_record): a string or a class element is a pointer a byte copy would share, and a record is a struct stored whole.

  • a.set(src, offset) copies all of src into a starting at offset, which defaults to 0. src has a's own element type — a readonly array may be the source, since it is only read — and anything else is `set` copies from an array of the receiver's element type, so it expects u8[], got i32[] (NL2396, reject_bytes_set_type), because a conversion per element is not a byte copy. One or two arguments (NL2397, reject_bytes_set_arity).
  • The range written is checked. 0 <= offset and offset + src.length <= a.length, or the program panics with slice out of range: [offset, offset + src.length) of length a.length and exits 1 — the same check and the same message as s.slice, since it asks the same question of a range (bytes_set_oob). A range that ends exactly at a.length, and an empty src at a.length, are in range (bytes_set_end). JavaScript throws a RangeError for exactly these offsets.
  • An overlapping copy behaves as if src were read first, which is TypedArray.prototype.set's promise, so set lowers to llvm.memmove. Nish itself can overlap an array only with itself (a.set(a)), but a C host may hand in two headers over one buffer, and then the copy is right in either direction (bytes_set_overlap). It lowers to llvm.memcpy only where the checker proves the two buffers distinct, and records the proof for the emitter: when either operand is an array literal or a new written at the call, or when both are different const locals each initialised with a fresh allocation — a literal, a new or a readFileBytesSync — since a second name for either buffer would not be such a local (bytes_set_disjoint). A parameter, a field, a let and a self-copy are "not proven", which is the memmove and never wrong (bytes_set_self).
  • a.fill(value, start, end) stores value into every slot of [start, end). value is the element type (`fill` expects u8 (the element type of u8[]), got string, reject_bytes_fill_value), and a ranged element checks it once, before the fill. start defaults to 0 and end to a.length; a negative end counts back from a.length, and each is then clamped into [0, a.length], so no range is out of range, a reversed range fills nothing, and fill never panics (bytes_fill). One to three arguments (NL2398, reject_bytes_fill_arity), each end a number (reject_bytes_fill_index).
  • A float offset is converted as JavaScript's ToIntegerOrInfinity converts it, for set's offset and for both of fill's ends: an f64 (every number in f64 mode) or f32 goes through llvm.fptosi.sat, never a bare fptosi, whose answer for NaN or an infinity is poison. NaN is 0, a fraction truncates toward zero (2.7 is 2, -0.5 is 0), and an infinity or a value past the i64 range saturates to the i64 bound on its side. fill then clamps that bound to an end of the array, so in f64 mode a.fill(v, 0, 1 / 0) fills all of it, as Uint8Array.prototype.fill does (bytes_offset_float); Infinity itself is not a name in Nish. set fails its range check with the saturated offset in the message — slice out of range: [9223372036854775807, -9223372036854775808) of length 4 for 1e300 or an infinity, where the end has wrapped — and exits 1 where JavaScript throws a RangeError (bytes_set_float_oob, bytes_set_float_panic).
  • An unsigned offset is never negative, whatever its width. A u64 of 2^63 or more has the i64 sign bit set, and fill still reads it as past the end: as a start it fills nothing, and as an end it fills to the length (llvm.umin, where a signed clamp would count it back from the end). set's range check compares unsigned, so the same offset panics, printed signed as every range panic prints (bytes_offset_u64, bytes_set_u64_oob). An i64 at INT64_MIN or INT64_MAX keeps the signed clamp above. The TypeScript reading of a 64-bit offset is class C: i64 and u64 are bigint under runtime/nish.mjs, and fill or set with a bigint offset throws a TypeError there, as all 64-bit arithmetic does (RUN_UNDER_NODE.md).
  • fill on a u8[] is one llvm.memset. On a wider element it is a counted store loop of the one value, the shape LLVM vectorises; opt -O2's loop-idiom pass makes a memset of that loop by itself when the value is a repeated byte, zero above all, which tests/run.js checks on bytes_fill_zero.
  • Both are statements, as writeFileSync is: `set` returns void and can only be used as a statement (reject_bytes_set_value, reject_bytes_fill_value_position). set answers undefined in JavaScript and fill answers the array the program already holds, so refusing the value costs nothing and keeps the two readings the same.
  • Both write their receiver, so a readonly array refuses them under their own names (Cannot `set` through readonly u8[] (declare it u8[] to write through it), reject_bytes_set_readonly, reject_bytes_fill_readonly), a scope's region refuses them on memory a task may read (reject_thread_region_fill, reject_thread_region_set_call), and a parameter they write is never readonly. Neither moves length, so neither disturbs a hoisted header or a length the bounds proof holds.
  • A window into a buffer is (buf, offset, length) by convention, not a view type: a second array layout would be one every array path in the emitter has to handle (wp34-network-stack.md §4 N2).

The TypeScript reading (wp33-round-trip.md §2): fill is class A — a u8[] is a plain Array under Node, and Array.prototype.fill has TypedArray.prototype.fill's meaning for every argument Nish admits. set is class A under runtime/nish.mjs, which adds the one method Array lacks to Array.prototype with the copy-first semantics and the native panic, and class C in a TypeScript host without it, where a plain array has no set (wp33-round-trip.md §3.3). Every bytes_* program with a main runs under both and agrees byte for byte.

Readonly arrays

The reference for the members a readonly T[] has is their doc comments on ReadonlyArray in runtime/nish.d.ts; the rules are here.

readonly T[] — or ReadonlyArray<T>, the same type under the other spelling — is a T[] that may not be written through. It is the same header, the same pointer and the same LLVM type; the golden (tests/cases/arr_readonly_param) is the IR the mutable spelling emits, instruction for instruction, because the annotation is a promise to the checker and not a value.

  • Everything that reads still works: a.length, a[i], for...of, indexOf, join, map, filter, some, every, findIndex, forEach, slice, passing it on to another readonly T[].
  • Everything that writes is refused: a[i] = v and a[i] op= v are Cannot assign to an element of readonly i32[] (declare it i32[] to write through it) (reject_arr_readonly_store), and push / pop are Cannot `push` through readonly i32[] (...) under their own names (reject_arr_readonly_push, reject_arr_readonly_pop), and so are set and fill (reject_bytes_set_readonly). a.length = n was already refused for every array.
  • A T[] widens into a readonly T[], and never back. The conversion emits nothing, because the two are the same pointer; the reverse is Argument 1 of `fill`: expected i32[], got readonly i32[] (reject_arr_readonly_widen), since laundering the promise away one call deeper would make it worth nothing.
  • It is shallow, as TypeScript's is: the element of a readonly T[][] is a mutable T[].
  • It is a type, not a parameter modifier, so it is legal wherever an array type is — a class field, a return type, a const's annotation — and the rule travels with it (arr_readonly_field, reject_arr_readonly_field). That is why the message says "declare it" rather than naming a parameter.
  • readonly goes on an array type and nothing else. TypeScript allows the modifier on array and tuple types only (TS1354), so readonly i32 is `readonly` is only permitted on an array type, got i32 (reject_arr_readonly_scalar) rather than a construct waiting to be supported — a program that used it would not be TypeScript either.

The reason to write one is the C ABI. --emit-header already spells an array parameter const nish_array * when the whole-program fixpoint proves nothing stores through it, which makes const a consequence that can quietly disappear when a callee three levels down starts writing. On a readonly T[] it is the signature keeping a promise instead, and the comment above the prototype shows the annotation that earned it (tests/cases/arr_readonly_header, checked in the WP8 block of tests/run.js):

/* sum(xs: readonly number[]): number -- xs: int32_t elements */
int32_t sum(const nish_array *xs);
/* fill(xs: number[], v: number): void -- xs: int32_t elements */
void fill(nish_array *xs, int32_t v);

The fixpoint is not consulted for a readonly T[] at all, and that is deliberate rather than a shortcut. writesThrough is a conservative may-write: every escape sets it, on the grounds that an alias might be written through later, so a readonly parameter that is merely returned or stored in a field would land in the written set and lose the const its annotation promised (tests/cases/arr_readonly_escape). The checker is what makes the promise true — no store, no push, no pop, and no widening back to a mutable T[] — and it is exact where the fixpoint is not (writtenArrayParams, src/interop-abi.ts).

process.argv is read-only under a rule of its own rather than this type (`process.argv` is read-only), because it is a value and not an annotation.

There are no other array or string methods (shift, find, reduce, toUpperCase, charAt, ...) and no toString (template literals and console.log convert numbers); string-to-number parsing is the bare Number(s) / parseInt(s) / parseFloat(s) under Numeric conversions. An unknown name is Unknown method `codePointAt` on string or Unknown method `shift` on i32[], each with the supported list (tests/cases/reject_str_method_unknown, reject_arr_method_unknown).

Typed-array names have no push or pop

The reference for the four names is the doc comment on Array in runtime/nish.d.ts; the rules are here.

Int32Array, Float32Array, Float64Array and BigInt64Array are the same type as i32[], f32[], f64[] and i64[], but in TypeScript a typed array has a fixed length and no push or pop, so a call to either is a TypeError there. A receiver spelled with one of the four names refuses both (NL2415):

`push` is not a method of `Float64Array`, whose length is fixed: declare it `f64[]` to push and pop

(tests/cases/reject_typed_push_i32, reject_typed_push_f32, reject_typed_push_f64, reject_typed_push_i64 and their reject_typed_pop_* twins). The rewrite is the one the message names: a binding that grows is declared f64[], which is what it meant. Every other array operation is unchanged, and so is the type: sameType still holds, a Float64Array is an f64[] wherever one is expected, and no IR moves.

The spelling is a fact about the program text, not a type. A receiver carries it when it is:

  • new X(n) itself, for the four names;
  • an identifier whose let, const or parameter is annotated with one of the names — directly, in parentheses, as the one member of X | null, or through a type alias declared or imported in the module (reject_typed_push_param) — or is unannotated and initialised with a receiver that carries it: const t = new Float64Array(n), which TypeScript infers as the typed array (reject_typed_push_inferred), or const u = t;
  • a field read o.f whose field is declared with one of the names;
  • a call whose callee declares one of the names as its return type (reject_typed_pop_receivers has one of each). A field or a return type written through a type alias carries it in every module that reads it, including one that does not import the alias, because the declaring module reads the spelling where its aliases are (tests/link/reject_typed_push_alias);
  • a call of a generic function whose return type is a type parameter — or an array of one — that an argument binds to a receiver carrying it: first(rows) with first<T>(xs: T[]): T and rows: Float64Array[], same(new Int32Array(n)) with same<T>(x: T): T. The spelling is the call's, not the instantiation's: first(plain) over an f64[][] is the same instantiation and is not refused (reject_typed_push_generic);
  • a value read out of a Map whose value type is spelled with a name, whether the Map is annotated Map<string, Float64Array>, made with new Map<string, Float64Array>() or is a field so declared: m.get(k) once narrowed, m.get(k) ?? d, and the variable of a for...of over m.values() (reject_typed_push_map). A Set holds no arrays and a Map's keys are none, since an array is not a key;
  • an element of an array spelled with a name: rows[i] or a for...of variable over rows, where rows is any of the above spelled Float64Array[] or Array<Float64Array>, or is an array literal of receivers that carry it, [new Float64Array(n)] (reject_typed_push_element). The array itself is an ordinary array, and rows.push(new Float64Array(n)) compiles;
  • a ternary or a ?? either of whose operands carries it: c ? xs : t is number[] | Float64Array to TypeScript, which has no pop either (reject_typed_push_ternary).

What does not carry it is a member of a generic class instantiated at one of the names: new Box<Float64Array>(t).value and a method of it that returns T still push natively where the same source throws a TypeError under Node, because a binding records one spelling and not one per type argument. Declare such a value f64[] where it is bound, which is what TypeScript needs to accept the push as well.

A typed array that flows into a binding spelled T[] may be pushed there: const a: f64[] = t; a.push(v), or a Float64Array passed to a parameter xs: f64[] that pushes (tests/cases/arr_typed_push_pop, whose IR is the f64[] program's). The refusal follows the spelling the program wrote, and TypeScript itself refuses const a: number[] = new Float64Array(n), so no program tsc accepts reaches that push.

Arena

The reference for Arena.used, Arena.mark and arena(), with their answers and their capabilities, is their doc comments in runtime/nish.d.ts; the rules are here.

Reading the arena the runtime allocates from (wp6-memory.md). The automatic placement described under Memory model frees what a program allocates; using a = arena() below frees a batch by hand, with the proof done by the compiler. These two read the arena's position, and the memory tests watch it through them. Both are available in both number modes. Arena.release and Arena.reset, which freed to a mark with no proof, were removed before 1.0: a call to either is an unknown builtin (NL2244, the arena_removed block of tests/run.js).

Signature Semantics Effect Test
Arena.mark(): i64 the current bump address (nish_arena_mark; 0 while the arena is empty) write mem_callee_scope_control, mem_reclaim_guards
Arena.used(): i64 bytes bumped in the current chunk (nish_arena_used); the number the memory tests watch write mem_loop_scope_control, mem_scope_dynamic_array

mark and used only read the arena, but they are recorded as writes so that a caller is never hoisted across an allocation, and a function that calls either — itself or through a callee — is never the callee of a tail call whose scope release moved ahead of it, so the position it reports is the one it would have reported before (Memory model, item 2). A function that calls Arena.mark in its own body (managesArena) never gets an automatic arena scope, for the function or for a loop's pass, so the position it reads is never moved under it by a scope it did not write.

using a = arena()

A block that declares using a = arena() takes the arena's mark where the declaration stands and releases to it on every exit of the block, and the compiler refuses the block if anything allocated after the declaration could still be reachable once it has released. It is the per-pass scope of item 2 written by the program, held to the same rule with the block as the pass, and refused rather than quietly dropped when the rule fails.

const tally = (rows: string[]): i32 => {
  let total = 0;
  for (const row of rows) {
    using a = arena();                // the mark
    const line = `${row}: ${row.length}`;
    total = total + line.length;      // a number leaves the block; the string does not
  }                                   // released here, and on every other exit
  return total;
};
  • arena() is a global builtin declared in runtime/nish.d.ts as declare function arena(): Disposable, so a program stays TypeScript; under Node its disposal does nothing. It is only the initialiser of a using declaration, which is a statement of a block (the using placement rule); anywhere else it is `arena()` must be the initialiser of a `using` declaration, `using a = arena()`: the arena releases what was allocated after it when the block that declares it ends, and an `arena()` written anywhere else has no block to end (NL2416, tests/cases/reject_using_arena_not_using). The binding holds the mark and is never read: `a` is the binding of a `using` declaration of `arena()` and cannot be read: it holds the mark its block releases to when it ends, and nothing but that release may see or move the mark (NL2417, reject_using_arena_binding).

  • The release is emitted at every exit of the block: its end, a return (after the returned value is computed, so the value may read what the block allocated), a break or continue that leaves it, and an orReturn() (tests/link/arena_using_exit_paths, tests/cases/mem_using_arena, mem_using_arena_or_return). It nests with everything else on the same stack, innermost first: a using s = scope() inside the block joins before the block releases, and a block inside a scoped loop pass or a function's automatic scope releases before they do; a return releases only the outermost of them, whose mark is the lowest. A return g(n) whose arguments are all numbers is a tail call with the release moved ahead of it, unless g reads the arena (mem_using_arena_tail). A panic ends the process. Statements of the block before the declaration are outside it.

  • Nothing allocated inside may outlive the block. From the declaration to the end of the block, each of these is refused, naming the value and where it would go:

    Route Refused when Code Test
    assigned to a local declared outside the block the value is not older than the block (a literal, a parameter or outer local, a field or element read, a call that allocates nothing) NL2418 reject_using_arena_outer_local
    stored into a field, o.f = v v is not older than the block, unless the field is a number, a boolean or an enum NL2419 reject_using_arena_field
    stored into an element, xs[i] = v likewise NL2420 reject_using_arena_element
    stored into memory any other way: handed to a callee that keeps it, held by an object or array literal always: the escape analysis stops following a stored value NL2421 reject_using_arena_stored
    push onto an array the array is not a const the block binds to a fresh array, whatever is pushed: a push moves the array's storage into the arena NL2422 reject_using_arena_push
    return v, r.orReturn() v is not older than the block; orReturn() unless the function returns a Result packed into a register, whose payloads are numbers NL2423 reject_using_arena_return, reject_using_arena_or_return, reject_using_arena_declarator_or_return
    a call to a function that stores an allocation into memory always NL2424 reject_using_arena_callee_stores
    a call the analysis has no facts for always; every callee has facts today NL2425 —
    s.spawn(...) on a scope declared outside the block always: the task runs, and stores, when that scope joins, after the release NL2427 reject_using_arena_spawn
    s.spawn(...) on a scope the same using declares ahead of arena(), using s = scope(), a = arena() an argument is not older than the block: s is disposed after the arena, so it joins after the release NL2427 reject_using_arena_spawn_declarator
    the same, ahead of the statement's last arena(), using a = arena(), s = scope(), b = arena() an argument is not older than the block: b releases before s joins NL2427 reject_using_arena_spawn_last

    A number, a boolean or an enum leaves the block freely, and so does a value older than it. The rule is the escape analysis's, so it is conservative: a block that stores a fresh string into an object it also allocated, or builds a Map, is refused even though nothing escapes, and what has to outlive the block is allocated before it opens. The first route reads `` `item ${i}` `` may be allocated inside the `using arena()` block at 6:5 and is assigned to `last`, which is declared outside the block: the block releases what it allocated when it ends, and the local would outlive it pointing into freed memory (reject_using_arena_outer_local).

  • A parallel body or a task may not open one, for the reason it may not call Arena.* (reject_using_arena_parallel, NL2351).

  • A map that answers numbers or strings may run inside one. The map's region joins before the call returns, so its result array is the block's alone, and the blocks of strings it adopts from its workers are the block's newest memory; the release frees both (tests/link/mem_using_arena_map). What the map answers may not outlive the block, as anything else allocated in it may not (tests/cases/reject_using_arena_map_escapes). A map whose body answers an array stores each one into its result, which the analysis stops following, as it does for a loop that pushes arrays into an array, so the block refuses it as NL2424 and says what to write instead: run that map outside the block (tests/cases/reject_using_arena_map_arrays).

  • A task filed on a scope the block declares may read what it allocated. That scope joins before the block releases, inner using s = scope() or using a = arena(), s = scope() alike, so the block may hand its task an array or an object it made (tests/link/thread_scope_reclaim). A scope declared ahead of the statement's last arena() is not one: that arena releases first (reject_using_arena_spawn_last). The task is followed as any callee is: one that answers its argument into an older destination stores it (NL2424, tests/cases/reject_using_arena_spawn_relay, as reject_using_arena_spawn_stores for any array answer), and a constructor that makes the argument and hands this out stores it too (reject_using_arena_spawn_constructor).

  • It costs no scope. A function that declares one keeps its automatic scope and its loops' pass scopes, because the block's mark is checked like theirs; only Arena.mark gives those up. And the arena-loop warning, NL9011, says nothing about a call inside the block, whose leftovers the block releases (tests/cases/perf_arena_using).

Map and Set

The reference for each member of Map and Set, with its parameters and its answer, is its doc comment in runtime/nish.d.ts; the rules are here.

The global Map<K, V> and Set<T> are JavaScript's: insertion-ordered, with keys compared by SameValueZero, so -0 is +0 and NaN is NaN. They need no import. They are generic classes written in Nish in std/collections.ts, which a module loads by naming Map or Set in a type or after new, and the layout and every decision below are wp32-map.md's.

export const main = (): i32 => {
  const seen = new Set<string>();
  const counts: Map<string, i32> = new Map();
  seen.add("a").add("b").add("a");
  counts.set("a", 1).set("b", 2);
  counts.delete("b");
  return seen.size + counts.size;   // 3
};
  • The members are JavaScript's, less what this version defers. A Map has size, get, set, has, delete, clear and forEach; a Set has size, add, has, delete, clear and forEach. set and add answer the receiver, so they chain, and delete answers whether the key was there (tests/cases/map_key_str, set_str). size is a read-only number: `size` of `Map<string, i32>` is read-only (reject_map_size_assign). get answers V | undefined, which has rules of its own, below. keys() and values() are iterators, and an iterator is only ever the iterable of a for...of, below. There is no entries, because a [key, value] pair is a two-type tuple (reject_map_entries, reject_map_iter_entries_call): m.forEach((value, key) => ...) visits each key with its value.

  • forEach is the for...of walk with the callback inlined. m.forEach(f) on a Map calls f with each value, and with its key too when f is written with two parameters, value first, as JavaScript does; s.forEach(f) on a Set calls f with each element. The checker rewrites the call to a template of std/collections.ts (forEachEntry, forEachValue, forEachElement), whose body is the same walk as for...of over the table, below: the same insertion order, an entry set during the walk visited and one deleted before the walk reaches it not, and clear ending it (map_foreach, set_foreach). It allocates nothing and looks no key up, where for (const k of m.keys()) with m.get(k) ?? 0 calls Map.probe and takes the nullish branch once per key, and rewinds the arena on every pass: under opt -O2, 92 instructions against 112 for a Map and 78 against 89 for a Set (IR_COOKBOOK.md). JavaScript passes a Map's callback the map as a third argument, and a Set's callback the element twice and the set third; Nish refuses a callback written with more than two parameters on a Map (NL2771, reject_map_foreach_third_param) or more than one on a Set (NL2772, reject_set_foreach_second_param), since the receiver is the one the call already names. The callback follows the array callbacks' rules above: a top-level function or an arrow written at the call (reject_map_foreach), at least one parameter, an answer of void, no assignment to a name of the function around it (NL2586), and m.forEach named without a call is refused (reject_map_foreach_uncalled). An arrow inside another arrow reads only its own parameters, so a nested walk goes through a top-level function (set_foreach). Everything else in the class is the table's own and is refused as if it did not exist: Unknown member `mask` on `Map<string, i32>` (reject_map_internal_field, reject_map_internal_method, reject_set_set).

  • new takes no arguments, because a class of this language has one constructor and no optional parameter: `new Map<string, i32>` takes no arguments in this version (reject_map_new_args, reject_set_new_args). The type arguments are written out, except that a declaration annotated with the type takes them from the annotation, as tsc infers them there — const m: Map<string, i32> = new Map() and the same with let (tests/cases/map_annotated_new); a bare new Map() is `Map` is generic: it must be written with its type arguments (reject_map_new_no_type_args).

  • A key is a string, a number of any width in either mode, f32, f64, a boolean, an enum or a class instance, the last by identity (tests/cases/map_key_str, map_key_i32, map_key_i64, map_key_u8, map_key_u16, map_key_u32, map_key_u64, map_key_f64, map_key_f32, map_key_bool, map_key_enum, map_key_class). An interface, an array, a nullable type and a Result are not keys: `Point` cannot be the key type of `Map<Point, i32>`: a key is hashed and compared by value, or by identity for a class instance (reject_map_key_interface, reject_map_key_array, reject_map_key_nullable, reject_map_key_result, and for a Set element, reject_set_key_interface). A float key is SameValueZero, and so is its hash: -0 and +0 are one key and every NaN is one key, and map_key_f64 prints under Node's own Map what it prints compiled (tests/differential/unmodified.js). A -0 key is stored as +0, as JavaScript stores it, so a walk over keys() or a Set yields +0 whichever zero was inserted, by every insert path: set, add, a fused update or guarded insert, and getOrInsert (map_key_negzero, which Node runs unmodified too, and map_key_f32_negzero for an f32 key).

  • A value is anything but void and an interface, whose record would be copied into the table rather than shared: `Point` cannot be the value type of `Map<string, Point>` (reject_map_value_interface, reject_map_value_void).

  • A string key is hashed under a per-process random key: SipHash-1-3 of its UTF-8 bytes, keyed with 128 bits drawn from the system's entropy source before main, so a program's input cannot choose keys that share a bucket. 4096 keys that collide under an unkeyed FNV-1a insert with no key compare (tests/cases/map_hash_collide, counted the same way as below), and two runs of one program hash under two keys. Iteration is still insertion order, so nothing a program prints depends on the key. When the entropy source fails the program panics before main: Map: the system's entropy source failed. On wasm32 the key comes from WASI's random_get, and a reactor module is keyed only once its host calls _initialize (docs/security/json.md, JSON-13).

  • An integer, boolean or enum key is hashed under a per-process random key too, drawn the same way: a key of 32 bits or fewer by multiply-add-shift, (a * x + b) >> 32, and an i64 or u64 key by a folded 128-bit multiply. 8192 i32 keys built to collide under 0.28's unkeyed fmix32 run within 8 times of as many ordinary keys, where 0.28 ran them about 215 times slower (tests/cases/map_key_int_collide). The i64 hash is keyed but not provably universal, and before _initialize a wasm32 reactor hashes every key of 32 bits or fewer to one bucket (docs/security/json.md, JSON-15). A float key and a class instance key are not keyed: a float hashes its canonical bits with fmix64 and an instance its address, so a program keying a table by floats a peer chose converts them to a string or a scaled integer first (JSON-16).

  • A pair of i32s is one i64 key, packed exactly for every pair: (toI64(hi) << toI64(32)) | toI64(toU32(lo)). toU32 takes lo unsigned, so it stays in the low half, so nothing is added and nothing overflows; with + in place of | the key leaves the i64 range at hi = -2^31 with a negative lo. Every operand is an i64 so the same line runs under Node, where an i64 is a BigInt (tests/cases/map_key_composite, both extremes included).

  • Every set, add, has and delete is one probe. A bucket holds eight bits of the key's hash beside the entry's index, and each entry keeps its full hash, so a probe reads a key only when both match: 10000 lookups of absent string keys make no key compare at all, and 10000 hits make exactly 10000 (tests/cases/map_fingerprint_miss, counted by tests/run.js through -Wl,--wrap=nish_str_eq). A table grows by doubling at three quarters full and never hashes a key again (map_growth); a deleted key leaves a tombstone that later probes walk past, and setting it again appends a new entry (map_delete_reinsert); and a table whose entries are mostly dead compacts in place instead of growing (map_compaction). A table holds at most 2^24 - 1 entries, which is what a 24-bit index field holds and one fewer than Node's 2^24: an insert at the cap first compacts away dead entries, and when there are none it panics with Map maximum size exceeded (or Set …) (CLI only: the case takes 16 million inserts).

  • Asking about one key twice is one probe (wp32-map.md §9.1). Three patterns keep the first probe's answer — the entry it found, or the empty bucket it stopped at and the key's hash — and write through it: setValueAt of the entry where the key was found, and insertAt the bucket with the hash already computed where it was not. The second hash and the second search are never made:

    Pattern Written Case
    update counts.set(w, (counts.get(w) ?? 0) + 1): the value asks about the key exactly once, with get or has map_fused_wordcount, map_fused_update_has
    guarded write if (m.has(k)) { m.set(k, E); … } or if (!m.has(k)) { m.set(k, E); … }, the set first in the branch map_fused_guard_has, map_fused_guard_not_has
    insert if absent if (!s.has(x)) { s.add(x); … }, the add first in the branch; the rest of the branch runs only where it inserted map_fused_set_insert

    The receiver and the key are spelled the same way in both calls, and each is a local, a parameter or a this.<field> path; the key may also be a literal (map_fused_this_field, map_fused_literal_key). Between the probe and the write — inside the value, and between the has and the branch's first statement — there is no call of any kind, no new, array or object literal, template literal or string +, and no assignment, ++, -- or compound assignment; &&, ||, ?? and the ternary may stay, and a ?? default still runs only where the key is missing. A pattern outside these rules is not an error: it compiles as the separate calls it is, one probe each, and prints what they print (map_fused_no_call, map_fused_no_assign, map_fused_no_key_spelling, map_fused_no_element_receiver, map_fused_no_template, map_fused_no_statement_before). The places are compared, not what they hold, so an alias of the receiver is not fused (map_fused_no_alias), and neither is a value that reads the key twice or reads another map (map_fused_no_two_gets, map_fused_no_other_map); a guard fuses the write that starts its then-branch, and a set in its else is an ordinary call (map_fused_no_else_branch). A pattern in a generic function or method is fused in every instantiation (map_fused_generic), and one inside a walk of the table it writes does what the separate calls do: a key it inserts is appended past the cursor and visited (map_fused_walk). tests/run.js counts one call of probe, no hash of its own and no get, set or has in the loop of map_fused_wordcount's golden (IR_COOKBOOK.md).

  • nish/map adds reserve(m, n) and getOrInsert(m, k, v), the two things a program asks of a table that JavaScript's Map has no call for (wp32-map.md §9.2). Their bodies in std/map.ts are what runs under Node — reserve does nothing, and getOrInsert is a get, and a set of v when the key was missing, answering the value either way — so a program that uses them runs unmodified there (examples/wordcount.ts). Natively neither body is called: every call is lowered in place, so std/map.ts writes no .ll of its own and a one-file program importing it is still one module.

    import { getOrInsert, reserve } from "nish/map";
    
    export const main = (): i32 => {
      const first = new Map<string, i32>();
      reserve(first, 64);                        // no rebuild until 64 entries
      const a = getOrInsert(first, "a", 1);      // 1, inserted
      const b = getOrInsert(first, "a", 2);      // 1, found: 2 is not stored
      return a + b + first.size;                 // 3
    };
    
    • getOrInsert is one probe: the value of the entry found, or insertAt of v through the bucket the probe stopped at. v is evaluated either way, as every argument is (tests/cases/map_extras_get_or_insert, whose loop tests/run.js counts as one probe and no getOrInsert call).
    • reserve is the table's reserveSlots, which doubles the bucket table until n entries, dead ones included, fit under the three-quarters bound, and re-files the buckets from the stored hashes. It never shrinks a table, a count that is not positive does nothing, and one past the 2^24 - 1 entry cap is the cap (map_extras_reserve). It moves no entry, so like getOrInsert it may be called during a walk of the table (map_extras_walk), and both are lowered in each instantiation of a generic caller (map_extras_generic) and in each module that calls them (tests/link/map_extras_two_modules). Only speed depends on it: each case prints under Node what it prints compiled (tests/differential/unmodified.js).
    • They are generic functions, checked as one: a first argument that is not a Map, or a key or value of the wrong type, is Argument 1 of `getOrInsert<i32, i32>`: expected Map<i32, i32>, got i32[] (reject_map_extras_not_map, reject_map_extras_key_type, reject_map_extras_value_type, and reject_map_extras_reserve_count for a count that is not a number); a missing argument is `getOrInsert` expects 3 argument(s), got 2 (reject_map_extras_arity); and a Set is not a table reserve takes: Cannot infer `K` for `reserve`: argument 1 is Set<string>, which does not match the declared `Map<K, V>` (reject_map_extras_reserve_set). The module has those two and nothing else: Module `nish/map` has no exported function `peek` (reject_map_extras_unknown_export, NL2376, the code any module's missing export now carries).
    • The lowering belongs to nish/map's two functions, found by the module that declares them and not by their names: a program's own reserve, or a getOrInsert from its own module ./map, is an ordinary function whose body runs (tests/link/map_extras_user_names).
  • std/collections.ts writes no .ll of its own. What a module uses of it is emitted into that module, internal, after the module's own functions, so a one-file program that names Map is still one module for -o <file.ll> and every instance is inlined, specialised or dropped with the rest of the module. Two modules that share one Map<string, i32> each call their own copy of its methods (tests/link/map_two_modules). Under -g each copy's subprogram names std/collections.ts (tests/cases/map_dbg). The symbols are the package's (@nish.Map$str$i32.set), and a map is filled and grown through a callee and across loop arena scopes like any other object (tests/cases/map_arena_callee).

  • m.get(k) answers V | undefined, as it does under tsc: a maybe. It is one probe, and a maybe is read in one of three ways, the three a program that type-checks under tsc --strict has (wp32-map.md §3):

    export const main = (): i32 => {
      const counts = new Map<string, i32>();
      for (const w of ["a", "b", "a"]) {
        counts.set(w, (counts.get(w) ?? 0) + 1);      // a default
      }
      const a = counts.get("a");                      // a `const`, tested
      if (a !== undefined) {
        console.log(`${a}`);                          // 2: `a` is an i32 here
      }
      return counts.get("z") === undefined ? 0 : 1;   // a test
    };
    
    • Where a maybe may stand: the initialiser of a const, unannotated or annotated with the maybe type; the left operand of ??; and an operand of === undefined or !== undefined, either way round (tests/cases/map_get_nullish, map_get_narrow). The type is spelled V | undefined or undefined | V, and Node | null | undefined for a nullable V (a scalar is no more nullable here than anywhere, reject_map_get_nullable_scalar), and only as the annotation of a const initialised from a call of get. On a let initialised from get it is refused as the unannotated let is (NL2361, reject_map_get_let_annotated), and anywhere else it is the union it looks like (reject_union_undefined), and undefined as a value is refused as it always was (reject_undefined_value, and in a module constant, which never holds a get result, reject_map_get_undefined_module_const).

    • A const bound to get is narrowed by the rules of Nullable types, with undefined in place of null: it reads as V in the region a test proves it present, if (a === undefined) return included, and nowhere else (map_get_narrow, reject_map_get_narrowing_leaks). A const is never assigned, so its narrowing never ends early; a let would need that rule, and is refused. Narrowed, it is still the maybe it was bound to, as under tsc: testing it again, or defaulting it with ??, reads the same found bit (map_get_narrowed_again). Compared with undefined, anything that is not a maybe is `undefined` is forbidden in Nish; use `null` with a `T | null` type (reject_map_get_undefined_compare).

    • a ?? d has TypeScript's precedence: the level of ||, with operands that bind as tightly as |, left associative (tests/parser/nullish). It does not mix with || or && without parentheses, as tsc reports TS5076: `||` and `??` cannot be mixed without parentheses is a syntax error (reject_nullish_mixed_or, reject_nullish_mixed_and), reported once however often an expression mixes them (reject_nullish_mixed_chain_or, reject_nullish_mixed_chain_and). d runs only when the value is missing, and it has type V, as does the whole expression: The right operand of `??` stands in for a missing value, so it must be `i32` (reject_nullish_default_type). When V is nullable, ?? replaces a stored null as well, as JavaScript's does, and the result is still V; !== undefined narrows to V, keeping the null (map_get_pointer_value). A ?? whose left operand is a T | null rather than a maybe is Nullable types' default, and one whose left operand is neither is refused in the words Phase 0 always used, Nullish coalescing `??` is forbidden in Nish (narrow with `!== null` instead) (reject_nullish, reject_nullish_nullable), which the nullish feature of the compatibility dialect admits.

    • Everywhere else a maybe is refused, and the message names the place, because the rewrite differs by place. Some fail under tsc too (an operand, a const annotated V); the rest are legal TypeScript that this version does not take, because a maybe never crosses a call, never reaches memory, and is narrowed only in a const (wp32-map.md §3.2):

      Place Message after `m.get("a")` is `i32 | undefined` Case
      a let, or an assignment to one and cannot be held in a `let` reject_map_get_let, reject_map_get_assign_let, reject_map_get_let_annotated (NL2361)
      an argument of a function, method, constructor or builtin and cannot be passed as an argument reject_map_get_argument, _argument_method, _argument_constructor, _argument_builtin (NL2362)
      a return, or a concise arrow body and cannot be returned reject_map_get_return, reject_map_get_const_read (NL2363)
      a field, or an object literal's and cannot be stored in a field reject_map_get_field, reject_map_get_object_field (NL2364)
      an array element, or an array literal's and cannot be stored in an array element reject_map_get_element, reject_map_get_array_literal (NL2365)
      a template literal hole and cannot be a template literal hole reject_map_get_template (NL2366)
      an operand of any other operator and cannot be the operand of an operator other than reject_map_get_arithmetic, reject_map_get_compare, reject_map_get_unary (NL2367)
      the default of another ?? (m.get(a) ?? m.get(b) ?? 0) and cannot be the default of another `??` reject_nullish_default_maybe (NL2371); write m.get(a) ?? (m.get(b) ?? 0)
      a const annotated V and cannot initialise a `const` annotated with its value type reject_map_get_annotated (NL2368)
      anywhere else, a dropped result included , which can only initialise a `const`, be the left operand of `??`, or be compared with `undefined` reject_map_get_statement (NL2369)

      get is called, never read as a value (Unknown field `get` on class `Map<string, i32>` (it is a method; call it), reject_map_get_property), and its key is checked against K (reject_map_get_key_type, reject_map_get_arity).

    • A maybe is two SSA values: the found bit, which is the probe's packed answer compared with 0, and the value, which valueAt reads only on the edge where the bit is set. There is no struct and no store. Each get in the map_get_* goldens is exactly one call of the instance's probe, and get never probes twice nor calls has. ?? on a get is a branch to the value or the default and a phi; on a const, whose value is loaded already, with a literal or a name as its default, it is one select (map_get_narrow). Where the ?? is wanted as an interface its class implements, the two class pointers are joined and the result converted once (map_get_interface). A const holds a phi of the value and V's zero, which no narrowed read can reach. A value built in the default goes wherever the ?? goes, as a ternary's arm does, so an object made there and stored in the map is allocated in the arena (map_get_default_escapes). Each value width is a golden: map_get_value_i32, map_get_value_f64, map_get_value_bool, map_get_value_string, an enum, i64, u64 and f32 in map_get_value_widths, and a class in map_get_pointer_value. Under -g a const bound to get is described by llvm.dbg.value of its payload, since it has no slot (dbg_map_get). The number-mode cases print under Node's own Map what they print compiled (tests/differential/unmodified.js).

  • for...of walks a table in insertion order: for (const k of m.keys()), for (const v of m.values()), and for a Set, for (const x of s), s.keys() and s.values(), which are one walk, as JavaScript's Set.prototype.keys is values (wp32-map.md §6.2, §6.3; tests/cases/map_iter_keys, map_iter_values, set_iter, set_iter_keys, set_iter_values). The variable takes K or V, and a deleted key is not visited. A walk in a generic function or method is checked per instantiation like the rest of its body (map_iter_generic).

    export const main = (): i32 => {
      const ages = new Map<string, i32>();
      ages.set("ada", 36).set("alan", 41).set("grace", 85);
      ages.delete("alan");
      let total: i32 = 0;
      for (const name of ages.keys()) {
        console.log(name);                  // ada, then grace
      }
      for (const age of ages.values()) {
        total += age;
      }
      return total === 121 ? 0 : 1;
    };
    
    • The table may change during the walk, with JavaScript's effect, because the walk re-reads the entry count every pass and no entry moves while a loop walks the table. Each row is a golden whose output Node's own Map prints too (tests/differential/unmodified.js):

      During the walk Effect Case
      set of a new key visited later in the same walk map_iter_set_new, set_iter_values
      set of a key already there the new value is seen if the walk has not reached the key yet, and the key keeps its place map_iter_set_existing
      delete of a key not yet visited it is skipped map_iter_delete_ahead
      delete, then set, of one key it is visited again, at the end map_iter_delete_reset
      clear nothing more is visited, then whatever is set after it is map_iter_clear
      enough inserts to grow the table invisible: the buckets move, the entries do not map_iter_growth
    • A walk defers compaction. Compacting is the one thing that moves entries, so each table counts the loops walking it: the count goes up where a loop is entered and down on every edge that leaves it — the fall-through, break, a return or an orReturn() out of any number of walks around it — and not on continue. A rebuild while it is above zero doubles the table instead, and the first rebuild after the walk compacts. Churn during a walk moves nothing under it (map_iter_compaction_deferred), and churn after a walk left by each kind of edge allocates nothing once warm, which is a compacting table (map_iter_break, map_iter_return, map_iter_or_return). Walks are lexical loops, so nested walks of one table and a walk in a function called from a walk are counted exactly (map_iter_nested, map_iter_callee). A break out of a switch inside a walk leaves the switch and not the walk, a return from inside it closes the walk, and a receiver reassigned mid-loop does not change the table the walk closes, which is the one it opened (map_iter_exit_edges). It costs two stores a loop and none an entry. A walk never counts as a bounded loop: a body that sets a new key every pass never ends, as in JavaScript. An insert that would pass the 2^24 - 1 entry cap while a loop walks the table panics rather than compacting.

    • The walk is four calls of the table's own code: walkOpen where the loop is entered, walkNext for the first live entry and after each pass, keyAt or valueAt for the variable, and walkClose on each edge out, all emitted internal into each module that walks, like the rest of the table (tests/link/map_walk_two_modules) (IR_COOKBOOK.md). Under -g the variable is described in its slot like any for...of variable (dbg_map_iter).

    • An iterator is not a value, so keys() and values() anywhere but as the iterable of a for...of are refused, naming the place:

      Place Message after `keys()` of `Map<string, i32>` Case
      a const, a let, or an assignment cannot be stored reject_map_iter_const, reject_map_iter_assign, reject_map_keys (NL2372)
      an argument cannot be passed as an argument reject_map_iter_argument (NL2373)
      a return, or a concise arrow body cannot be returned reject_map_iter_returned (NL2374)
      anywhere else, a dropped result or an uncalled m.keys included can only be the iterable of a `for...of` reject_map_iter_statement, reject_map_iter_uncalled, reject_map_values (NL2358)

      for (const e of m) over a Map itself would walk entries(), whose [key, value] pairs are two-type tuples: `for...of` over `Map<string, i32>` needs `entries()`, with the rewrite, for (const k of m.keys()) (reject_map_iter_entries, NL2375). It reports once: the variable is an error that its body reads without a second report (reject_map_iter_recovery). A spread is refused as it is in any array literal (reject_map_iter_spread, NL2237), for...in is Phase 0's NL1056 as it is over anything (reject_map_iter_for_in), keys() takes no argument (reject_map_iter_arity), and the variable has the type it walks (reject_map_iter_variable_type).

  • A module's own Map or Set wins in that module. A module that declares or imports a class, interface, alias or enum of either name gets no implicit import, as the declaration shadows the global under tsc (tests/link/map_own_class), and its get is its own method, answering what its signature says (tests/cases/map_get_user_class). A type parameter called Map or Set is not a use of the global either, so a module whose only Map or Set is a type parameter loads nothing and compiles as it did before Map existed (tests/cases/map_type_param_shadow). It does not shadow the global, though: as for any class name in scope, a module that also uses the global Map refuses a type parameter of that name with `Map` is generic: it must be written with its type arguments (tests/cases/reject_map_type_param_beside_global), where tsc would shadow it. A class name is program-wide, though, so a program in which one module declares Map and another names the global is refused at the declaration: `Map` is declared here and main.ts uses the global `Map`; a class or interface name is program-wide, so one program cannot have both (tests/link/map_clash, NL3030).

  • Interop: --emit-header declares an instance as an opaque struct, typedef struct nish_gen_Map_str_i32 nish_gen_Map_str_i32;, so a C host can hold one it was given and pass it back; --emit-dts and --emit-napi leave a function that takes or returns one unbridged, with the comment they write for a class.

Typed JSON: nish/json-typed

The reference for each export, with its parameters, its answer and the codes it records, is its doc comment in std/json-typed.ts; the rules are here.

jsonDecode(d, out) reads one JSON object into a class and jsonEncode(w, value) writes one back, and the code that does it is written by the compiler from the class's fields. They are generic functions in std/json-typed.ts, and every call of one is lowered in place, in the module that makes it, so nothing is looked up by name at run time and no class carries anything it would not otherwise have.

import { JsonWriter } from "nish/json";
import { JsonLimits } from "nish/json-tokenizer";
import { JsonDecoder, jsonDecode, jsonEncode } from "nish/json-typed";

class Player {
  name: string = "";
  score: i32 = 0;
}

class Team {
  players: Player[];
  captain: Player | null = null;
  constructor() {
    this.players = [];
  }
}

export const main = (): i32 => {
  const d = new JsonDecoder(body, 0, toI32(body.length), new JsonLimits());
  const t = new Team();
  const code = jsonDecode(d, t);     // 0, or the first error's code
  const w = new JsonWriter(out, 0);
  jsonEncode(w, t);                  // {"players":[...],"captain":...}
  return code;
};
  • The decoder is JsonDecoder, a JsonTokenizer over the input and a sticky error, and the encoder is the JsonWriter the call is given. The code written for a class is straight-line calls of their methods, one per field: for decoding, a loop over the object's keys that sends each key to the field it names and reads the value with the method of the field's type; for encoding, w.key(name) and the value's method per field, in declaration order. A nested class, an array's element class, a | null field's class and a Map's values each get the same code of their own, so it recurses as the data does (tests/link/std_json_typed, whose main.ll is the golden of the code written for a one-field class and tag.ll of the code for a u64, an enum and arrays of each; IR_COOKBOOK.md). Because the encoding is calls of the writer's methods, it is written in the writer's modes: a writer set with setIndent(2) encodes what JSON.stringify(v, null, 2) writes, and a setStrict(true) writer fails a NaN or infinite f64 field with JSON_WRITE_NON_FINITE (tests/link/std_json_writer).

  • A key is sent to its field by its length, then by one byte. The code written for a class of three or more fields switches on the key's length in bytes, so a key is compared only with the names of that length. Where three or more names share a length, it switches once more, on the key's byte at the one position where those names differ most, and calls keyIs for each name that has that byte there, in declaration order; a byte none of them has is an unknown key at once. One or two names, of a class or of one length, are compared in turn with keyIs, which is no more calls. A key with an escape has no byte to switch on there, and is compared with every name of its length, so "\u0069d" is the field id (tests/link/std_json_typed, "wide: every key escaped"). A key is therefore compared with the names of its length that share its byte at that position, usually one, rather than with every name in turn, and a field name that is not ASCII is dispatched by its UTF-8 bytes like any other ("names: escaped field names decode").

  • What a field may be. T is a declared class, and every field of every class the call reaches is one of these; anything else is refused at the call's argument, naming the class and the field:

    Field type JSON Decoded with Encoded with
    i32, u32, i64 an integer written without a fraction or an exponent, in the type's range d.i32(), d.u32(), d.i64() w.i32, w.u32, w.i64
    u64 an integer written in digits alone, with no sign, fraction or exponent, up to 2^64 - 1 d.u64(), through the tokenizer's numberU64 jsonEncodeU64
    a numeric enum E the number of one of E's members d.i32(), then a switch over the members' values w.i32
    a string enum E the string of one of E's members d.enumString(), then the string's bytes against the members' strings w.string of the member's string
    f64 (and number under --number-mode f64) any number; past the double range an infinity, which the writer writes as null d.f64() w.f64
    boolean true, false d.boolean() w.boolean
    string, string | null a string, and null for the second d.string(cur) w.string, jsonEncodeStringOrNull
    a declared class C, C | null an object, and null for the second C's own decoding, into the object the field holds C's own encoding
    T[], T[] | null for any of the above an array, and null for the second d.i32Array(cur) and its kin, or C's or E's decoding per element jsonEncodeI32Array and its kin, or C's or E's encoding per element
    Map<string, V>, Map<string, V> | null, for any V in this table, | null or not, a Map included an object whose keys are the map's, each value a V, and null for the second the object's keys in order, each value decoded as a V and set the map's entries in its order, each value encoded as a V
    a tagged union U of declared classes, U | null, U[] an object whose tag key names its variant, and null for the second the tag's string, then that variant's decoding, into the object the field holds when its tag matches the tag first, then the variant's own encoding
    JsonNumber from nish/json-typed, alone, | null, in an array or as a Map's value any number, kept as the text it is written in d.number(n), which copies the number's bytes into n jsonEncodeNumber, which writes them back

    `jsonEncode` cannot handle `Sample.ratio`, which is `f32`: a field read or written as JSON is an `i32`, `u32`, `i64`, `f64`, `boolean`, `string` or a declared class, an array of one of those, or a `string`, a class or an array with `| null` (NL2485, tests/cases/json_reject_field_type; two classes down through an array's element, json_reject_nested_field; in a | null field's class, json_reject_nullable_field; in a class a Map holds, json_reject_map_value_class; i32[][], json_reject_nested_array, and Level[][], json_reject_enum_nested_array; a generic class of the program's own, json_reject_generic_field). The message goes on to name the kinds added since: a `u64` and an enum are too, alone or in an array, and so is a `Map` whose keys are `string` and whose values are any of these, with `| null` or not. A JSON object's keys are strings, so a Map keyed by anything else is refused, an enum included (json_reject_map_key, json_reject_map_key_enum), and so is a map whose values typed JSON cannot write, at any depth (json_reject_map_value, json_reject_map_value_nested). A Set and an array of maps are refused as well (json_reject_set_field, json_reject_map_array). A Map keyed by a string enum is refused with the rest (json_reject_map_key_enum_string): a map's keys are read and written as strings, and a string enum is not one. An enum that mixes strings and numbers is not one the language declares at all (NL1043, json_reject_enum_mixed), an enum is never | null, numeric or string, so a Role | null field is refused where it is written (NL2010, json_reject_enum_nullable) while a Role[] | null is a field like any other, and a const enum is refused where it is declared (NL2285, json_reject_const_enum). A value that is not a declared class is `jsonEncode`'s value is `Point`, which is not a class: typed JSON reads and writes an object through the fields of a declared class, and an interface, a generic class or any other type is not one in this version (NL2486, json_reject_interface, json_reject_not_class). An interface is a record stored inline wherever it is held, and decoding into one is left for a later version. A call inside a generic function is checked for each class an instantiation binds (json_reject_through_generic).

  • Decoding is strict, and every outcome is a code. The keys are the field names, byte for byte, or the names the class's options give them. Every field is required unless its class's options make it optional, each key may appear once, and a key that names no field is an error unless d.ignoreUnknown is set, when the key's value is stepped over, checked but not kept. The first error sets d.failed and d.errorAt, the byte offset of the token it was found at, and every later call does nothing, as the tokenizer's and the writer's errors do; jsonDecode answers d.failed, and d.reset starts over:

    Outcome Code errorAt
    a value of the wrong JSON type: a string for an i32, an array for a class, null for a field that is not | null, a top-level value that is not an object, an integer that is no member's value for a numeric enum, a string that is no member's value, or a value that is not a string, for a string enum JSON_DECODE_TYPE (-20) the value
    a number its field cannot hold: a fraction or an exponent for an integer, or out of range, a negative number (-0 included) for a u64 JSON_DECODE_RANGE (-21) the number
    a key that names no field JSON_DECODE_UNKNOWN (-22) the key
    a key given twice, a field's or a Map's, however it is escaped JSON_DECODE_DUPLICATE (-23) the second
    a field the object does not give JSON_DECODE_MISSING (-24) the object's }
    more keys, over every Map field of the decoding, than d.maxMapKeys (4,096 unless set) JSON_DECODE_MAP_LIMIT (-25) the first key past it
    a union's tag missing, not a string, or a string no variant writes JSON_DECODE_TAG (-26) the tag's value, or the object's { when it has none
    text that is not JSON, or that crosses a JsonLimits limit the tokenizer's JSON_TOK_ERR_* (-1 to -13) the tokenizer's

    Nothing an input holds panics: the recursion is the data's, so the tokenizer's depth limit bounds it. That bound is the caller's JsonLimits.maxDepth, 128 by default: each level is one native frame, so a limit raised into the hundreds of thousands can run off the stack, and one kept to a few thousand cannot. The writer refuses a 65th open container (JSON_WRITER_MAX_DEPTH), so a value decoded from deeper than 64 levels encodes as JSON_WRITE_TOO_DEEP (tests/link/std_json_typed, which reaches each row at each depth, the limits included). A duplicate of a key ignoreUnknown steps over is not detected, because its value is never read. After an error every field of out holds a value of its type, some decoded and some as they were: an enum field or element never holds a number that is no member's, because such a number is not stored.

  • A numeric enum is its number. It is read with d.i32(), so a number that is not an i32 integer is that field's JSON_DECODE_RANGE, and an integer that is not the value of one of its members is JSON_DECODE_TYPE; a member's name, "High", is a string and a type error too. Two members of one value are one number (tests/link/std_json_typed, "enum: two members of one value both read 1"). It is written as w.i32 of its value, as JSON.stringify writes a TypeScript numeric enum.

  • A string enum is its member's string. It is read with d.enumString(), which takes the value as a string without copying it, and its bytes are matched against the members' strings the way a key is sent to its field: by length, then by one byte, then with keyIs, so the read allocates nothing, and an escaped string, "\u0061dmin", is the member its unescaped bytes spell. A string that is no member's, a prefix of one included, and a value that is not a string are JSON_DECODE_TYPE, and the field or element keeps the member it held. It is written as w.string of the member's string, which is what JSON.stringify writes for a TypeScript string enum (tests/link/json_enum_string; IR_COOKBOOK.md). A field, an array, a | null array and a Map<string, Role>'s values are read and written so.

  • A u64 is exact to 2^64 - 1. It is read through the tokenizer's numberU64, so 18446744073709551615 decodes and 18446744073709551616, -1, -0 and 1.5 are JSON_DECODE_RANGE. It is written as its digits, which JSON.stringify cannot do for a number past 2^53: tests/run.js holds round-trip-kinds.txt to JSON.stringify(JSON.parse(line)) with each integer past 2^53 kept as its own digits (JSON.rawJSON).

  • A class's options are one static readonly json literal. A class gives a field another key on the wire, and lets a key be left out, in an object literal written in the class, which the compiler reads when it writes the class's code:

    class Account {
      static readonly json = { rename: { userId: "user_id" }, optional: ["tags"] };
      userId: i32 = 0;
      tags: string[] | null = null;
    }
    

    rename maps a field to its key, a string literal of any bytes, so {"user_id":7} is decoded into userId and userId is encoded as "user_id"; the field's own name is then a key no field has. The key is dispatched by its length and bytes like any other, escapes and non-ASCII included (tests/link/std_json_typed/options.ts, "rename: escaped wire names decode"). optional lists fields whose key may be missing: a missing one sets the field to its initializer, on a fresh object and on one decoded into again, so a reused object does not keep what an earlier input gave it ("optional: sets the missing fields back to their initializers"), and allocates nothing to do it. Every other missing key is still JSON_DECODE_MISSING, and an optional key that is given is read like any other: once, and of its field's type. The member is not a field: it has no slot, the class's layout is what it would be without it, the program cannot read it, and a class without it is decoded exactly as before. It is the one static member a class may declare, and its options are checked where the class is declared, whether or not any call decodes the class, each misuse with its own code:

    Misuse Code Case
    static json without readonly but with a type, static readonly json with a type annotation, or on a generic class NL2492 json_reject_options_mutable_typed, json_reject_options_annotated, json_reject_options_generic, and on a generic class nothing instantiates, json_reject_options_generic_unused
    options, rename, a key or an optional entry not written as the literal it must be: a module constant, an array for the object, a string for optional's array NL2493 json_reject_options_alias, json_reject_options_alias_whole, json_reject_options_not_object, json_reject_options_optional_not_array
    an option other than rename, optional and presence NL2494 json_reject_options_unknown
    a name that is not one of the class's fields NL2495 json_reject_options_misspelt_rename, json_reject_options_misspelt_optional
    a rename to a value that is not a string NL2496 json_reject_options_rename_number
    two fields with one key, a rename to another field's own name included, or a field renamed or made optional twice NL2497 json_reject_options_wire_collides, json_reject_options_wire_twice, json_reject_options_optional_twice
    an optional field with no initializer NL2498 json_reject_options_no_initializer
    a rename to a key that is not valid UTF-8, a lone surrogate such as "\ud800" included, which no input could give NL2499 json_reject_options_surrogate
    presence or one of its entries not written as an object literal of string literals NL2493 json_reject_presence_not_object, json_reject_presence_not_literal
    presence naming a field, or a presence field, the class does not declare NL2495 json_reject_presence_misspelt_field, json_reject_presence_misspelt_flag
    a presence field that is not a boolean, is readonly, is the field it records, records two fields, is recorded by a presence field of its own or is renamed or made optional; a field recorded twice NL2592 json_reject_presence_not_boolean, json_reject_presence_readonly, json_reject_presence_itself, json_reject_presence_shared, json_reject_presence_of_presence, json_reject_presence_recorded, json_reject_presence_renamed, json_reject_presence_optional, reject_json_presence_twice

    A spread in the literal is `User.json` spreads another value into the typed JSON options: …, because the options are read from the class's text, where a spread names nothing (NL2711, reject_json_options_spread), and a computed key is refused by Phase 0 as it is anywhere (NL1041, json_reject_options_computed), a subclass by extends (json_reject_options_subclass), and a second static readonly json is a duplicate member (reject_json_options_twice). The options are not a value: User.json names a class where a value is due, and is refused as any class name there is (json_reject_options_read). Only static readonly json may leave out its type: written static json = { ... }, or as an instance field, it is an ordinary field to the parser and needs one (a type annotation is required, json_reject_options_mutable, json_reject_options_instance), and an instance field called json with a type is an ordinary field, read and written under its own name (tests/link/std_json_typed/options.ts, "instance json: an ordinary field decodes"). The options are not a member, so a field or a method called json may come before or after them, as in TypeScript, whose statics have a namespace of their own ("order: a json field before the options decodes", "order: a json method before the options decodes"). Under Node the literal is a static property of the class that nothing reads.

  • presence records which keys were given. A third option names, for a field, a boolean field of the same class that every decoding sets to whether the input gave that field's key; a key given null is given:

    class UserPatch {
      static readonly json = { optional: ["email"], presence: { email: "hasEmail" } };
      email: string | null = null;
      hasEmail: boolean = false;
    }
    

    So {} leaves hasEmail false, and {"email":null} and {"email":"a"} set it true, which optional alone cannot tell apart from a key given the initializer. Any field may be recorded, of any kind the decoder takes and optional or not (tests/link/json_presence). The presence field is off the wire: no key decodes into it, jsonEncode leaves it out, it takes no part in the key-uniqueness rule, and no option renames it, makes it optional or records it. It is set when the object closes, from the per-field flag the decoder already keeps to report a duplicate or a missing key: one store i1 per recorded field, no allocation, and a class that records nothing compiles to the code it did before. Each object decoded sets its own, so a nested class, an array element and a map's value answer for their own keys, and a second decoding into one object answers for its own input. A decoding that fails may have set some presence fields and not others (none, when the input is not an object), so they are read only after JSON_DECODE_OK. A JSON Merge Patch (RFC 7396) is applied with it: the patch is a class whose fields are all optional and recorded, and null on a field that is not | null is JSON_DECODE_TYPE, so the patch is refused before any of it is applied (examples/merge-patch.ts, tests/link/json_merge_patch).

  • A JsonNumber is a number's text. JsonNumber, from nish/json-typed, holds a number as the bytes it was written in, so a field of it decodes and encodes any number byte for byte: an integer wider than a u64, a fraction with more digits than an f64 holds, 1E+400 and -0 alike (tests/run.js holds each line of round-trip-options.txt to JSON.stringify(JSON.parse(line)) with every number kept as its source). It is read as a number whole, not as an object, so anything but a number is JSON_DECODE_TYPE, and its length is bounded by JsonLimits.maxNumberBytes like any number's. Decoding into one reuses its bytes, so Arena.used() stays flat over decodes into one object. It reads its text as nish/json-value reads a number: text(), isI64() and i64(), isU64() and u64(), and f64(), the nearest double; setText(t) sets it when t is one JSON number and answers whether it was. A new one holds 0. Its methods run under Node, and tests/run.js holds them to the native lines (readers.ts).

  • A Map<string, V> is an object. Decoding empties the map the field holds once the object opens, then reads each key as a new string and its value as a V, made fresh for the key (a zero, "", the enum's first member, null, or a new object, array or map, which a Map's value class is made with new C() for, so its constructor takes no parameters, NL2488, json_reject_map_value_constructor), and sets it, so the map's entries are the input's, in the input's order. A key the object gave already is JSON_DECODE_DUPLICATE before its value is read, whether it is spelled the same way or through an escape (docs/security/json.md, JSON-3). Encoding writes the entries in the map's own order, which is insertion order. That differs from a JavaScript object in one case: a key that reads as an array index ("1") keeps its place, where JSON.parse would move it first ("map: a key like an index keeps the input's order"). Unlike the rest of a decode, a map's entries are made anew each time, so decoding into a map allocates its keys and values. A value whose read fails is not set, so a map never holds a value the input did not give ("map: and the map holds only the entry read before it"). The maps of one decoding take at most d.maxMapKeys keys between them, 4,096 unless set, the keys of a map inside another map included, and the key past it is JSON_DECODE_MAP_LIMIT before it is read ("map limit: one key past it: code", "map limit: 820 small maps, 4,100 keys, are refused"); d.reset starts the count again. At 0 or below no map may take a key, and {} is the one map value accepted. The global Map hashes a string under a per-process random key, so a body cannot choose keys that collide, and this limit bounds what all the maps of one decoding hold together (docs/security/json.md, JSON-13).

  • A tagged union is an object with its tag. A field, an array's element or the value of jsonEncode may be a discriminated union whose variants are all declared classes, and the union is written as the variant's object with its tag key first, the tag's value the name of its enum member, or for a string enum the member's string:

    enum Kind { Num, Neg }
    class Num {
      readonly kind: Kind.Num = Kind.Num;
      value: f64 = 0;
    }
    class Neg {
      static readonly json = { tag: "negate" };
      readonly kind: Kind.Neg = Kind.Neg;
      arg: Expr;
      constructor() { this.arg = zero(); }
    }
    type Expr = Num | Neg;
    const zero = (): Expr => new Num();
    
    const e = jsonDecodeUnion(d, zero());  // {"kind":"negate","arg":{"kind":"Num","value":2}}
    jsonEncode(w, e);                      // the same text back
    

    With enum Kind { Num = "num", Neg = "neg" } the same union writes "kind":"num", and Neg's tag still writes "negate" (tests/link/json_enum_string). A variant's tag option replaces its member's name or string on the wire, and rename renames the tag key as it renames any field. Every variant writes its tag under one key and as a different string, or the union is refused where a call reaches it (NL2721, json_reject_union_tag_key; NL2720, json_reject_union_same_tag); a tag that is not a string literal is NL2722 (json_reject_union_tag_literal), one on a class with no field whose type is one enum member is NL2724 (json_reject_union_tag_no_field), and tag: "" is the same as no tag: the variant writes its member's name, or string. A tag on a class that has such a field but that no union holds is not refused and does nothing: the class compiles while no JSON call reaches it, and one that does refuses the field (below). A variant that is not a class, an interface, is NL2486 (json_reject_union_interface), the code of any typed JSON type that is not a declared class. A field typed with an enum member that no union tags a variant with is no tag, and is refused as any other field type is (json_reject_enum_member). jsonDecode decodes into the object it is given and a union's variant is known only from the input, so a union is decoded with jsonDecodeUnion(d, prev), which answers the object it decoded into: prev itself when the input's tag is prev's, reused as any nested object is, and otherwise a new variant made with new V(). On a failure it answers prev, with the code in d.failed, at the top only: a union field, element or Map value already holds the new variant when the failure comes, decoded up to it, as any failed decode leaves what it read. Its type argument is inferred from prev and never written; jsonDecode with a union is `jsonDecode`'s value is `A | B`, a tagged union: … (NL2719, json_reject_union_decode), and jsonDecodeUnion with a class is NL2723 (json_reject_union_not_union). A union field decodes into the object it holds the same way, and an array's element past its end is made fresh. The tag may come anywhere in the object: when it is the first key the variant is known at once, and otherwise the decoder reads ahead to it, stepping over the values before it, and rewinds, so each byte is read at most maxDepth + 1 times (docs/security/json.md, JSON-17). A tag that is missing, not a string or no variant's is JSON_DECODE_TAG. A variant used alone, as a class, decodes and encodes its tag too, and a tag that is not its own is JSON_DECODE_TAG. Each union gets one decoding and one encoding function, .json.decode.union.<U> and .json.encode.union.<U>, which every place it appears calls, so a recursive union such as Expr above recurses through them (tests/link/json_tagged_union). A Map<string, Expr> field decodes each value through that same function and encodes it through the other.

  • Decoding reuses what out holds. A nested object is decoded into the object its field holds, an array's elements into the elements it has, with new ones made past its end and the rest dropped, and a string field keeps its string when the input's is equal. So decoding one shape into one object again and again allocates nothing: Arena.used() is flat over 1,000 decodes of a three-level object (tests/link/std_json_typed). A string that changes is a new string, since strings are immutable, and only that: it is made by one String.fromBytes, straight from the input when the string has no escape and from its unescaped bytes when it has one, and those bytes are reclaimed at the call. A class the decoder has to make itself — an array's element past the end, or the object of a | null field that holds null — is made with new C(), so its constructor takes no parameters: `jsonDecode` makes a `Pet` with `new Pet()`, and its constructor takes 1 parameter(s): an array's element and a `| null` field's object are made by the decoder with no arguments, so their class's constructor takes none (NL2488, json_reject_constructor, json_reject_constructor_nullable). Decoding assigns every field, so a readonly one is refused: `Key.id` is `readonly`, and `jsonDecode` assigns every field of the class it decodes into (NL2487, json_reject_readonly). A decode stores what it makes into the object it fills, so a call inside a using a = arena() block is the NL2424 of that block (json_reject_arena_block). An encoding only reads the object and keeps nothing, so it may run inside one (tests/link/std_json_typed). Two fields holding one object are decoded into it twice, the second winning.

  • Encoding is JSON.stringify's text for the same object, its keys in declaration order, which is also the order an object's keys come back in when it is decoded from text whose keys are in that order: tests/run.js holds each line of tests/link/std_json_typed/round-trip.txt, decoded and encoded natively, to JSON.stringify(JSON.parse(line)). A full buffer is the writer's JSON_WRITE_FULL, and an object that holds itself ends at the writer's depth limit with JSON_WRITE_TOO_DEEP, because the encoding stops at the first failure rather than recursing on.

  • The TypeScript reading is class C (wp33-round-trip.md §2). Under Node the type arguments are erased, so there is no layout to write the code from, and the bodies in std/json-typed.ts that Node runs end the program with panic("nish/json-typed: jsonDecode and jsonEncode are written by the compiler from the class's fields, and under Node the types are erased") rather than guess a field's type from the value it holds (tests/run.js, "under Node, jsonDecode panics"). The translation is the code the compiler writes, written as TypeScript for each class: a function per class of the same calls into JsonDecoder and JsonWriter, which everything else in the module already runs under Node: tests/run.js runs the decoder's readers there and natively and holds both to the same lines (tests/link/std_json_typed/readers.ts). A program that imports std/json-typed.ts by path rather than as nish/json-typed is compiled as written and gets those bodies natively too.

Capabilities

Each builtin's label is the Capability: line of its doc comment in runtime/nish.d.ts; the rules are here.

Every builtin carries exactly one capability label, or a deliberate none, and the compiler works out which capabilities every function, module and package can reach — directly, through any chain of calls, across packages — with one witness call chain for each (wp35-capabilities.md). A program is refused for what it reaches only under a capability policy; without one, the set is a report.

Capability Builtins
clock Date.now, monotonicNanos, wallClockNanos, statMtimeSync (tests/cases/caps_clock)
entropy crypto.getRandomValues, Math.random (caps_entropy)
env getenv, geteuid (caps_env)
exit process.exit, and exit from nish:process (caps_exit)
ffi any call to a declare function (caps_ffi)
fs.read readFileSync, readFileSyncOrNull, readFileBytesSync, readdirSync, realpathSync, isDirectorySync, lstatOwnerModeSync, isExecutableSync, fileOpen, fileReadAt, fileSize, fileClose (caps_fs_read, caps_file_read)
fs.write writeFileSync, appendFileSync, mkdirSync (caps_fs_write), writeFileBytesSync, renameSync, unlinkSync (caps_fs_replace), writeFileSyncMode, createFileExclusiveSync, mkdirSyncMode (caps_fs_mode)
net every nish:net export (caps_net)
process.spawn spawnSync, spawnSyncTo (caps_process_spawn), spawnPiped (caps_process_spawn_piped), spawnPipedWithin (caps_process_spawn_piped_within), spawn, kill, waitPid (caps_process_wait), setNice, setIdlePriority, setCpuAffinity (caps_process_nice)
signal signalFd, readSignal (caps_signal)
tty every nish:term export (caps_tty)
unsafe the eight nish:unsafe exports: uncheckedGet, uncheckedSet, wrappingAdd, wrappingSub, wrappingMul, readCString, readStruct, cErrno (tests/link/caps_policy_deny_unsafe; "capabilities: each of the eight nish:unsafe exports carries unsafe at its call")

Everything else is none: printing (console.*, write, writeError, writeBytes), process.argv, process.platform and process.arch, availableParallelism and totalMemory (caps_host), panic, Arena.* and arena(), Math.* other than random, the conversions, the constant-time builtins, secureZero, the string, array, Map and Set surface, and nish/threads (caps_pure). A nish: import carries the label of the builtin it renames.

  • A function is judged per instantiation: pick<i32, double> and pick<string, firstLine>, or apply<lengthOf> and apply<(name) => ...>, are separate entries with separate sets (caps_generic, caps_fnarg). A module's set is the union over its functions, a package's over its modules, and the program's is main's closure (tests/link/caps_package). A parallel body cannot reach a capability at all: every builtin that carries one writes memory the runtime shares (tests/link/caps_parallel, reject_caps_parallel_clock).
  • A program is deterministic when it reaches none of clock, entropy, env, ffi, fs.read, fs.write, net, process.spawn and tty: the same argv and stdin then give the same output. tty is among them although stdin is an input a deterministic program may read, because nish:term reads it against the clock (waitStdin's timeout, readStdin's "nothing yet") and asks the terminal itself. exit and signal keep a program deterministic (caps_exit, caps_signal).
  • --emit-capabilities <file.json> writes the report: version, entry, deterministic, the program's set, then each package, module and exported function (and every instantiation of one) with its set and a witness chain per capability, { "function", "at": "path:line:col", "calls" } per call, and then its "panics": every panic site the function has, { "kind", "at": "path:line:col" } followed by the keys --emit-panics writes after its position ("proven", "callee", "via", "allowed"). Keys are in a fixed order, sets in the order of the table above, packages, modules and functions sorted, and every path relative to the entry's directory, so the file is byte-stable and diffable (.caps.json beside each caps_* case). nish run --capabilities — or --capabilities on a compile — prints one line on stderr, capabilities: fs.read (not deterministic) or capabilities: none (deterministic), before the program runs, and leaves its stdout alone. --emit-capabilities without a file is a usage error, exit 2, and so is either flag with --emit-ast, which stops before the checker; with --emit-checked both are answered beside the dump, and --fix, which answers before the program is analysed, takes neither. nish run refuses --emit-capabilities as it refuses every product it keeps to itself.

The capability policy

--allow <cap>[,<cap>...] and --deny <cap>[,<cap>...], each repeatable, and "nish": { "capabilities": { "allow": [...], "deny": [...] } } in the root package.json (the one nearest above the entry, where noPanic is read) are a capability policy, and a program whose entry reaches a capability the policy does not grant is refused (wp36-capability-policy.md). The same object's dependencies key grants a dependency its gated capabilities, as a dependency's capabilities says (NL3041, NL3042):

main.ts:6:36: error: `main` reaches `net`, which the capability policy does not grant (the root package's `deny`); the chain that reaches it: main.ts:6:36 calls dial, client.ts:5:10 calls netAddress
  • A denied capability is refused when it is reached. Giving --allow or an allow list makes the policy an allowlist: a reached capability it does not list is refused, exit and signal included. The command line can only narrow the manifest: denials union and allowlists intersect.
  • What is judged is every function outside code can call: main alone in a closed --link build or under nish run, and in an open one (-o alone, a sidecar, a declare function, wasm) main and every exported function of every module, or every function at all under --no-strict-exports (tests/wordings/nl2459_policy_library, nl2459_policy_unexported). Each refused capability is one error (NL2459), reported in source order, spanned at the first call of the witness chain and naming every hop of it as --emit-capabilities writes them (tests/wordings/nl2459_policy_deny, nl2459_policy_allowlist; tests/link/caps_policy_manifest, caps_policy_two_denied). The command line can only narrow (caps_policy_cli_cannot_widen, caps_policy_allow_intersect, caps_policy_allow_narrowed, caps_policy_manifest_allow_cli_deny).
  • unsafe can be denied — --deny unsafe refuses a program that reaches a nish:unsafe export anywhere in its closure (tests/link/caps_policy_deny_unsafe) — and never allowed: the import is the opt-in, so an allowlist does not refuse it and --allow unsafe is a usage error.
  • unsafe travels like every other capability — to each caller, however far up, and to the module and package above it, including a dependency that imports nish:unsafe when the entry package does not — and keeps a program deterministic ("capabilities: unsafe reaches a caller two frames up, with the whole chain as its witness", "capabilities: a dependency that imports nish:unsafe gives unsafe to its package and to the entry that calls it"). No flag turns a check off for a whole program, so nish:unsafe is the only way to one and --deny unsafe sees every unchecked site.
  • A name that is no capability, --allow unsafe, a capability both allowed and denied, and a directory scope (fs.read=<dir>) without --sandbox, which alone enforces one (Directory scopes under The sandbox), are usage errors, exit 2. In package.json they are errors where the manifest writes them: the wrong shape of capabilities, allow or deny written twice included (NL3032, tests/link/caps_policy_manifest_shape, caps_policy_manifest_repeated), a name that is no capability (NL3033, caps_policy_manifest_unknown), unsafe under allow (NL3034, caps_policy_manifest_unsafe) and a capability under both (NL3035, caps_policy_manifest_both).
  • The "nish" field is read strictly, as the no-panic scope says: a key is decoded as JSON.parse decodes it, so a policy written with an escape is honoured (caps_policy_manifest_escaped_policy, caps_policy_manifest_escaped_entry), and anything the reader could otherwise not vouch for — a second "nish" or capabilities, a key other than noPanic, capabilities, host, compat and netRings — is NL3036 (caps_policy_manifest_repeated_nish, caps_policy_manifest_repeated_policy, caps_policy_manifest_nish_unknown_key).
  • A policy changes no byte of the IR: a program inside it compiles and runs as it does with none (tests/link/caps_policy_granted, caps_policy_manifest_granted). nish run takes both flags and refuses before it links; --fix and --emit-ast refuse them.

A dependency's capabilities

A dependency — a package loaded from node_modules — may reach ffi, fs.read, fs.write, net, process.spawn and unsafe, the gated capabilities, only when the root package.json grants them to it by name; it reaches every other capability without one (wp40-dependency-capabilities.md):

{ "nish": { "capabilities": { "dependencies": { "pkg_http": ["net"] } } } }

A dependency that reaches one it is not granted is refused (NL2576), in the dependency, naming the package, the capability and the function:

node_modules/pkg_http/src/index.ts:6:10: error: the dependency `pkg_http` reaches `net` through `dial`, which the root package does not grant it; grant it with "nish": { "capabilities": { "dependencies": { "pkg_http": ["net"] } } } — the chain that reaches it: node_modules/pkg_http/src/index.ts:6:10 calls netAddress
  • A dependency's capabilities are the union over every function its modules define, exported or not, called or not (tests/link/caps_dep_unexported), and what it reaches through another dependency is its own: granting the package that dials refuses the one that calls it (caps_dep_transitive). One error per package and capability, spanned at the first call of the witness chain from the package's first function, in source order, that reaches it (caps_dep_order; one refused and one granted program per capability, caps_dep_net_refused … caps_dep_unsafe_granted).
  • The standard library is never refused, and what decides is where a module was resolved, not its package's name: a dependency that calls itself nish is judged like any other (caps_dep_std_not_refused, caps_dep_named_nish). The ungated capabilities need no grant (caps_dep_ungated).
  • A function argument counts where it is instantiated: the root's function passed to a dependency's template makes the instantiation, which is the dependency's, reach what that function reaches, so the dependency is refused for it (caps_dep_fnarg_callback).
  • Only the root manifest grants: no flag does (--allow net leaves the dependency refused, caps_dep_cli_cannot_grant), nor the dependency's own manifest (caps_dep_self_grant), and a grant never widens the root's own policy — a capability the root denies is refused by NL2459 whoever is granted it (caps_dep_root_deny_wins).
  • dependencies is read as strictly as the rest of the field: a value that is not an object, a package named twice, a grant that is not an array and an entry that is not a string are NL3041 (caps_dep_manifest_not_object, _repeated, _not_array, _non_string), a name that is no capability is NL3033 (caps_dep_manifest_unknown) and one that is not gated NL3042 (caps_dep_manifest_ungated). unsafe may be granted, keys are decoded as JSON, and a grant to a package the program does not load is no error (caps_dep_manifest_unsafe, _escaped_name, _unloaded).
  • fs.read and fs.write are gated: a dependency that reads or writes a file needs its grant like one that dials (caps_dep_fs_read_refused, caps_dep_fs_write_refused).

A grant entry may carry an argument, capability=argument, which narrows it: a directory for fs.read and fs.write, a program for process.spawn, a numeric host, or a Unix socket's absolute path, for net. An entry without one grants the capability whole, as before:

{ "nish": { "capabilities": { "dependencies": {
  "pkg_tool": ["fs.read=./data", "process.spawn=git", "net=127.0.0.1"]
} } } }
  • A directory is resolved from the root package's directory and must exist when the program is built (NL3047); a program is found on PATH, or as a path from that directory (NL3046); a host is an IPv4 or IPv6 address, the form nish:net takes (NL3045). ffi, unsafe and the ungated capabilities take no argument (NL3048; all four in caps_dep_manifest_arg_refused). A capability may be named more than once, one argument each.

  • What the compiler can see it checks: a string literal a dependency hands readFileSync, writeFileSync, spawnSync or netAddress outside its grant is refused (NL2594), at the literal:

    node_modules/pkg_conf/src/index.ts:2:38: error: the dependency `pkg_conf` hands `readFileSync` "/etc/passwd", which its `fs.read` grant does not cover: it is granted fs.read=data; name it in the grant, or grant `fs.read` whole
    

    (caps_dep_arg_fs_too_narrow, _spawn_too_narrow, _net_too_narrow; the grant that covers each, caps_dep_arg_granted).

  • What it cannot see — a computed path, program or host — the build leaves to --sandbox, which scopes Landlock to the granted directories and programs (The sandbox, tests/link/sandbox_dep_*). Hosts have no run-time check: seccomp cannot read the address a socket is given, so a net=<host> grant binds only the literals the build sees.

  • The lockfile records the arguments ("pkg_conf": ["fs.read=data"]). A grant past them — a new directory, program or host for a capability the row records narrowed — is NL2593 until the lock is updated; a row that records the capability whole accepts any argument (caps_lock_args).

  • @nish_sandbox takes a fifth operand for the programs a process.spawn=<program> grant allows, null when none is named, and a sixth, the number of restricted rings to make before the filter (WP63 S3; 0 unless main reaches net with NET_LISTEN_RING and not process.spawn): call void @nish_sandbox(i32 32, i32 1, i8* null, i8* null, i8* null, i32 0).

  • process.spawn=exec:<program> grants the program as process.spawn=<program> does, and lets it start others: under --sandbox a granted program is held to its own start (no program it starts may exec again), and an exec: one is not, so its subtree may run anything the allowlist's EXECUTE reaches, its loaders included. The prefix is read before the program is resolved: process.spawn=exec: with no program is NL3046 (caps_dep_manifest_spawn_exec_refused); the literal check treats exec:<program> as <program> (caps_dep_arg_spawn_exec_too_narrow, caps_dep_spawn_exec_granted); a program granted both ways is exec:; and the lockfile records the prefix, so adding it is NL2593 (caps_dep_lock_spawn_exec_widened). The program is listed with three leading slashes, the same file to the kernel. For tests/link/sandbox_dep_exec_optin, granted exec:env and true, on an x86_64 glibc machine (illustrative: the paths, the loader and the length are the machine's; tests/run.js pins only the /// entry for env):

    @.sandbox.exec = private unnamed_addr constant [78 x i8] c"///usr/bin/env\00/usr/bin/true\00//usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2\00\00"
    

The capability lockfile

--update-capability-lock writes nish-capabilities.lock.json beside the nearest package.json above the entry (beside the entry when there is none), recording every capability each dependency reaches, gated or not. While the file exists, every build is held to it:

{
  "version": 1,
  "packages": {
    "pkg_http": ["net"]
  }
}

A dependency that reaches a capability its row does not record is refused (NL2577), in the dependency, with the same witness chain as NL2576:

node_modules/pkg_http/src/index.ts:6:10: error: the dependency `pkg_http` reaches `net` through `dial`, which `nish-capabilities.lock.json` does not record for it; review the chain, then record it with `--update-capability-lock` — the chain that reaches it: node_modules/pkg_http/src/index.ts:6:10 calls netAddress
  • Only growth is an error. A row that records more than the dependency reaches, or a row for a package the program no longer loads, is accepted (tests/link/caps_lock_shrink, caps_lock_stale); a dependency with no row is held to an empty one (caps_lock_no_row, caps_lock_no_row_pure). One error per package and capability, in NL2576's order (caps_lock_growth).
  • The comparison runs after both policies: a build the root's policy or the dependency grants refuse is refused by them, and --update-capability-lock on it writes nothing (caps_lock_refused). The flag skips the comparison and rewrites the file, even a malformed one.
  • The file is byte-stable: packages sorted, capabilities in the fixed order of the report, two-space indentation, one trailing newline, [] for a package that reaches nothing. Neither the standard library nor the root package has a row.
  • It is read strictly: text that is not JSON, a version other than 1, a key other than version and packages, a name that is no capability, and a key or capability written twice are NL3043, spanned in the lockfile (caps_lock_bad_json, _version, _key, _capability, _repeated).
  • The lockfile changes no byte of the IR. nish run takes the flag, and checks the program against the file before it looks in its cache, so the file is not part of the cache key and a cache hit is held to it too. --fix and --emit-ast refuse the flag.

--deterministic

--deterministic, on a compile or nish run, promises that the program's f64 and f32 results are the same bits natively on x86-64 and AArch64, on wasm32 and in the --emit-ts reading under Node (wp50-wasm-host.md §9):

  • Math.sin, cos, exp, log and pow lower to std/math.ts's sinF64, cosF64, expF64, logF64 and powF64 on every target, as on a page module, and a float % to fmodF64 (an f32 widened and the exact result narrowed back). The --emit-ts reading imports those functions, and tanF64, atan2F64 and hypotF64, from ./nish/math.ts in place of Math. + - * /, sqrt and the exact builtins need nothing: they are correctly rounded everywhere, and every C unit and the program's IR are compiled with -ffp-contract=off with or without the flag.
  • f64ToBits and f32ToBits give the canonical quiet NaN, 0x7ff8000000000000 and 0x7fc00000, for every NaN (quietBitsF64 and quietBitsF32); targets differ in the sign and payload of a NaN an operation makes.
  • A program that reaches a nondeterministic capability — clock, entropy, env, ffi, fs.read, fs.write, net, process.spawn or tty — is refused with the chain that reaches it, as the capability policy refuses one: NL3052 the file system, NL3053 another machine or program (net, process.spawn), NL3054 C code or a host import (ffi), NL2700 the clock and the random source, NL2701 the environment and the terminal.
  • Number(s), parseFloat(s) and parseInt(s) read a string with one grammar on every target, the one --emit-ts reads: a 0x integer, read by std/math.ts's numberExact and parseFloatExact (which the flag calls in their place) and correctly rounded on any libc, and no hex float, so Number("0x1p3") is NaN and parseFloat("0x1p3") is 1 where a native build without the flag lets strtod read 8 (deterministic_parse). Every decimal spelling already agreed: glibc, musl and Node round them all correctly.
main.ts:2:15: error: `main` reaches `clock`, which `--deterministic` refuses: the clock and the random source answer differently on every run; the chain that reaches it: main.ts:2:15 calls Date.now

The float corpus, tests/differential/corpus/float/, holds every operation and Math builtin over ±0, subnormals, the normal extremes, ±Infinity, NaN, integers near 2^53, multiples of π/4, NaN payloads read through f64ToBits and f32ToBits, and 94 strings through Number, parseFloat and parseInt to one expected.txt on each target (tests/run.js, sections deterministic and float-corpus; the reject_deterministic_* cases).

The sandbox

--sandbox holds a running native program to the capabilities its main reaches. The policy above refuses at compile time a program that reaches more than it grants; the sandbox makes the kernel refuse, at run time, what the program's code does not reach, ffi callees and constructors included (wp37-os-sandbox.md, security/sandbox.md):

@.sandbox.preinit = internal constant void ()* @.sandbox.apply, section ".preinit_array", align 8
@llvm.used = appending global [1 x i8*] [i8* bitcast (void ()** @.sandbox.preinit to i8*)], section "llvm.metadata"

define internal void @.sandbox.apply() {
entry:
  call void @nish_sandbox(i32 0, i32 1, i8* null, i8* null, i8* null, i32 0)
  ret void
}

define noundef i32 @main(i32 noundef %argc, i8** noundef %argv) #1 {
entry:
  call void @nish_sandbox_check()
  • The flag. --sandbox is strict; --sandbox=best-effort runs on a kernel that lacks part of the confinement and says in one line on stderr what it could not withhold. Any other mode is a usage error, exit 2. It is accepted by --link, by nish run (before the file) and on the IR-only route, alone or beside the flags that write a file next to the IR; the .ll then carries the call, and linking it by hand keeps the confinement (tests/cases/sandbox_hello). --emit-ast, --emit-checked, --emit-ts, --fix, --emit-napi and --emit-napi-async refuse it, exit 2, since none writes a program with a main to confine, and so does an entry module that declares no main and holds no top-level code to run as one (--sandbox on a script confines the main its code becomes). --sandbox and --sandbox=best-effort on one command line are refused, exit 2; the same one twice is one.
  • The call. The first operand is main's capability mask, numbered as --emit-capabilities numbers the bits (clock 0 … tty 10, unsafe 11), the second 1 for strict and 2 for best-effort; the two i8* operands are the fs.read and fs.write directory scopes below, null when there are none. nish_sandbox runs from .preinit_array, after the loader and before any constructor of the executable, an ffi object or a shared library, and applies, in order: PR_SET_NO_NEW_PRIVS, a drop of every Linux capability (each set read back), a Landlock ruleset masked to the ABI the kernel answers, and a seccomp filter. main calls nish_sandbox_check first, which refuses to run a program whose C library never ran .preinit_array (musl), with exit 126 and one line: nish: --sandbox: the C library did not run .preinit_array, so the program was not confined.
  • What each capability's absence withholds. Without fs.read, Landlock withholds reading files and directories; without fs.write, writing, making, removing, renaming and linking (within one directory too) and truncating, and seccomp answers EPERM to chmod, chown, utimensat, setxattr and their siblings, which Landlock does not decide. Without process.spawn, execve, execveat, fork, vfork and a clone without CLONE_THREAD kill the process, clone3 answers ENOSYS, and Landlock withholds executing files and device ioctls (ABI 5); with it, executing and reading files and device ioctls are granted beneath /, since a child must load its binary and libraries, so a program that reaches process.spawn can read any file the user can whether or not it reaches fs.read. Without net, socket, socketpair, connect, bind, listen, accept and accept4 kill it. With net but without fs.write, a pathname unixListen or unixListenMode is -13: Landlock counts making the socket file as a write (MAKE_SOCK), while an abstract name binds. The sandbox uses no Landlock right that covers connecting to a pathname socket, so net alone still lets unixConnect reach any pathname listener whose file and directories the user may reach; only abstract names outside the sandbox are scoped, from ABI 6. ffi, unsafe, tty, clock, entropy, env, exit and signal withhold nothing of their own; the ABI 6 scopes under Always apply whatever main reaches.
  • Directory scopes. Under --sandbox, --allow fs.read=<dir> and --allow fs.write=<dir> grant the capability beneath <dir> alone. The directory is resolved, symlinks and all, when the program is built, and must exist then, or it is a usage error, exit 2: compile: `fs.read=<dir>` names no directory; a scope is a directory that exists when the program is built. Any other capability scoped, or a scope in --deny, is refused, exit 2, and so is a scope without --sandbox, as before. A scope narrows the policy like any --allow list, so the same list names every other capability the program reaches. A capability also granted whole is granted whole. The scopes are emitted as one constant per capability, each directory ended by a NUL and the list by a second: @.sandbox.read = private unnamed_addr constant [N x i8] c"/srv/data\00\00", passed as nish_sandbox's third (fs.read) and fourth (fs.write) operand, null when main does not reach the capability. Landlock grants the scoped access beneath each directory and nothing else, except that a main reaching process.spawn keeps reading every file beneath / (READ_FILE, so that exec can load a program), scope or not; chmod, chown, utimensat, setxattr and their siblings still answer EPERM inside an fs.write scope, since seccomp cannot read a path. A main that reaches ffi is widened by the policy's net, fs.read, fs.write and process.spawn grants, a scope included, since foreign code reaches them without the checker seeing it. A directory cannot hold a comma.
  • The CPU count. Before it confines anything, nish_sandbox reads the CPU count (the affinity mask and the cgroup quota) and keeps it as the thread pool's ceiling, since without fs.read the quota under /sys/fs/cgroup cannot be read afterwards. NISH_NUM_THREADS still lowers the pool. availableParallelism() reads the count again on each call and is not bounded by that ceiling, so in such a program it no longer sees the quota.
  • Always. ptrace, process_vm_readv, process_vm_writev and pidfd_getfd kill; io_uring kills without net, and with net but not process.spawn only io_uring_enter is let through, on the premade rings' descriptors, matched by descriptor number (see the premade rings below); on x86-64 a system call numbered with bit 30 (x32) kills, and a call from any other architecture's entry kills. A killed process ends by SIGSYS, exit status 159 from a shell. From Landlock ABI 6 a sandboxed process cannot signal or reach the abstract sockets of a process outside it.
  • The premade rings. With net granted and process.spawn not, a program whose main reaches tcpListenFlags with NET_LISTEN_RING gets its rings before the filter loads: nish.netRings in the root package.json (an integer from 1 to 64, default 1; anything else is NL3058) restricted rings, each taking only ACCEPT, RECV, SEND, CLOSE, ASYNC_CANCEL and NOP, with the submission flags FIXED_FILE, BUFFER_SELECT and CQE_SKIP_SUCCESS; any other submission completes with -13. Ring listeners take them as they fill: once every ring made is in use, a listener whose rings are full answers -23 from tcpAccept until a connection closes, a ring listener opened after that serves descriptors, and a kernel or limit that refuses them leaves the process on descriptors for good, which netRingState() reports. One more ring, the keeper, takes only NOP and is entered by nothing; a ring a closed listener held has the keeper put at its descriptor number, so no premade ring's number is ever free. close, dup2 and dup3 naming a ring's or the keeper's number as the descriptor they close or replace kill, except a dup3 from the keeper, which is how the runtime puts the keeper at a freed ring's number, and close_range fails with ENOSYS, as on a kernel older than 5.9; on every other descriptor they work as before (NURING-36 in docs/security/net-uring.md).
  • Refusals. A program whose sandbox cannot be applied prints one line, nish: --sandbox: ..., and exits 126 before main runs. Strict refuses a kernel without Landlock, and one below ABI 3 when fs.write is not granted, since TRUNCATE cannot be withheld there, or when it is granted only beneath a directory, since below ABI 3 that grants nothing beneath /; best-effort runs on both, naming what it could not withhold (without Landlock, files are not confined at all). A scope directory missing at run time is refused by strict, nish: --sandbox: cannot open the directory <dir> (ENOENT); best-effort leaves its rule out, so nothing beneath it is granted, and names it once. Neither mode refuses a kernel without IOCTL_DEV (ABI 5) or the signal and socket scopes (ABI 6); best-effort names them.
  • Targets. A darwin target, given or the host's, is refused (NL3044, tests/wordings/nl3044_sandbox_darwin). The wasm and WASI targets (wasm32, wasm32-wasi) accept the flag and emit no call, since the host already holds a module to its imports; --profile wasm refuses IR built with --sandbox for the host. A .ll built with --sandbox and linked on a platform other than Linux refuses to run, exit 126.
  • No byte of a program's IR moves without the flag: the entry, its @llvm.used slot and the two declarations are emitted only under --sandbox ("sandbox: without the flag the IR is the sandboxed IR less the sandbox's lines"). --runtime-decls, which lists every declaration the runtime answers, lists the two whatever the flag.
  • Reserved names. The runtime calls prctl and syscall from C, so in every build, sandboxed or not, a declare function of either name is refused as a reserved C symbol (NL3037).