Declarations

Program structure

A module (one .ts file) may contain, at the top level, only function declarations (a const bound to an arrow, or the legacy function keyword), class and interface declarations, const declarations (Module constants), type aliases (Type aliases), numeric enum declarations (Enums), declare function declarations (Calling C), import statements, and export modifiers on those declarations. The entry module may instead hold top-level code, which runs as its main. In any other module, and in an entry that declares main, any other top-level statement is rejected with Only top-level function declarations are supported in Phase 1 (found <Kind>), where <Kind> is TypeScript's name for the statement, ExpressionStatement or IfStatement (tests/cases/reject_top_level_stmt); a top-level let names the reason instead (Top-level `let` is not supported; a module has no top-level code, so only `const` is available, reject_const_top_level_let), and var is refused wherever it is written (Forbidden constructs). Modules have no top-level code, so there is no initialisation order to worry about — which is exactly why a top-level const must be a value the compiler can compute for itself. The one piece of state a module keeps is module storage, a const array laid out and zero-filled at compile time, which is a declaration, not code.

Top-level code

An entry module that declares no main may hold statements at the top level, the way a script does. They run as the program's main: the compiler moves them, in order, into an exported main(): void before anything else reads the tree, so the native program is the one a hand-written main gives and exits 0 unless the code calls process.exit (tests/cases/script_basic, script_exit).

const LIMIT: i32 = 4

const capped = (n: i32): i32 => (n > LIMIT ? LIMIT : n)

const start = 1
let total = start
for (let i = 0; i < 3; i++) {
  total += capped(i + 2)
}
console.log(total)              // 10
  • What moves: every top-level statement (an expression, if, a loop, switch, a block, a labelled statement, throw, try), every top-level let, and every top-level const that is not exported and that no declaration names. return, break and continue stay refused.
  • What stays: functions, classes, interfaces, enums, types, imports, and a const that a function, a class or another staying constant names. That const is a module constant with all its rules, since a function cannot read a local of main; LIMIT above is one. A variable of the code that a function names is refused with `count` is a variable of the entry module's top-level code, which runs as `main`, so a function cannot read it (reject_script_local_in_function).
  • Declare before the code reaches it. Under Node a top-level statement runs where it stands, before the declarations below it exist; natively they all exist first. So a statement that reaches a function, class, enum or constant, by naming it or through a function that does, must stand below it: `twice` is declared below the top-level code on line 5 that reaches it (reject_script_declared_below).
  • One scope. Under Node the code's variables share the module scope with its declarations, so a variable may not take a name the module declares or imports: `go` is a variable of the top-level code and also a name the module declares or imports (reject_script_name_collision).
  • Only the entry, and only without main. An imported module has no top-level code, and neither has an entry that declares main in either spelling (reject_top_level_stmt, reject_const_top_level_let). An entry of nothing but constants is a module, not a script.
  • There is no top-level await: await is written only inside an async function (NL2641, Async functions), so a script that waits declares export async function main instead.
  • --emit-ts writes the code back where it stands, and its entry.ts imports the module rather than calling main.

Functions

A function is declared as an arrow bound to a module-level const (wp22-arrow-functions.md):

const add = (a: number, b: number): number => {
  return a + b;
};

const double = (n: i32): i32 => n * 2; // a concise body is that one `return`

The function keyword declares the same thing and is accepted as the legacy spelling; the two compile to identical IR, instruction for instruction (tests/cases/fn_arrow).

  • The arrow form is a const — let is Function `f` must be declared `const`, not `let` (tests/cases/reject_arrow_let) — binding one name (const f = (): i32 => 1, g = 2 is A function declaration binds one name, reject_fn_two_names), with the signature on the arrow itself. Annotating the const is Function `f` takes its signature from the arrow; drop the annotation on `f` (reject_arrow_annotated): the signature is written once, on the arrow. A function returning void uses a block.
  • A concise body (=> n * 2) means exactly what a block with one return means, everywhere the difference could show. It takes the return type as its contextual type, so an object literal gets its struct, a class value converts to an interface it implements, a numeric literal takes the declared width and [] takes its element type; and a value it allocates is returned rather than local, so the caller brackets the call with the arena reclaim (Memory model). tests/cases/fn_arrow_concise pins all five, each of which the block form has always had.
  • A capture-free top-level function is a value in either spelling: a name bound to a non-generic, non-async function this program defines, with no rest parameter, is a value of its function type, so const alias = double is a function reference. It can be held in a local, a field, an array element, a return value or a type alias, passed, called and compared with === and !==, and nothing else (tests/cases/fnref_field, fnref_array, fnref_eq, fnref_stores). An arrow that is not a top-level const is no reference: one that captures is this arrow captures `k`: a function reference names a top-level function and carries no environment (tests/cases/reject_arrow_as_value), and one that does not is an arrow in value position is not yet a function reference: declare it as a top-level function and name it (reject_fnarg_arrow_value).
  • Every parameter and the return type must be annotated (Parameter `x` needs a type annotation; Function `f` needs an explicit return type annotation, tests/cases/reject_missing_return_type, reject_arrow_return_type). The return type is asked of every function and method as the module is read, a generic one nothing instantiates included (reject_method_return_template, reject_method_return_generic).
  • Functions may call each other in any order; signatures are collected before bodies are checked (tests/cases/locals, cf_fib). This holds for the arrow form too, which is not what a reader who knows that a const is not hoisted in JavaScript would expect: a module-level const bound to an arrow is a function declaration, so it is in scope for the whole module and not only below its own line, and two of them may call each other (tests/cases/fn_arrow_hoisting). Nothing else would let a file be converted from one spelling to the other without reordering it (wp22-arrow-functions.md §8b).
  • Calls take exactly the declared number of arguments (expects 1 argument, tests/cases/reject_arity), each of exactly the declared type (Argument 1 of `area`: expected Shape, got Rect, tests/cases/reject_cls_not_implements).
  • Parameters are immutable: p = 1, p++, p += 1 are Cannot assign to `p` because it is a parameter (tests/cases/reject_assign_param, reject_cf_incdec_param).
  • Parameters and the body's top-level block share one scope: let x in the body of f(x) is Duplicate declaration of `x` (CLI only).
  • A function's name shares one declaration namespace with classes, interfaces, type aliases, enums and module constants, whichever declaration came first. A class or interface of the same name is `Point` is already declared as a class or interface (tests/cases/reject_cls_name_is_function); everything else is `value` is already declared in this module (tests/cases/reject_fn_after_const).
  • A function may be generic, and each instantiation is compiled on its own; see Generic functions. Rejected forms: generators (reject_generator), an async function without its Promise<T> (Async functions, reject_async_function), destructured parameters (Destructured parameters are not supported, reject_param_destructured), optional parameters (x?: T, Optional parameters are not supported, reject_param_optional, which names x: T = <default> instead; the compatibility dialect's optional-params feature accepts one read through ??, The compatibility dialect), a default or rest parameter anywhere but the declarations Default and rest parameters lists (reject_param_default, reject_param_rest), overloads (Duplicate function `f`, tests/cases/reject_fn_duplicate; a signature with no body, function f(): void;, is Functions must have a body; the compatibility dialect's overloads feature accepts signatures before the implementation), an anonymous export default function () (Functions must be named, reject_fn_anonymous), function expressions, an arrow anywhere but as a top-level const or a function argument (reject_fnarg_arrow_value, reject_arrow_as_value), and nested function declarations (Unsupported statement in Phase 1: FunctionDeclaration, reject_fn_nested, in a template nothing instantiates too, reject_fn_nested_template; an import in a body is Unsupported statement in Phase 1: ImportDeclaration).
  • A non-void function must return on every path (Function `f` must return a value of type i32 on every path, tests/cases/reject_missing_return, reject_cf_missing_return_loop); see Termination.

Default and rest parameters

const scale = (x: number, by: number = 2): number => { return x * by }
const area = (w: number, h: number = w): number => { return w * h }
const total = (...xs: i64[]): i64 => { let s: i64 = 0; for (const x of xs) { s = s + x } return s }

scale(3)        // scale(3, 2)
area(4)         // area(4, 4)
total()         // total([])
total(1, 2, 3)  // total([1, 2, 3])

A function, a method with a body, a constructor, and an arrow bound to a module const may give trailing parameters a default or end in one rest parameter. Both are filled in at the call site: the callee is compiled as if every argument were always passed, with no branch on a missing one, so a call that passes every argument compiles to the same IR it did before the function had a default (tests/cases/param_default, param_rest, param_method, param_generic, across modules tests/link/param_modules).

  • A default is a constant or an earlier parameter: a numeric literal (a negative one too), true/false, a string literal, an enum member, or the name of a parameter written before it, or null for a T | null parameter (next rule). Anything else is refused (reject_param_default_expression, reject_param_default_later, reject_param_default_array, reject_param_default_fraction).
  • A T | null parameter defaults to null, and only to null: opts: Options | null = null makes a call that leaves opts out pass a typed null pointer, the same IR as passing null itself (tests/cases/param_default_null, across modules tests/link/param_modules). This is how an options argument is written without undefined: fetch(url) and fetch(url, init) are one function. null is refused as the default of a type with no null (reject_param_default_null_not_nullable), any other default of a T | null is refused, since every call that omits it would build the value (reject_param_default_object), and T | null of a type parameter takes no default (reject_param_default_null_generic).
  • A default's parameter is annotated, n: i32 = 1, never n = 1 (reject_param_default_untyped), and its type is a number, boolean, string, enum or a T | null defaulting to null (reject_param_default_parameter_type), not a type parameter (reject_param_default_generic). Every parameter after one with a default has one too (reject_param_default_order).
  • A rest parameter is last, has an array type and no default (reject_param_rest_not_last, reject_param_rest_not_array, reject_param_rest_default), and a function has a default or a rest, not both (reject_param_rest_and_default). The call gathers its trailing arguments into one array of exactly that many elements, on the stack unless the callee lets it escape — the same array an explicit literal total([1, 2, 3]) builds.
  • A rest of records takes only fresh object literals. The array copies each inline record (an interface no class implements) into its slot, so a named record passed through a rest would be a copy the callee's write never reaches, where TypeScript would write the caller's own: bumpAll(p) is A rest parameter copies each `Cell` record it gathers, ... (reject_param_rest_record). Pass [p] to a Cell[] parameter, where the copy is written out. A class instance is a reference, so a rest of class instances gathers the caller's objects.
  • Not on a function-typed parameter, which a call names by value (reject_param_function_default, reject_param_function_type), and a function with a rest parameter is never passed for a function parameter (reject_param_rest_function_value) nor held in a function reference (NL2729, reject_fnref_rest): called through either, Node would gather differently.
  • No spread argument: total(...xs) is Spread arguments are not supported (reject_param_spread_call); pass xs to a parameter of array type.
  • Too few or too many arguments name the range the function takes (reject_param_arity).
  • A host sees neither. Defaults and rest are filled in by a Nish call site, so an exported function's symbol takes every parameter: the --emit-dts typings, the --emit-header prototype and the --emit-napi shim require each argument, and take a rest parameter as one plain array (wp8-interop.md).
  • What a call evaluates is what TypeScript evaluates: each argument once, left to right, whether it fills a default's place or gathers into a rest (tests/cases/param_eval_order). A rest array the callee returns, stores in a field or lets a caller keep is allocated in the arena, not on the stack (param_rest_escape); a default on an imported class's constructor or method is filled in by the importing module (tests/link/param_default_import).

Module constants

const TOKEN_NAME: i32 = 1;
const TOKEN_END: i32 = TOKEN_NAME + 1;
export const PROMPT: string = "> ";

A top-level const names a value the compiler already knows. It is not a global variable: no symbol is emitted, no initialiser runs, and every use of the name lowers to the value itself — AREA below is the literal 64 in the IR, not a load (tests/cases/const_basic, const_fold, dump_checked_const). That is what lets a module keep its table of kinds and limits while still having no top-level code and therefore no initialisation order.

  • The type annotation is required, as on every other top-level declaration (Module constant `answer` needs a type annotation, tests/cases/reject_const_no_annotation), and so is an initialiser.
  • The type must be i32, i64, f64, number, boolean, or string. A class or interface constant would need an allocation, and there is no code to run it (must be a number, boolean, string, or a `readonly` array of numbers, tests/cases/reject_ffi_handle_module_binding). There are two exceptions with rules of their own: a readonly array of numbers written out as a literal, which is a constant table, and a new Array<T>(n) or typed-array new of a compile-time length, which is module storage.
  • The initialiser is an ordinary Nish expression restricted to literals and other constants: the operators of Operators over numeric, boolean, and string literals, other module constants (declared anywhere in the module, or imported), and parentheses. Anything else — a call, a new (other than module storage's), a local, a parameter — is A module constant's initialiser must be a literal, another constant, or arithmetic over them (tests/cases/reject_const_not_constant, reject_const_unknown_name). There is deliberately no second, larger language inside const: a constant can compute exactly what a runtime expression can.
  • Folding follows the language's own arithmetic, whichever mode the compiler is in, because a constant must compute what the instruction it replaces computes. By default that instruction is checked, so a fold that overflows is attempt to compute with overflow in a constant — the run-time operation would panic, and a constant has no run time to panic at (tests/cases/reject_const_overflow_arith). A wrap the program means is written wrappingAdd, wrappingSub or wrappingMul from nish:unsafe, the one call a constant's initialiser may make, and that fold wraps: const WRAPPED: i32 = wrappingAdd(2147483647, 1) is -2147483648 (tests/cases/unsafe_wrapping). Negation is arithmetic too, so -2147483648 is fine (its result fits) while -(-2147483648) is refused. A bare integer literal takes the width of the constant it initialises, so const BIG: i64 = 1000000000 * 10 is computed in 64 bits (const_i64). Mixing widths is an error, as it is anywhere else (reject_const_mixed_widths). f64 folds in IEEE-754 (const_f64), and + on two strings concatenates at compile time, interning the result once (const_string).
  • The two failures the divisor check catches at run time are refused at compile time instead: 1 / 0 and MIN / -1 in a constant are attempt to divide by zero in a constant and attempt to divide with overflow in a constant (tests/cases/reject_const_div_zero; see Checked integer division).
  • A value that does not fit its annotation is an error, not a wrap: const TOO_BIG: i32 = 3000000000 is Constant `TOO_BIG` does not fit in i32 (tests/cases/reject_const_overflow), and a folded value of the wrong type is declared i32 but its value is string (reject_const_type_mismatch).
  • A constant cannot be assigned: Cannot assign to `LIMIT` because it is a module constant covers =, op=, ++ and -- (tests/cases/reject_const_assign, reject_const_incdec).
  • A local shadows a constant of the same name, as it would in TypeScript: the scope chain is consulted first (tests/cases/const_shadow).
  • A constant may name a constant declared later, in the same module or in an imported one, because folding is by need rather than in source order. A cycle is Module constant `A` is defined in terms of itself (tests/cases/reject_const_cycle).
  • A string constant is a value, so it is a receiver like any other string: NAME.length is the folded literal's byte length (tests/cases/const_string_member).

Constant tables

const CLASS: readonly u8[] = [0, 0, 1, 1, 2, 3]
const STEPS: readonly i32[] = [-1, 0, 1, LIMIT]

A top-level const of type readonly T[] whose initialiser is an array literal is a constant table: its elements are folded at compile time and laid out as one read-only data global, with a constant array header over it, so no code runs to build it and nothing can write it (tests/cases/const_array_u8, const_array_numeric). A use of the name is the header's address — the same readonly T[] value any other read-only array is — so CLASS[c], CLASS.length, for (const v of CLASS) and passing it to a readonly T[] parameter all work, and CLASS[c] with c: u8 into a 256-entry table is one load the bounds prover has already discharged.

  • The element type is u8, u16, u32, i32, i64 or f64 — the numbers whose bytes are their value. Anything else is Constant table `XS` cannot hold string (tests/cases/reject_const_array_element_type).
  • The annotation must be readonly T[] (or ReadonlyArray<T>): the data is constant, and a plain T[] could be written through. const XS: i32[] = [1, 2, 3] is must be declared `readonly i32[]` (tests/cases/reject_const_array).
  • Each element is a constant expression under the rule above — literals, other module constants and arithmetic over them (tests/cases/reject_const_array_nonconst) — and must fit the element type: 256 in a readonly u8[] is Element 2 of constant table `XS` does not fit in u8 (tests/cases/reject_const_array_element). An empty literal is refused (has no elements, reject_const_array_empty).
  • Writing is refused like any write to a readonly array: an element store, push, pop or any other mutating method (tests/cases/reject_const_array_write, reject_const_array_push), and handing the table to a mutable T[] — a parameter or a local (reject_const_array_mutable, reject_const_array_alias).
  • The symbols are @nish_const_<module>_<name>.data (private unnamed_addr constant) and @nish_const_<module>_<name> (the header, internal constant), <module> the module's index. An exported table follows module storage's rule: the header in its own module is not internal, and an importer declares it external constant.

Module storage

A top-level const whose initialiser is new Array<T>(n), or new of one of the typed-array names, is module storage: a fixed array the module keeps between calls, laid out by the compiler and zero-filled before the first call, so no code runs to build it and the module still has no top-level code (tests/cases/storage_module, docs/wp50-wasm-host.md §5). It is the one place a module keeps state, and the place a page reads and writes a wasm module's memory in place (Host modules). It is legal in every build, native included.

interface Body { x: f32; vx: f32; hits: u32; alive: boolean }

export const SLOTS: i32 = 8
export const bodies = new Array<Body>(SLOTS)
export const scale = new Float64Array(2)

export const step = (): void => {
  for (let i = 0; i < bodies.length; i++) {
    if (bodies[i].alive) bodies[i].x = bodies[i].x + bodies[i].vx
  }
}
@nish_storage_0_bodies.data = global [8 x %struct.Body] zeroinitializer, align 16
@nish_storage_0_bodies = global %struct.nish_array { i64 8, i64 8, i8* bitcast ([8 x %struct.Body]* @nish_storage_0_bodies.data to i8*) }, align 8
  • Two globals. <symbol>.data holds the elements and <symbol> is an ordinary array header over them, so every read and write is the indexing an array already lowers to. The symbol is nish_storage_<module>_<name>, with <module> the module's place in the program, so two modules may each keep a private storage of one name, and none takes a C symbol such as write. The data has external linkage: the --profile wasm-host link exports it, and the loader reads its address from that export. The header is internal unless the storage is exported; another module reaches an exported one as @<symbol> = external global %struct.nish_array (cookbook).
  • The length is known at compile time: a literal, a module constant, or arithmetic over them (wrappingAdd and the other nish:unsafe folds included), from 1 to 16,777,216 elements and at most 256 MiB (268,435,456 bytes). Anything else is Module storage `X` needs a length known at compile time: a literal, a module constant, or arithmetic over them, because the page reads its layout from the type (NL2676, reject_storage_dynamic_length), and a length outside the range is Module storage `X` has length 0; a storage holds from 1 to 16777216 elements (NL2680, reject_storage_length_zero; past 256 MiB the message names the element's size, reject_storage_too_many_bytes).
  • The element is pointer-free, and zero is one of its values: a number, a boolean, an enum or a ranged integer whose zero is a member, a HostHandle, or an interface no class implements whose fields are all of these. A string, an array, a class, a Map or a | null (but C | null for a class C, which is root storage) is Module storage `X` cannot hold T: its bytes are zero-filled where a page can read them, … (NL2686, reject_storage_string_element, _array_element, _class_element, _map_element, _nullable_element, _record_string_field). An interface a class implements, generic classes included, is a class's shape and is refused the same way (reject_storage_implemented_interface, _generic_implementer). A field may not be another record, since a record field is a pointer to it: Module storage `X` cannot hold Out.p (In): a record field that is itself a record is a pointer to it, and storage holds no pointer; spell the inner fields on the outer record (NL2685, reject_storage_nested_record). An enum or range without a zero is Module storage `X` is zero-filled, and 0 is not a value of T; give the type a zero member, or store the number and convert it where it is read (NL2687, reject_storage_enum_no_zero, _range_no_zero).
  • An annotation must say what the new makes: const xs: f64[] = new Int32Array(4) is NL2679 (reject_storage_annotation). The annotation may be left off.
  • It never moves and never resizes. push, pop, sort, any other method and X.length = are Module storage `X` has a fixed length and a fixed place, so `X.m` is not available: index it, and keep a count beside it for the slots in use (NL2688, reject_storage_push, _pop, _sort, _length_assign). X[i] and X.length are what remain.
  • Its name is a place, not a value. Binding, returning, storing or passing X itself is Module storage `X` is a place, not a value: index it (`X[i]`) or read `X.length`, lend it only to a parameter declared `u8[]`, `Int32Array`, `Float32Array`, `Float64Array` or `BigInt64Array` or to a host function's buffer window, and never alias, return or store the array itself (NL2677, reject_storage_pass_whole). The two exceptions are lendings: a scalar storage passed to a parameter spelled with a typed-array name (which has no push, pop or length =), as in const total = (xs: Int32Array): i32 => …; total(SCORES) (tests/cases/storage_lend), or a byte storage passed to a u8[] or readonly u8[] parameter, the spelling a byte buffer has (tests/cases/storage_lend_bytes, which nish/host/text writes through), and to a host function's buffer window (Host modules). A record storage is never lent. A lending to a function of the same module holds only while that function keeps the parameter in place: it may index it, read .length (never assign it or ++ it), loop over it with for of, call a method that reads it (at, indexOf, slice, map, …), call fill, sort, reverse or copyWithin as a statement, copy it with String.fromBytes, and lend it on to another such parameter or a buffer window. Anything else (binding it under another name, returning it, passing it to an i32[] parameter, which could push) is Module storage `X` cannot be lent to `f`, which lets its parameter `a` go further than indexing: … (NL2684, reject_storage_lend_respell_arg, _const, _return). The rule holds wherever the callee is declared: a function another module declares is judged by its own body, read in its own module, so a lending on from there resolves the next callee by that module's names — its imports and import aliases (import { good as sum }) included — and is judged the same way, for as many hops as it lends on (tests/link/storage_lend_import, reject_storage_lend_import). A callee name the calling function binds itself (a parameter, a local, a pattern) is not a declared function and is refused.
  • A record element is a place too. X[i].x reads and writes a field, and X[i] = X[j] or X[i] = v copies a whole element, field by field. Binding, returning or storing X[i] is NL2674 (reject_storage_bind_element); passing it is An element of module storage `X` cannot be passed as an argument: it is a place, not a value; pass the index, or the fields the callee needs (NL2675, reject_storage_pass_element), except out.push(X[i]) onto an array of the same record, which copies; and for (const p of X) over a record storage is NL2678 (reject_storage_for_of): loop over the index. A for-of over a scalar storage binds numbers and is allowed.
  • It is memory every function reaches, so it moves the inferred attributes: a function that reads storage is at most readonly, one that writes it is neither readonly nor readnone (tests/cases/storage_attributes), and a parallelMap body that writes it is refused as any shared write is (reject_storage_parallel_write).
  • Under --emit-ts it is a typed view, and only it. The reading writes one more file, nish-storage.ts, holding one ArrayBuffer laid out as the wasm build's layout says, memory ({ buffer, grow }, whose grow throws: the buffer never moves), layout itself, views(), loadSync and load as the wasm loader has them (they take the module's exports where the wasm loader takes its bytes), and each storage as a view of the buffer; the module's new is written as that view (src/emit-ts-storage.ts, docs/wp50-wasm-host.md §5.4). A scalar storage is the typed array its element reads as (Int32Array, Float32Array, …; a boolean one is a byte array behind a proxy that reads !== 0). A record storage is an array of objects whose fields are getters and setters over the buffer, so X[i].x = v writes the bytes a page reads, and a whole-element copy is Object.assign. Each storage starts on an 8-byte boundary from 0, so the reading's offsets differ from the wasm build's; a page reads them from layout and never spells one. A local f32[] is still a plain JavaScript array there: only module storage is a view.

Root storage: an object graph kept between calls

Module storage whose element is C | null for a class C is root storage: each slot holds one object, and everything that object reaches — its fields, their arrays and strings, the objects they hold — outlives the call that built it (tests/cases/root_storage, docs/wp55-persistent-root.md). It is how a wasm module keeps a game, a document or any class instance across init, step and draw.

class World {
  balls: Ball[]
  tick: i32 = 0
  constructor() {
    this.balls = []
  }
}

const world = new Array<World | null>(1)

export const init = (): void => {
  world[0] = new World()
}

export const step = (): void => {
  const w = world[0]
  if (w !== null) {
    w.tick = w.tick + 1
  }
}

On a page module each root is a record of its data, its length and the descriptor of its class, gathered into one table the keep walks (tests/cases/root_storage_page, whose root is head, a list of Node):

%nish.root = type { i8*, i32, i8* }
@nish_storage_0_head.root = constant %nish.root { i8* bitcast ([1 x %struct.Node*]* @nish_storage_0_head.data to i8*), i32 1, i8* bitcast (%nish.trace* @nish.trace.0 to i8*) }
@nish.roots = internal constant [1 x %nish.root*] [%nish.root* @nish_storage_0_head.root]

define void @nish_root_keep(i64 %mark) {
  call void @nish_runtime_root_keep(i64 %mark, i8* bitcast ([1 x %nish.root*]* @nish.roots to i8*), i32 1)
  ret void
}
  • Natively it is module storage and nothing more. A native program never hands its arena back between calls, so the graph simply stays.
  • On a page module the loader keeps the graph and releases the rest. Where the loader of a program without root storage releases each export call's arena, this one calls nish_root_keep. The runtime traces what the roots reach from descriptors the compiler writes (%nish.trace, one per class, string and array type the roots reach; tests/cases/root_storage_page), slides it down over the garbage, rewrites every pointer to it, and frees what is left. It does so only once the kept region passes twice the live bytes and 64 KiB, so a call that changes a little copies nothing, and memory stays bounded however long the page runs (node tests/run.js persist: 100,000 steps with memory flat).
  • What a root reaches is traceable. A number, a boolean, an enum, a string, an array, a class or a discriminated union of classes. Anything else — a Map, a Set, an interface a class implements, a Region, a Secret, a CPtr, a handle class or a C struct layout — is Root storage `X` cannot keep C.f (T): the runtime moves what a root reaches between calls, … (NL2766, tests/wordings/nl2766_root_storage_map).
  • It is not exported: the objects move, so a page has no layout to read. export const world = new Array<World | null>(1) is Root storage `X` cannot be exported: the runtime moves the objects its elements point to between calls, so a page has no layout it could read; export a function that reads it instead (NL2767, tests/wordings/nl2767_root_storage_export), and layout leaves root storage out.
  • No threads, no regions. Each allocates from an arena of its own, which a root could be left pointing into, so root storage in a program that loads nish/threads or nish:region is Root storage `X` cannot be kept in a program that loads `nish/threads` or `nish:region`: … (NL2768, tests/wordings/nl2768_root_storage_threads).
  • No native shim. A build that writes --emit-napi, --emit-napi-async or --emit-header gets a shim that releases the arena after a call that passes a string or an array, which would free what a root reaches, so root storage there is Root storage `X` cannot be kept in a build that writes a native shim (`--emit-napi`, `--emit-napi-async` or `--emit-header`): … (NL2769, tests/wordings/nl2769_root_storage_napi). A plain native program keeps it.
  • No WASI loader. A --profile wasi build with --emit-dts writes a loader that releases the arena after every call, and only a page module emits the nish_root_keep a loader calls instead, so root storage there is Root storage `X` cannot be kept in a WASI build that writes a loader (`--emit-dts`): … (NL2770, tests/wordings/nl2770_root_storage_wasi_dts).
  • A freestanding module without a loader (--target wasm32 and no --emit-dts) still emits nish_root_keep(mark); the host that calls its exports writes its own loader, and must call nish_root_keep(mark) where it would call nish_arena_release(mark), or what a root reaches is freed.
  • Every function may store through a root, so a program with root storage does not prove a parameter-only allocation stays in its caller's scope from the signature alone; the compiler's other proofs are unchanged.

Type aliases

type Byte = u8;
type Bytes = Byte[];
type Reading = Sample | null;

A type alias is a second name for a type that already exists. It is not a type of its own: type Byte = u8 says that Byte and u8 are two spellings of one type, exactly as Int32Array is a spelling of i32[] (see Types). So sameType never sees the alias's name, there is no new LLVM type, and an alias emits nothing at all — the same program written with and without its aliases produces byte-identical IR, which is what tests/cases/type_alias_ir and tests/cases/type_alias_expanded are: one program twice, with one golden between them.

  • The right-hand side may be any type the language accepts: a scalar, string, void, T[], readonly T[], T | null, Result<T, E>, a declared class or interface, or another alias (tests/cases/type_alias_ir). Everything is refused exactly where it would be in an annotation, so type Bad = any is `any` is forbidden and not a rule of its own, and an object type, type Point = { x: i32; y: i32 }, is the syntax error an inline object type is anywhere else (tests/cases/reject_type_object_literal_alias): declare interface Point instead.
  • Order does not matter. An alias may name a class, an interface or an alias declared further down the file, because resolution is by need rather than in source order — the rule module constants follow. A cycle is Type alias `Feet` is defined in terms of itself (tests/cases/reject_type_alias_cycle), the wording the constant cycle uses, because it is the same mistake.
  • An alias is resolved even when nothing uses it, so a broken right-hand side is reported where it is written rather than never.
  • Generic aliases are forbidden, unlike a generic function: an alias only renames a type that already exists, so there is nothing for a type argument to apply to. type Box<T> = T[] is Generic type parameters are forbidden on a type alias in Nish; a generic function, class or interface is monomorphised, and an alias only renames a type that already exists (NL1054, tests/cases/reject_type_alias_generic).
  • A built-in type name may not be aliased. type string = i32 is `string` is a built-in type name and cannot be used for a type alias (tests/cases/reject_type_alias_builtin). A keyword like string is resolved from the syntax, and i32, Array, ReadonlyArray, Result and the typed-array names before any declared name, so such an alias would otherwise sit there meaning nothing.
  • The name shares one declaration namespace with functions, classes, interfaces and module constants: a clash is `Point` is already declared in this module whichever came first (tests/cases/reject_type_alias_duplicate).
  • An exported alias is imported, and in the importer it is the type it names. export type Pt = Point in shapes.ts and import { Pt } from "./shapes" elsewhere make Pt a second spelling of Point there too: a field, parameter or return type, a local, an array element, and a Point from new passes where a Pt is declared (tests/link/alias_export_class). The right-hand side is resolved in the module that wrote it, whichever module asks first, so export type Conn = Socket | null works when Socket is a class the exporter itself imported, and the importer never has to import Socket (alias_export_imported_class). An array, a T | null and a Result alias are the types they name (alias_export_array, alias_export_nullable, alias_export_result), so is an alias of a generic class's instantiation, which the exporter instantiates even when an importer asks first (alias_export_generic), a chain of aliases across modules resolves by need (alias_export_chain: Table is Line[], Line is an imported Row, Row is Cell[], Cell is i32), and unlike a class an alias may be renamed on import — import { Conn as Handle } — because it has no symbol for its name to be part of (alias_export_rename). Nothing is emitted in either module, so the importer's IR is byte-identical to the same program with every alias written out (alias_export_ir and alias_export_ir_local, whose .ll goldens are the same files).
    • A mistake on the right-hand side is the exporter's. It is reported against the module that wrote the alias, even when another module asked for the alias first (tests/link/alias_export_broken), and a cycle that runs through two modules' aliases is Type alias `B` is defined in terms of itself, at the first alias of the cycle the report reaches (alias_export_cycle).
    • The import errors are the ones every import has. An alias declared without export is `Conn` is declared in `./conn` but not exported (add `export`) (tests/link/alias_import_not_exported); a declaration of the imported name is `Conn` is already declared in this module (alias_import_clash); a second import of it is `Conn` is already imported from `./conn` (alias_import_duplicate); and an alias is a type, not a value: new Conn() is Unknown class `Conn`, as it is for a local alias (alias_import_as_value). A module does not re-export an alias it imported. An imported alias is bound before the signatures, for the reason an imported enum is.
  • Aliases are top-level only. type X = i32 inside a function body is Unsupported statement in Phase 1: TypeAliasDeclaration (CLI only).

Enums

enum Kind {
  If = 1,
  While = 2,
  Return = 3,
}

enum Level {
  Low,   // 0
  Mid,   // 1
  High,  // 2
}

enum Role {
  Admin = "admin",
  Guest = "guest",
}

An enum is of one of two kinds. A numeric enum's members are integers; a string enum's members are each given a string literal, and its rules past the ones below are in String enums. Either kind declares a distinct type whose representation is i32: a numeric member's integer, or a string member's ordinal. Distinct is the point, and it is stricter than TypeScript, where a numeric enum member is assignable to number and a string one to string: the rule this language is built on is that two values are compatible only when their types are identical (Types), and an enum that were silently i32 would be the one type compatible with something it is not spelled as. So Kind and i32 are two types, in both directions, as Role and string are, and the only values of Kind are the members it declares.

Nothing is emitted for a numeric enum (a string enum adds its one table of strings, String enums). A member is folded to its integer the way a module constant is folded to its value, so Kind.While lowers to the literal 2, a Kind parameter is an i32 parameter, and there is no symbol, no table and no %struct anywhere. tests/cases/enum_ir and tests/cases/enum_expanded are one program written twice — with the enum, and with i32 module constants — and their .ll goldens are byte-identical files.

  • A member's value is a numeric literal, or the one before it plus one. Auto-numbering is TypeScript's: the first member without an initialiser is 0. An enum any of whose members is given a string literal is a string enum instead. A member that is neither — a computed value in a numeric enum, or a numeric member beside a string one — is Enum members must be numeric literals in Nish, or every member a string literal (an enum lowers to a plain integer, and one numbering cannot mix with strings) (NL1043, tests/cases/reject_enum_computed, reject_enum_string_mixed); a numeric literal that is not an integer is Enum member `Kind.Half` must be an integer literal (reject_enum_float), and one outside the width is Enum member `Kind.Big` does not fit in i32 (reject_enum_overflow).

  • An enum must declare at least one member, because an empty one is a type no value can inhabit: Enum `Kind` must declare at least one member (tests/cases/reject_enum_empty). Two members of one name are Duplicate member `If` in enum `Kind` (reject_enum_duplicate_member).

  • A value is spelled Kind.If and nothing else. A name that is not a member is Enum `Kind` has no member `Nope` (tests/cases/reject_enum_unknown_member), and there is no conversion in either direction: const n: i32 = k is Cannot initialize i32 variable `n` with Kind (reject_enum_not_i32) and const k: Kind = 1 is Cannot initialize Kind variable `k` with i32 (reject_enum_from_i32).

  • === and !== are the only operators. Two values of one enum compare; everything else is refused because an enum is not numeric — + is Operator `+` requires two operands of the same numeric type or two strings, got Kind and Kind (tests/cases/reject_enum_arith), < is Operator `<` requires two numeric operands, got Kind and Kind (CLI only), and the bitwise operators are Operator `|` requires two operands of the same integer type, got Flag and Flag (reject_enum_bitwise). Bit flags therefore stay i32 module constants: A | B on an enum would produce a value that is no declared member, which is the one thing the type exists to prevent (wp23-language-surface.md §10 question 1 leaves the wider form open).

  • A switch takes an enum discriminant and enum-member labels (tests/cases/enum_ir), which is what an enum is for: the statement lowers to LLVM's switch exactly as an integer discriminant does. A label of any other type is `case` label is i32 but the discriminant is Kind (reject_enum_case_label), so the integer a member stands for is not a label. See switch.

  • A member is also a type: Kind.While names the type whose one value is that member. A slot of a member type — a field, a parameter, a return, a local, an element, an object-literal field or a Result arm — holds that member and nothing else, which is what lets a readonly field record which of an enum's cases its object is, as the tag a later discriminated union reads:

    class ArrayNode {
      readonly kind: Kind.While = Kind.While;
    }
    

    The slot is entered by the member's own constant, or by a value already of that member type; the enum's other members, and any Kind value that is not a constant, are Type `Kind.Array` holds only the member `Kind.Array`, got `Kind.Scalar` (NL2519, tests/cases/reject_enum_member_type_other) and …, got a value of type `Kind` (reject_enum_member_type_widened), because a Kind could be any member. Every way into a slot is held to it: a call argument, an assignment to a let or a field, a push, an element write, an array literal, an object-literal field, Ok, unwrapOr's default, a ternary arm and a return (reject_enum_member_type_argument, _assign, _field_assign, _push, _element_write, _array_literal, _object_field, _result_ok, _unwrap_or, _ternary, _return), and so is an argument of a generic call, whether the member type binds a type parameter (fold(xs, Kind.Array) with xs: Kind.Array[]), sits in a concrete parameter beside one, or is getOrInsert's value (tests/cases/enum_member_type_generic, reject_enum_member_type_generic, _fold, _get_or_insert, _generic_variable, _generic_method, _class_param); a parenthesised constant, (Kind.Array), is the constant. Two generic calls tsc accepts are refused, because the language binds a type parameter only from the arguments, and binds it from the constant before it reads a function argument: const y: Kind.Array = id(Kind.Array) binds T to Kind, since the slot the call returns into binds nothing (reject_enum_member_type_generic_return), and apply(take, Kind.Array) binds T to Kind before take: (x: Kind.Array) => Kind.Array is read (reject_enum_member_type_generic_function_arg). Pass a value already of the member type, id(a) and apply(take, a), and both bind Kind.Array. An unannotated let keeps the member type of a value already of it, as tsc's does, and widens a member constant to its enum: let w = keep(a) holds only Kind.Array, let w = Kind.Array any Kind (reject_enum_member_type_inferred_let). So does every place whose type is inferred from what is written: a ternary whose arms are both Kind.Array, an array literal of them ([a] is a Kind.Array[]), ?? between two Kind.Array[], each of those passed to a generic function, and an arrow's result, which binds the type parameter it returns: map([a], (k) => k) is a Kind.Array[] and apply(a, (k) => k) a Kind.Array, while an arrow that answers the constant, (k) => Kind.Array, answers Kind, as tsc widens it (tests/cases/enum_member_type_inferred, enum_member_type_arrow, reject_enum_member_type_ternary_let, _ternary_const, _ternary_generic, _array_inferred, _array_let, _array_generic, _nullish, _arrow_map, _arrow_apply). Two arms or two elements that differ where one is a member type, or an array of one — c ? a : b over Kind.Array and Kind.Scalar, c ? a : k over a Kind, c ? a : Kind.Array, [a, Kind.Array], or [[a], [Kind.Array]] — are refused, Ternary branches must have the same type, got Kind.Array and Kind.Scalar: an enum member type is not widened to its enum, so annotate the type it holds, e.g. `Kind`, and the array literal's message reads the same way, naming Kind[] for the nested case: tsc types them as a union of the members (the enum itself when they are all of its members), or as the member, or as the enum, by whether the constant is fresh, so the variable is annotated instead (reject_enum_member_type_ternary_mixed, _ternary_enum, _ternary_constant, _ternary_nested, _array_mixed, _array_constant, _array_nested_mixed). An object literal is always annotated, so its fields always have a type to enter. Two member types of one enum are two types (reject_enum_member_type_between), and a field initializer of a member type is held to the same rule (reject_enum_member_type_field_init). The other direction is free: a member type converts to its enum, and a read of a member-typed slot is the enum wherever a value is computed with, so it compares with ===, switches and passes as a Kind, and converts to i32 no more than a Kind does (reject_enum_member_type_read). In IR a member type is its enum's i32: tests/cases/enum_member_type lowers to the same bytes as the program with every member type written Kind (enum_member_type_widened, whose golden is byte for byte the same).

    • The qualifier is an enum, local or imported, and the name one of its members. import { Kind as K } spells the exporter's member types K.Array (tests/link/enum_member_type_import). Anything else before the dot is `Point.x` is not a type: only a member of an enum may be named with a dot in a type, and `Point` is not an enum (NL2517, reject_enum_member_type_not_enum), a missing member is Enum `Kind` has no member `Missing` to name as a type (NL2518, reject_enum_member_type_unknown), and type arguments are Unsupported type reference `Kind.Array<i32>` (reject_enum_member_type_args). One dot is read; A.B.C is a syntax error (reject_enum_member_type_nested).
    • Two members of one value are one type, as tsc reads them, spelled by the member declared first (a string enum has no two such members, String enums): with List = 1 beside Array, a Kind.List slot is reported as `Kind.Array` (reject_enum_member_type_same_value), and Kind.List enters a Kind.Array[] (tests/cases/enum_member_type_slots).
    • A member type is a type wherever one is written: a type alias's right-hand side (type ArrayTag = Kind.Array), a type argument (Box<Kind.Array>, Map<Kind.Array, i32>, Set<Kind.Array>), an element type and an arrow argument's return type, and a -g build describes it as its enum (enum_member_type_slots).
    • A zero-filled array of a member type holds 0, so new Array<Kind.Array>(n) is `new Array<Kind.Array>(n)` would zero-fill with 0, which is not the member `Kind.Array`; build it with `[]` and `push` instead (NL2520, reject_enum_member_type_zero_fill), and a member whose value is 0, Kind.Scalar here, is allowed (enum_member_type_slots). The parallelMap, parallelFrom and parallelReduce templates zero-fill the array their workers write, so a body answering such a member type is refused at the call, … answers the member type `Kind.Array`, and `parallelMap` zero-fills the array its workers write, which a `Kind.Array` cannot hold; declare the body to answer `Kind` (NL2521, reject_enum_member_type_parallel, _parallel_from, _parallel_reduce); a body answering the member at 0 runs in all three (tests/link/enum_member_type_parallel).
    • A field initializer may be a member's constant when the field is the enum or one of its member types: readonly kind: Kind = Kind.If stores the integer before the constructor runs, as a numeric literal would (enum_member_type_slots). Another enum's member, a member the enum does not declare, a member given to a number field and any other dotted initializer are refused (reject_enum_member_type_field_init_enum, _field_init_level, _field_init_unknown, _field_init_i32, _field_init_dotted).
    • Typed JSON refuses a member-typed field (tests/cases/json_reject_enum_member): it reads an enum as any of its members, which a member type cannot hold. The one exception is a union's tag: a member that tags a variant of a union is written as its name, or a string enum's member as its string, the union's typed JSON tag (tests/link/json_tagged_union, tests/link/json_enum_string). A member type crosses to C and JavaScript exactly where its enum does, which is nowhere.
  • A built-in type name may not be taken: enum i32 { ... } is `i32` is a built-in type name and cannot be used for an enum (tests/cases/reject_enum_builtin_name), for the reason the same rule exists for a type alias — the name would never be looked at. The name shares the one declaration namespace with functions, classes, interfaces, type aliases and module constants: a clash is `Kind` is already declared in this module whichever came first (reject_enum_duplicate).

  • An exported enum is imported, and in the importer it is the exporter's enum. export enum Kind { ... } in kinds.ts and import { Kind } from "./kinds" elsewhere bind one type, not a copy: the importer may use Kind as a field, parameter or return type, a local, a Map key and a switch discriminant, Kind.If folds to the integer kinds.ts gave it, and a Kind crosses a call between the two modules as itself (tests/link/enum_export_uses). One enum reached along two import paths is one type (enum_export_diamond), two modules may import each other's enums (enum_export_cycle), and unlike a class an enum may be renamed on import — import { Kind as Level } — because it has no symbol for its name to be part of (enum_export_rename). Nothing is emitted in either module, so the importer's IR is byte-identical to the same program with the enum declared in it (enum_export_ir and enum_export_ir_local, whose .ll goldens are the same files).

    • Every enum declaration is a type of its own. A Kind a module declares and the Kind it could have imported are two types, so a member of one is not an argument where the other is declared: Argument 1 of `weight`: expected Kind, got Kind (tests/link/enum_import_distinct). An imported enum converts to and from i32 no more than a local one does (enum_import_i32, enum_import_from_i32).
    • The import errors are the ones every import has. An enum declared without export is `Kind` is declared in `./kinds` but not exported (add `export`) (tests/link/enum_import_not_exported); a declaration of the imported name is `Kind` is already declared in this module (enum_import_clash, enum_import_clash_function); a second import of it is `Kind` is already imported from `./kinds` (enum_import_duplicate, enum_import_duplicate_function); and an enum is not a function, imported or not: Kind(2) is Unknown function `Kind` (enum_import_as_call). A module does not re-export an enum it imported.
    • Why it is bound before the signatures. An enum is a type with no layout, so a signature that names an imported one has to find it declared, where an imported class can be resolved provisionally and bound later. Every module of the program therefore declares its names and binds its imported enums before any module collects a signature (load in src/compilation.ts).
  • const enum and declare enum are refused. const enum asks for the inlining every member already gets — `const enum` is not supported: an enum member is already folded to its integer, so `const` would ask for nothing (tests/cases/reject_enum_const_enum) — and `declare enum` is not supported is the ambient form, which has nothing to declare against (tests/cases/reject_enum_declare; declare const enum too, since declare is written first).

  • An enum does not cross to C or JavaScript. --emit-header, --emit-dts and --emit-napi have no spelling for one, so a function whose signature mentions a numeric enum is skipped with the note those generators write for every type they cannot carry (weight(k: Kind): number: not declared; no C spelling for one of its types.), and one whose signature holds a string enum is refused (NL4019, String enums). An enum is a type inside one module, as it is across a module boundary: give a host an i32 or a string and keep the enum behind it.

  • Enums are top-level only. enum K { A = 1 } inside a function body is Unsupported statement in Phase 1: EnumDeclaration (CLI only).

String enums

An enum whose members are each given a string literal is a string enum. It is the spelling Nish takes for a closed set of strings: a union of string literals, "admin" | "guest", is refused (NL1053, and a lone string literal type is NL2668), because a value has one fixed layout and a string enum's is an i32.

enum Role {
  Admin = "admin",
  Guest = "guest",
  Nobody = "",
}

const rank = (r: Role): i32 => {
  switch (r) {
    case Role.Admin:
      return 3;
    case Role.Guest:
      return 1;
    default:
      return 0;
  }
};

const name = (r: Role): string => String(r); // name(Role.Admin) is "admin"
  • A value is its member's ordinal. A string enum is a distinct type, as a numeric one is, stored as an i32: the member's position in declaration order, 0, 1, 2, and never the bytes of its string. Every rule of Enums above holds for it as it does for a numeric enum: === and !== between two of its values are the only operators, and they compare ordinals; a switch over one lowers to LLVM's switch on the ordinal; a member is also a type (kind: Role.Admin), and so tags a variant of a discriminated union (tests/cases/enum_string_union_tag); a value is a Map or Set key, an element, a field and a Result arm; it crosses a thread, a channel and a task as the i32 it is; and it is exported and imported as a numeric enum is (tests/cases/enum_string, tests/link/enum_string_export). It is never | null (NL2010), as no enum is.

  • Its strings are one table per module. Each module that names the enum's strings holds them as @.enumstr.<Name>, an array of the module's string constants indexed by ordinal; an importer gets its own copy of the table, so nothing is looked up across a module boundary (tests/link/enum_string_export). Nothing else is emitted: Role.Guest folds to 1 as Kind.While folds to 2.

    @.enumstr.Role = private unnamed_addr constant [3 x i8*] [i8* bitcast ({ i64, [6 x i8] }* @.str.0 to i8*), i8* bitcast ({ i64, [6 x i8] }* @.str.1 to i8*), i8* bitcast ({ i64, [1 x i8] }* @.str.2 to i8*)], align 8
    
    define internal noundef i32 @rank(i32 noundef %r) #0 {
    entry:
      switch i32 %r, label %sw.default [
        i32 0, label %sw.case
        i32 1, label %sw.case.1
      ]
    
  • String(r) is the member's string. It loads the table's entry at the ordinal, so it allocates nothing and its answer is the module's constant:

    %19 = sext i32 %18 to i64
    %20 = getelementptr inbounds [3 x i8*], [3 x i8*]* @.enumstr.Role, i64 0, i64 %19
    %21 = load i8*, i8** %20, align 8
    call void @nish_print(i8* %21)
    

    String takes a string enum's value and nothing else; on any other argument it is Unknown function `String` (NL2247, reject_string_call_non_enum).

  • A member's value is a string literal, written once. Every member of a string enum has an initializer: String enum member `Role.Guest` has no initializer: every member of an enum whose members are strings is given its own string literal (NL1064, tests/cases/reject_enum_string_missing). A numeric member beside a string one is NL1043 (reject_enum_string_mixed, and in typed JSON json_reject_enum_mixed), and a template literal is An enum member's value cannot be a template literal: write it as a plain string literal, which is fixed when the enum is declared (NL1065, reject_enum_string_template); a computed value, "gu" + "est", is A string enum member's value must be a string literal: a computed value is not supported, because the enum's strings are a table fixed when it is declared (NL1066, reject_enum_string_computed). The value is valid UTF-8, since it is written to and matched against JSON: a lone surrogate, "\uD800", is Enum member `Role.Bad` is not valid UTF-8: a string enum's values are written to and matched against JSON, which is UTF-8 (NL2787, reject_enum_string_utf8). The empty string is a value like any other (tests/cases/enum_string).

  • Two members do not share a string. Enum member `Role.Root` repeats the string of `Role.Admin`: each member of a string enum has its own string, because the string is how JSON and `String` name the member (NL2786, tests/cases/reject_enum_string_same_value). A numeric enum allows two members of one value, which are one type; a string enum does not, because its string is how the member is named outside the program: a decoded "admin" must name one member, and String must answer one string per member. tsc would also read two members of one string as one value, where Nish's ordinals would make them two.

  • A Role is not a string, in either direction. A string is never compared with one: Cannot compare a `Role` with a string using `===`: an enum value is not its string, so compare it with a member, such as `Role.Admin` (NL2788, tests/cases/reject_enum_string_compare_literal). A Role where a string is wanted, const s: string = r or ${r} in a template, is A `Role` is not a string: write `String(r)` for the member's string (NL2789, reject_enum_string_to_string, reject_enum_string_template_use), and a string where a Role is wanted, const r: Role = "admin", is `"admin"` is a string, and a `Role` is not one: write the member itself, such as `Role.Admin` (NL2790, reject_enum_string_from_string). Arithmetic is refused as on any enum, r + 1 being Operator `+` requires two operands of the same numeric type or two strings, got Role and i32 (NL2231, reject_enum_string_arith), and const enum is NL2285 as above.

  • Typed JSON reads and writes its string. A string-enum field, array, | null array or Map value is the member's string on the wire, and a union tagged by a string enum's member writes that string as its tag (Typed JSON).

  • Under --emit-ts it is a const object. enum Role { Admin = "admin" } is written as

    export const Role = { Admin: "admin", Guest: "guest" } as const;
    export type Role = (typeof Role)[keyof typeof Role];
    

    so each member is the string tsc gives it, String(r) is the identity and prints what the native program reads from its table, and a member type is typeof Role.Admin (tests/emit-ts/control/string-enums). tsc would accept r === "admin", const s: string = r and const r: Role = "admin" on that output; Nish refuses all three, so it is the stricter of the two, and every program it accepts is one tsc accepts too.

  • A string enum does not cross to C, N-API or wasm yet. An exported function whose parameter or result is a string enum, or an array, | null or Result that holds one, is refused under --emit-header, --emit-dts, --emit-napi and --emit-napi-async: `rank`'s parameter `r` is `Role`, which holds a string enum: a string enum has no C, N-API or wasm spelling in this version, so take or return a `string` and convert with `String(r)` and a `switch` (NL4019, tests/cases/reject_enum_string_export, reject_enum_string_export_napi). It is an ordinal natively and a string in TypeScript, and a sidecar cannot yet say which a host passes.

  • Open questions. A Role.parse(s) that answers the member a string names is not built: today a program tests s === String(Role.Admin) for each member, or decodes the string with typed JSON; the C, N-API and wasm spelling of a string enum is open (NL4019); and a union of string literals stays refused, string enums being the spelling Nish takes instead.

export and import

  • export is accepted on function, class, interface, enum, type and const declarations (tests/cases/export_fn, tests/link/class_export, tests/link/enum_export_uses, tests/link/alias_export_class, tests/link/const_export). Every other export form is refused by its rule, once: export { a }, with or without from, export * from, export * as ns from and export type { A } are `export { ... }` / `export * from` are not supported; put `export` on the function declaration itself (tests/cases/reject_export_list); export default <value> and export = are `export default` / `export =` are not supported; use a named `export function` (reject_export_assignment); export default function f is `export default` is not supported; use a named `export function` (reject_export_default), and export default class C or export default interface I `export default` is not supported; use a named `export` (reject_cls_export_default, reject_generic_export_default); an anonymous one is Functions must be named or Classes must be named. export import is Only functions can be exported for now, plus classes and interfaces (found `export` on ImportEqualsDeclaration) (or ImportDeclaration; reject_export_import_equals), and export as namespace X is Only top-level function declarations are supported in Phase 1 (found NamespaceExportDeclaration). Several in one module report one each (reject_module_sweep_together), unless one is a Phase 0 form below, which is then reported alone.

  • What export promises depends on the build. In every build it makes a declaration importable by another module of the program. A build whose output something else links or loads — -o alone (IR for a C host to link), --emit-header, --emit-dts, --emit-napi, --emit-napi-async, --profile wasi or a wasm --target — also makes an exported function, and every method of an exported class, callable by that host with any arguments, so the compiler assumes nothing about them from the program's own calls (tests/cases/arr_range_call_exported, where a C host's pick(7) panics). A --link build with none of those flags and no declare function is its own final link: the executable holds the program's modules and the runtime and nothing else, so every call to an exported function is one the compiler sees, and it may, for example, prove an exported method's index in range from what every caller passes (tests/link/range_export, AWFY Permute as an exported class, whose swap carries no bounds check; tests/link/range_export_unproven, where one caller proves nothing, so the check stays and panics). The .ll such a build writes beside the executable is that executable's, not an object for another link; build with -o alone, or with a sidecar, to hand a host the program. main is the one exception: the runtime calls it, and it is never assumed anything about. The rule is hostVisible in src/visibility.ts, and docs/ARCHITECTURE.md (the call-site ranges row of the attribute soundness rules) is why it is sound.

  • An exported function is not named after a C library function. A module of the program's own package has no symbol prefix, so an exported function's symbol is its name, in the entry module and in every other module of the package, and under --no-strict-exports every function's is. Named write, read, free, exit, sin or another function the C library defines for the runtime or for the program's own arithmetic, it would be the definition those references reach in the linked program (console.log printing nothing, a crash at the first file read, Math.sin answering what the program's sin returns), so it is `write` is the C library's `write`: an exported function of the program's own package keeps its name as its symbol, so the references the C library and the runtime make to that name would reach this function instead; rename it (tests/cases/reject_export_c_symbol_const, reject_export_c_symbol_function, reject_export_c_symbol_libm, reject_export_c_symbol_libm_f32, tests/link/export_c_symbol_sibling, export_c_symbol_sibling_function; with --no-strict-exports, reject_export_c_symbol_no_strict and reject_export_c_symbol_no_strict_exported). A name that begins with _, which C reserves for its library and start-up code (C11 7.1.3), or with napi_ or node_api_, which Node-API reserves, is refused the same way with begins with `_`, which C reserves for its library and start-up code or begins with `napi_`, which Node-API reserves for its functions (or node_api_) (reject_export_c_symbol_reserved_prefix, which pins __libc_start_main and __register_atfork, each of which crashed; reject_export_c_symbol_napi_prefix). The rule is the same in a wasm or an N-API build, where the exported name is the one JavaScript calls: rename the function there too. The names are reserved on every target, whichever target each one binds on: a name only wasi-libc defines (strdup, unlink) is refused in a native build as well, and no flag lifts the rule. The names and where each comes from are in src/c-symbols.ts: the symbols the runtime leaves undefined on every platform it builds for, the libm calls behind every Math builtin and % on a float in double and in float, with the ones LLVM rewrites them into, the memcpy family, the rest of the C allocator, and, because --profile wasi links wasi-libc statically, every name a wasi-libc member that link pulls in defines or references (an exported sbrk trapped at start), and every name LLVM's TargetLibraryInfo recognises as a library function on the targets the compiler emits for (abs, atan, rename, qsort: LLVM folds or rewrites a call to one of them whatever its definition, so abs(-3) answered 3; reject_export_c_symbol_tli), and every function C23's <math.h> and C99's <complex.h> declare, in double, float and long double (tgamma, erff, ilogbl, cexp), whether or not the LLVM in use knows it, because a later LLVM than CI's recognises more of them (#651). tests/run.js reads them back with llvm-nm from the runtime's objects, runtime-wasm.c, the N-API shim, and two programs — one that calls every Math builtin, one that multiplies i64s, copies arrays and prints strings — linked at three optimisation levels and compiled for wasm32, the wasi-libc members a traced --profile wasi link of each pulls in, and fails on a name the file does not reserve ("c-symbols: ..."); it reads the TargetLibraryInfo names with llvm-tli-checker for five target triples. A function whose symbol is not its ABI is spelled apart instead of refused: an internal one whose name is reserved takes a $ at the end of its symbol (@floor$), because LLVM folds a call to a function it knows by name as the C library's, and a private floor called as floor(1.5) answered 1 (tests/link/c_symbol_private_libm), and a hidden one, which another module's instantiation calls, takes it whatever its name (#610, under function parameters; tests/link/thread_no_strict_hidden_task, under --no-strict-exports). A method, whose symbol carries its class, and a function of another package, whose symbol carries the package, may have any of these names (tests/link/export_c_symbol_allowed). main is the entry's alone (tests/link/main_in_import), and emitted as @nish_main.

  • A wasm module's function is not named memory. wasm-ld exports the module's linear memory as memory, and scripts/build.sh --profile wasm links with --export-all (--profile wasm-host with an export list of the module's exported functions and memory), so in that module an exported function of that name, or any function under --no-strict-exports, is a second export of it, and Node refuses the module ("Duplicate export name 'memory'"). The wasi command module exports only memory and _start, so the function does not collide there; the name is reserved there too, so that a program that builds for wasi also builds as a freestanding wasm module. On a wasm --target, under --profile wasi, and in an --emit-dts compile, whose loader is for the module scripts/build.sh --profile wasm or wasm-host links its IR into, it is `memory` is the name a wasm module exports its linear memory under: an exported function of the program's own package keeps its name as its symbol, so the module would export two things named `memory` and no host could load it; rename it (NL3038; tests/cases/reject_export_wasm_memory, reject_export_wasm_memory_wasi, reject_export_wasm_memory_dts, tests/link/export_wasm_memory_sibling; with --no-strict-exports, reject_export_wasm_memory_no_strict; without the flag a memory nobody exported stays internal and compiles, "wasm memory: a --target wasm32 compile keeps a memory nobody exported" in tests/run.js). IR compiled for the host with -o alone reaches build.sh without the compiler knowing the build is wasm, so build.sh --profile wasm, wasm-host and wasi refuse IR that defines an exported @memory with exit 2 ("wasm memory: ..." in tests/run.js). A native build has no linear memory to collide with and keeps the name (export_memory_native, tests/link/export_memory_native_sibling). The linker's other names, _start, __heap_base, __data_end and __indirect_function_table, are already refused by the rule above. A wasm program with module storage reserves two more: its loader answers the storage as layout and views beside the exports (§5.2 of docs/wp50-wasm-host.md), so in such a program an exported function of either name is `views` is the name the wasm loader answers the program's module storage under: an exported function of the program's own package keeps its name as its symbol, so the loader would answer two things named `views` and the page could not reach this function; rename it (NL3051, tests/cases/reject_storage_loader_name). A program without module storage keeps both names (export_loader_names_no_storage).

  • The only import form is a named import: import { square, cube as pow3 } from "./math" (tests/link/two_file). The specifier must start with ./ or ../, name one of the three builtin modules below, name a standard-library module as nish/<module>, or be a package name — hash or @scope/hash, each of which may be followed by a subpath (@scope/hash/blake3), resolved through node_modules and the nish export condition (tests/link/package_bare; a package that is not installed is Cannot find package `hash`, tests/cases/reject_bare_package). Anything else is Import specifier ... must be relative (tests/cases/reject_bare_import). .ts is optional and ./x.js maps to ./x.ts; a relative path resolves relative to the importing file. Every other import form is refused by its rule, once, before anything is loaded: a default import, import d from "./m", import d, { a } from "./m" or import { default as d } from "./m", is Default imports are not supported; use `import { d } from "./m"` (reject_default_import); a namespace import is Namespace imports (`import * as ns`) are not supported; import functions by name (reject_namespace_import); a side-effect import is Side-effect imports (`import "./m"`) are not supported; modules have no top-level code (reject_side_effect_import); import type, and type in front of one name, is Type-only imports are not supported (reject_import_type_only); a specifier that is not a string literal is Import specifier must be a string literal (reject_import_template_specifier); and import x = require("./m") or import x = A.B is Only top-level function declarations are supported in Phase 1 (found ImportEqualsDeclaration). Three are refused by Phase 0, because no later release could give them a meaning: import attributes (with { type: "json" }, assert { ... }, reject_import_attributes), a module export name written as a string (import { "a-b" as b }, reject_import_string_name) and import defer (reject_import_defer) — see Forbidden constructs. type, as, from, defer, require and assert are not reserved, and are the modifier only where TypeScript reads one: import type from "./m" is a default import of a binding called type, and { type }, { type as t } and { type as as } each import a function called type (tests/parser/names-modules.ts). A missing file is Cannot find module `./does_not_exist` (reject_missing_module).

  • A file is one module however it is named. The command line and the imports may spell one file differently — ./types.ts, its absolute path, a path through a linked directory, and the types.ts an import from "./types" resolves to — and it is loaded once, under the first spelling that reached it, so its declarations never clash with themselves and its .ll is written once (tests/link/root_named_twice, tests/link/identity_symlink_cwd). A module's identity is its real path: the file the operating system opens under a spelling, with every symbolic link resolved, as Node and tsc (preserveSymlinks: false) identify one. A command-line path is taken as the operating system reads it, so far/../types.ts with far a link into other/ is other/types.ts, a second module beside ./types.ts (tests/link/identity_symlink_dotdot); an import specifier is joined to its importer's name, as a relative URL is. What a module is called — its IR header, its diagnostics, its output file — is still the first spelling, so no emitted byte depends on where the links point.

  • Two modules never share an output file. -o <dir>/ names each .ll after its module's basename, after its path relative to the entry when two modules share a basename, and — when those still meet, as types.ts and far/../types.ts do — a later module takes <stem>_2.ll, <stem>_3.ll and on, in load order (tests/nish/cli.ts).

  • The standard library is imported as nish/<module> (tests/link/std_bare_specifier). It resolves to std/<module>.ts beside the running compiler rather than relative to the importing file, so the same specifier works at any depth. A module the library does not have is Module `nish/toml` is not part of the standard library (it has: collections, compress/checksum, compress/deflate, compress/gzip, compress/zlib, crypto/aes, crypto/base64url, crypto/chacha20poly1305, crypto/ct, crypto/ed25519, crypto/hkdf, crypto/hmac, crypto/p256, crypto/pki, crypto/pki-roots, crypto/rsa, crypto/sha256, crypto/sha512, crypto/x25519, crypto/x509, json, map, pair, testing, text, threads) (reject_std_unknown_module), and one that would leave the library — an empty segment, or a segment beginning with a . — is refused rather than resolved: Module `nish/../../escape/lib` climbs out of the standard library; a `nish/` specifier names a module inside it, so no segment may be empty or begin with a `.` (reject_std_escaping_module). The module is named by that specifier, not by where the library is installed: nish/text carries std/text.ts in its ; ModuleID, its source_filename, its DIFile and — with no -o — the path its .ll is written to, whichever directory the compiler sits in and however the compiler was invoked (tests/link/std_bare_specifier). Two installs of one compiler therefore emit the same bytes for the same program, which is the reason the specifier is refused when it could not be a name. Unlike a nish: builtin this is ordinary Nish source: it is compiled into the program that imports it and is subject to every rule in this document. What it costs is what you call — speed and size link with --gc-sections, and a module imported but never called is byte-for-byte free (std/README.md). Its symbols are scoped to package nish, so a program may declare a function a std/ module also exports (tests/link/std_package_scope); the package is decided by the specifier rather than by where the file sits, because reading it off the path would answer differently for an installed compiler and a checkout of the same version.

  • nish/pair is Pair<A, B>, two values answered by one call. It is interface Pair<A, B> { first: A; second: B; } and nothing else: a generic interface, so an object literal takes its type from the annotation and must set both fields, and there is no tuple syntax and no constructor behind it. It crosses a module boundary like any exported interface, over scalars (tests/link/std_pair_scalar), a string and an f64 under --number-mode f64 (std_pair_f64), an array (std_pair_array), another Pair (std_pair_nested), and held in a class field and an array (std_pair_held). The instantiations are ordered — a Pair<i32, string> returned where a Pair<string, i32> is declared is Return type mismatch: function returns Pair<string, i32> but expression is Pair<i32, string> (reject_std_pair_swapped) — and a literal that leaves a field out is Object literal for `Pair<i32, boolean>` is missing field `second` (reject_std_pair_missing). It lowers as any generic interface does (gen_interface): one struct per instantiation, %struct.Pair$i32$bool, returned by pointer. Use it to return two values, not to store them side by side; docs/wp23-language-surface.md §5 is the reasoning.

  • nish/crypto/<primitive> is a module in a subdirectory of the library, one per primitive: sha256, sha512 (SHA-512 and SHA-384), hmac, hkdf, ct (constant-time compare), base64url, x25519, ed25519 (Ed25519 signatures), chacha20poly1305 (ChaCha20-Poly1305), aes (AES-128 and AES-256 with GCM), p256 (P-256 ECDSA), p384 (P-384 ECDSA verification), rsa (RSA signature verification), x509 (DER, PEM and self-signed P-256 certificates), pki-roots (the trust anchors from SSL_CERT_FILE, SSL_CERT_DIR or the system bundle; native only) and pki (RFC 5280 path validation and RFC 6125 hostname matching of a TLS server's chain) (tests/link/crypto_base64url is the case that resolves a nested specifier). Each is ordinary Nish source under the rules above, and a byte string is a u8[]. They are written branch-free on secret data, and WP34 N6's disassembly check (tests/ct-asm.js) verifies that of the functions its ct_asm_* fixtures hold, on x86-64 and aarch64; the rest is discipline. std/README.md lists what each module exports, the specification it reproduces, which of its functions are verified, and the rules they share.

  • A package is imported by name, and what is resolved is its source. import { scale } from "pkg_bare" looks for node_modules/pkg_bare in the importing file's directory and in every directory above it — above the directory the compiler was run in included, which is the ordinary npm layout (tests/link/package_above) — stepping over an ancestor already called node_modules wherever the module's own name spells one, as Node does (tests/link/package_doubled) — reads that package's package.json, and compiles the file its exports map offers for the nish condition (tests/link/package_bare, docs/wp21-packages.md §2):

    { "exports": { ".": { "nish": "./src/index.ts" }, "./util": { "nish": "./src/util.ts" } } }
    

    Source, not a built artifact, and that is the decision rather than a stage: the whole-program fact fixpoint, monomorphisation, and --number-mode being a compile-time ABI each make a prebuilt library strictly worse than the source that is already there (docs/wp21-packages.md §3). The consequence a reader should carry away is that a dependency is compiled, checked and optimised as if it were your own code, and its non-exported functions are internal and dead-stripped like your own.

    • The number mode rides in the condition. nish-i32 and nish-f64 are the mode-qualified spellings, and the compiler asks for its own mode first and then plain nish — so a package correct under either declares nish, a package that needs different source per mode declares both, and a mismatch is a resolution failure at the package boundary rather than a wrong answer at run time. "First" is the compiler's order and not the manifest's: the mode-qualified condition wins wherever the package declares it, unlike Node, which would take whichever of the two the manifest names first (tests/link/package_mode_order, tests/link/package_mode_order_f64). Node's order is right for rival conditions and wrong for these two, because nish-f64 is not a rival of nish but nish refined by the mode: honouring the order would let a package write nish above nish-f64 and never have its f64 source compiled, with nothing said about it — the silent mismatch this spelling exists to make loud.

    • Presence of the condition is the claim, and every failure to find it names its cause (docs/wp21-packages.md §6, §11). Each is refused at the import, before a byte of the package is checked, with its own --json code:

      • A package whose entry offers no Nish condition in any spelling — an ordinary npm package, with import and require rows or a bare ".": "./index.js" — is Package `plainjs` has no Nish entry point: its `exports` gave this compiler no file to compile for `.`, because that entry declares none of the conditions this compiler compiles source from (`nish`, `nish-i32`, `nish-f64`) (NL3020, tests/link/package_not_nish, tests/link/package_no_condition).
      • A package that offers only the other mode's condition is Package `pkg_f64` supports Nish in f64 mode only: its `exports` offers `nish-f64` for `.` and neither `nish-i32` nor `nish`, and this program is compiling in i32 (`--number-mode i32`) (NL3017, tests/link/package_other_mode, and the reverse in tests/link/package_other_mode_f64).
      • A package.json that is not well-formed JSON, when no entry point could be read out of it, names the manifest and the line and column where it breaks: Package `pkg_broken` has a malformed manifest: …/package.json:4:3 is not well-formed JSON (NL3021, tests/link/package_malformed). One the reader got a file out of before the break compiles, as it always has.
      • Anything else — no exports at all, no key for the subpath asked for, or a value the reader does not follow — is Package `pkg_one` has no Nish entry point: its `exports` gave this compiler no file to compile for `./extra` (NL3014, tests/link/package_no_subpath). The sentence reports what the compiler came away with rather than what the manifest declares, because a nish-f64 that names something other than a file outranks a nish that is there and correct, and is what the message is about.

      A package that is not installed at all is Cannot find package `@x/y` (tests/cases/reject_bare_package).

    • A compiler floor goes in engines.nish, npm's slot for it, and it is checked whether or not the entry point resolves. The one range read is a floor, >=X.Y.Z or >=X.Y, blanks allowed around the version; a package whose floor is above this compiler is Package `pkg_future` needs a newer compiler: its `engines.nish` asks for `>=999.0.0` and this is nish 0.8.0 (NL3018, tests/link/package_engines_floor), and one met compiles (tests/link/package_engines_met). Any other shape — a caret, a tilde, an upper bound, a bare version, a value that is not a string — is refused rather than taken as met, because a floor the compiler cannot read is one it cannot claim to meet: declares `engines.nish` as `^0.1.0`, which is not a range this compiler reads (NL3019, tests/link/package_engines_range).

    • A package is its real directory, and a program compiles one copy of it. One package reached both directly and through a symlink — pnpm's layout, and npm's whenever it cannot hoist — is one package, as it is to Node's resolver, and its own dependencies are looked for beside where it really is (tests/link/package_symlink). One package name at two real directories — npm's duplicate copies, or two versions of it under two dependencies — is refused at the import that reached the second, naming both directories and both versions: Package `hash` is at two places, …/node_modules/hash (1.0.0) and …/node_modules/user/node_modules/hash (2.0.0): Nish compiles one copy of a package per program, so every import of it has to reach the same directory (NL3029, tests/link/package_two_dirs, docs/wp21-packages.md §7; a copy whose manifest declares no version is said to have no version, tests/link/package_two_dirs_unversioned). A directory no link leads through keeps the spelling the search found it by. One a link leads through is named by its real directory, spelled relative to the working directory when the program was named relatively, and every module under it is in that package — so a workspace link to a directory outside any node_modules (node_modules/foo -> ../packages/foo) keeps foo's own relative imports in foo (tests/link/package_workspace).

    • Only the nish conditions are honoured, and a subpath is an exact key: default, import and node are skipped rather than matched, and the "./*" pattern form is not read. A target is a string beginning with ./ and naming a file inside the package.

    • A package's symbols carry its prefix (@pkg_bare.scale), so two packages may each keep a private helper(); the root package — the program being compiled — has no prefix and its IR is unchanged (docs/wp21-packages.md §9).

    • A diagnostic in a dependency says it is the dependency's. An error or warning spanned in a package's own source keeps its code and wording, adds = note: this is in the dependency `pkg_broken` 0.3.1, not in this program's own source under the excerpt (the version as its manifest gives it, left out when it gives none), and under --json carries "package":"pkg_broken" (tests/link/package_dependency_error). What decides is where the module was resolved, so a dependency that calls itself nish is still one, and the standard library never is.

    • A builtin with no runtime on the target is refused by what the program reaches, once every module is checked. A freestanding --target wasm32 build has no file, clock, environment, process, signal, terminal or entropy call (runtime/runtime-wasm.c), so a program whose main or any exported function of its own modules reaches one — readFileSync, getenv, spawnSync, monotonicNanos among the ones no call-site rule refuses — is refused with the N3 rule's words (NL2404), one error per capability, spanned at the first call of the witness chain, which names every hop; when the chain passes through a dependency the message names the one nearest the builtin: `getenv` reaches the operating system, and a wasm32 build has none to reach; the dependency `pkg_conf` calls it in `load` (tests/link/package_wasm_hostless, tests/link/wasm_hostless_own, reject_read_file_wasm). A module that calls one in a function the program never reaches is not refused: a wasm32 program may import nish/cli for Style (npm test "wasm hostless"). --target wasm32-wasi and --profile wasi have the calls, and refuse none of them.

  • An imported constant contributes no declare and no relocation: it is folded into every use site in the importing module exactly as it is in the exporting one (tests/link/const_export, tests/differential/corpus/const_modules).

  • Functions may be renamed on import (as); classes and interfaces may not, because the type name is part of the ABI (%struct.Point, @Point.constructor): Classes cannot be renamed on import (tests/link/class_import_rename).

  • Importing a name the module declares but does not export, or does not declare at all, is an error with distinct messages (tests/link/not_exported, tests/link/unknown_export); importing the same local name twice, or a name also declared locally, is an error (tests/link/duplicate_import).

  • An imported class or interface brings the layouts of the classes and interfaces its own members mention — field types, and the parameter and return types of its methods and constructor, through arrays and nullables — so a module that imports Registry may hold, call and read the Entry values Registry.all(): Entry[] hands it without importing Entry (tests/link/reachable_struct). Only the layout travels, not the name: a type annotation still needs the import, and const e: Entry in that module is Unsupported type reference `Entry` (tests/link/reachable_struct_annotation). An imported function brings its own signature's layouts the same way — import { lex } where lex(): Token gives this module Token values it never names (tests/link/reachable_struct_return) — and the layouts travel however many modules apart the declaration is (tests/link/reachable_struct_chain).

  • Import cycles are allowed (tests/link/cycle); a shared dependency is compiled once (tests/link/diamond).

  • Linkage. An exported function is an external C-ABI symbol; every other function gets internal linkage by default, so LLVM may inline, specialise or drop it (tests/cases/export_fn, tests/link/strict). --no-strict-exports makes every function external again, which is what a C driver calling a non-exported function needs (tests/cases/export_no_strict).

    • A method another module calls is in the link. The layouts an import brings (above) include a class its module does not export — Conn.space(): Space hands an importer of Conn a Space whether or not Space is exported — and the importer may call that value's methods, generic ones included. Such a method (or instantiation of one) is hidden rather than internal: one the final link resolves and the output exports to nothing, with the Result ABI an exported one has. A method only its own module calls stays internal (tests/link/package_hidden_method, #442). The class's name stays its module's: importing it is `Space` is declared in `pkg_q` but not exported (add `export`) (tests/link/package_hidden_class_import), and naming it in an annotation is Unsupported type reference `Space` (tests/link/package_hidden_class_annotation).
  • A function name is unique across the whole program, exported or not, in either mode (tests/link/duplicate_export, duplicate_internal). internal linkage keeps a name out of the linker's way but it does not buy a second namespace: the whole-program attribute analysis is keyed by symbol name, so two functions sharing one would be emitted with each other's attributes — a miscompile rather than a link error, which is why the rule does not depend on the flag. A proposal to qualify a private function's symbol by its module when it would clash is WP57; the rule above holds until it ships.

main

export const main = (): number => {   // or `: void`; `: i32` in f64 mode
  console.log("hello");
  return 0;                             // the process exit code
};
  • Only the entry module (the first file on the command line) may declare export const main (Only the entry module may declare, tests/link/main_in_import); --link requires it (tests/link/no_main), unless the entry holds top-level code, which becomes it.
  • It takes no parameters (`main` cannot take parameters, tests/cases/reject_main_params; the command line is process.argv) and returns void (exit code 0, tests/cases/entry_main_void) or an i32-lowered number (tests/cases/entry_main, tests/link/two_file). In f64 mode declare main(): i32; main(): number is rejected with `main` must return void or an i32 number ... under --number-mode f64 declare `main(): i32` (CLI only).
  • The user's function is emitted as @nish_main; the compiler adds a C @main(i32 %argc, i8** %argv) wrapper that builds process.argv when the program reads it (call void @nish_argv_init(i32 %argc, i8** %argv), tests/cases/argv_echo, tests/link/argv_import), calls it, frees the arena, and returns the code. A non-exported function main is an ordinary function named @main (for C drivers such as tests/driver.c).

Classes

class Point {
  x: number;
  y: number;
  label: string = "p";          // literal initializer

  constructor(x: number, y: number) {
    this.x = x;
    this.y = y;
  }

  manhattan(): number {
    return this.x + this.y;
  }
}
  • Fields need a type annotation (Field `x` of class `Point` needs a type annotation, tests/cases/reject_cls_field_no_type). An initializer must be a literal (a number, a negated number, a string, or a boolean) of the field's type (initializers must be literals (CLI only); tests/cases/cls_initializers). Layout is declaration order with natural alignment and padding, exactly as clang lays out the same C struct (tests/layout/structs.ts).
  • Constructor: at most one, with a body (Constructor of class `Point` must have a body (no overload signatures), tests/cases/reject_cls_ctor_no_body), no return type annotation (Constructors cannot declare a return type, reject_cls_ctor_return_type), and no parameter properties (Parameter properties (`constructor(public x: number)`) are not supported; declare the field and assign it, reject_cls_param_property; public, private, protected, readonly and override are a parameter's modifier only before its name on their line, so constructor(public: i32) is a parameter called public, tests/parser/names-classes.ts). new C(args) takes exactly the constructor's arity (`new Point` expects 2 argument(s), got 1, tests/cases/reject_cls_ctor_arity). A class without a constructor may be newed with zero arguments and gets its initializers stored inline (tests/cases/cls_initializers).
  • Definite assignment (tests/cases/reject_cls_uninitialised, reject_cls_uninitialised_no_ctor, reject_cls_read_before_assign): after the constructor returns every field holds a value. The check is syntactic, top to bottom: this.f = e; statements (and chains) mark f; an if marks what both arms mark; assignments inside loops, nested expressions, ternaries or calls do not count; every return must see every field assigned; reading this.f, calling this.m(), or using this as a value before every field is assigned is an error. A class with no constructor must initialise every field (has no initializer and no constructor assigns it).
  • Methods need a body (Method `scale` of class `Point` must have a body, tests/cases/reject_cls_method_no_body: a signature without one is an overload, and there are none) and an explicit return type; they are ordinary functions @Class.method with this first. this is valid only inside a method or constructor (`this` is only valid inside a method or constructor, tests/cases/reject_cls_this_outside) and may be aliased or passed on (const self = this (CLI only); tests/cases/cls_this_method_call). Methods may call each other and free functions, recursively.
  • Field access: p.x reads, p.x = v and p.x op= v write, for the arithmetic and the bitwise compound operators alike (tests/cases/cls_field_write, cls_compound_field, cls_field_bitwise_assign); ++/-- on fields is not supported (Only simple variables can be assigned). Unknown fields: Unknown field `z` on class `Point` (tests/cases/reject_cls_unknown_field, reject_cls_assign_unknown_field).
  • readonly fields may be assigned only as this.f = v in their own class's constructor (tests/cases/cls_readonly_ok; Cannot assign to readonly field `value` of `Id` outside its constructor, reject_cls_readonly_write). A compound assignment is a write like any other, so this.f |= bit is refused too, in the declaring class's own constructor as much as anywhere else (reject_cls_field_bitwise_readonly). public, private, protected are accepted and ignored.
  • Equality: === / !== on two values of the same class compare identity (pointer equality) (tests/cases/cls_this_method_call, Account.same); < and friends are rejected (tests/cases/reject_cls_ordering). Both operands must have the same declared type.
  • No inheritance (WP25). extends on a class is `extends` is not supported: Nish has no inheritance (tests/cases/reject_cls_extends), and super in every spelling — super(...), super.m(), a bare super — is `super` is not supported: Nish has no inheritance (reject_cls_super). A class is never a subtype of another class, so every method call in a program names exactly one symbol and there is nothing for a vtable to decide. The widening extends used to provide is implements, below: repeat the fields you were going to inherit as the class's first fields and name an interface that declares them. Shared behaviour is a free function over that interface.
  • Rejected: static on a field, a method or the constructor (tests/cases/reject_cls_static, reject_cls_method_static, reject_cls_ctor_static) — the word is still a legal member name, and static?: i32 is an optional field called static (reject_cls_field_named_static), and on an interface field too, where the message says which kind it is (reject_cls_static_interface), with one exception: a class's typed JSON options, static readonly json = { ... }, which is not a field and is written without a type (Typed JSON) — readonly on a method or the constructor, which has no storage to be read-only (reject_cls_method_readonly, reject_cls_ctor_readonly), getters/setters (Getters and setters are not supported in class `Box` (use a method), reject_cls_accessor; get and set are still member names, tests/parser/names-bindings.ts), optional fields (cannot be optional), index signatures (Index signatures are not supported in class `Point` (object layout is fixed), reject_cls_index_signature), ! assertions, abstract on the class or a member (Abstract classes are not supported, reject_generic_abstract), a declare class that is not a C struct layout (a method, accessor, constructor or initializer in one is refused as the declare class section lists, reject_cls_declare, reject_ffi_struct_*), a class with no name — export default class { } or a class expression — (Classes must be named, reject_cls_anonymous; a named class expression is Unsupported expression in Phase 1: ClassExpression), a member named by a string or a number (Members of `Point` must have plain identifier names, reject_cls_field_string_name), a static { } block (Unsupported class member in `Point`: ClassStaticBlockDeclaration, reject_cls_static_block), type parameters on the constructor (reject_generic_method_constructor; a method may declare its own — see Generic methods), decorators (reject_decorator), computed member names (Computed property names are forbidden in Nish (member names are fixed at compile time), reject_cls_computed_field, reject_cls_computed_method; [Symbol.dispose] is the one bracketed name read, see Scoped tasks), overloaded constructors.

Generic functions

A function may declare type parameters, and every instantiation is compiled to its own specialised function: one define per (template, type-argument tuple), named by appending $ and each argument's mangled type to the template's name. There is no boxing, no type dictionary and no runtime type information, and the IR of an instantiation is the IR of the monomorphic function somebody would have written by hand, with the symbol renamed (tests/cases/gen_identity).

const identity = <T>(x: T): T => x;

export const main = (): i32 => {
  console.log(identity(7));      // @identity$i32
  console.log(identity("hi"));   // @identity$str
  return 0;
};
  • Both spellings take type parameters. function identity<T>(x: T): T and const identity = <T>(x: T): T => x declare the same thing, as they do without them (tests/cases/gen_identity).
  • A type argument is inferred from the arguments, and is never written at a call site. identity<i32>(7) is Type arguments are not written at a call site in Nish: `T` is inferred from the arguments, so write `identity(...)` (tests/cases/reject_generic_call_type_args). One token of lookahead cannot tell f<i32>(x) from (f < i32) > (x), and inference already covers the spelling. Inference matches the shape of each declared parameter against its argument's type — T against i32, T[] against i32[], Result<T, string> against Result<i32, string> — left to right, first binding wins (tests/cases/gen_infer_two, gen_result_payload).
  • A type parameter that appears in no parameter is an error at the declaration, because it could never be inferred and the function could never be called: Cannot infer `T` for `empty`: a type parameter is inferred from the arguments, and `T` appears in none of them; give `empty` a parameter that mentions `T` (tests/cases/reject_generic_function).
  • Every other type rule applies to a type argument unchanged. A type parameter is not a hole through which something the language refuses can be smuggled in: Result<T, void> is still refused at an instantiation whose E is void, T | null is still only over a pointer type, and each instantiation's body is checked with T bound to a concrete type.
  • Each instantiation gets its own facts. The attribute fixpoint and the escape analysis are keyed by symbol, so eq<T> is readnone at T = i32 and readonly at T = string — one source line, two sets of attributes (tests/cases/gen_eq_purity).
  • Recursion terminates or is refused by name. A recursive call whose type argument is the parameter itself, or a type that does not mention it, is accepted: countDown<string> asking for countDown<i32> is two instantiations and no more (tests/cases/gen_recursive_ground). A call that puts the parameter under a type constructor has no end and is refused: Monomorphising `grow` would not terminate: `grow<i32>` asks for `grow<i32[]>`, which puts `i32` under a type constructor instead of passing it on, so the chain has no end; pass `T` itself, or a type that does not mention it (tests/cases/reject_generic_polymorphic_recursion). Behind the rule are two compiler limits — 256 instantiations of one template, 4096 in one module — whose message says it is a limit rather than a rule of the language.
  • $ may not appear in a declared name. It separates a generic's name from its type arguments in every symbol the compiler emits, so a function called identity$i32 would be the symbol identity<i32> already has: `identity$i32` cannot be the name of a function in Nish: `$` separates a generic's name from its type arguments in the symbols the compiler emits (tests/cases/reject_generic_dollar_name). A class or interface, generic or not, is held to it too: a class Box$i32 would be the struct Box<i32> already is, and one type with it wherever either is declared (#193; tests/link/class_dollar_name, …_imported). Locals, parameters and fields are unaffected — only names that become symbols are restricted.
  • An instantiation is spelled as written wherever a person reads it, and mangled wherever a machine does. Every diagnostic, the --emit-checked dump and the -g DWARF name say identity<i32>, identity<Box<i32>>, Box<i32> and Box<i32>.get; the symbol, the %struct name, the DWARF linkageName and a header's NISH_SYMBOL binding keep identity$i32, identity$$Box$i32 and Box$i32. Every instantiation's DISubprogram carries the template's line, so a breakpoint in a generic body has one address per instantiation, as a C++ template's does (tests/cases/dbg_generic). A polymorphic-recursion chain through a class reads `grow<i32>` asks for `grow<Box<i32>>` (tests/cases/reject_generic_recursion_through_class).
  • An exported instantiation reaches a host as nish_gen_<name>. Its symbol holds a $, and a . when an argument is an array, a nullable or a Result (identity$arr.i32), so neither C nor JavaScript can spell it. --emit-header declares nish_gen_identity_i32 and nish_gen_identity_arr_i32, bound to the symbols with NISH_SYMBOL, and --emit-dts, the wasm loader and --emit-napi export the same names; the comment above each says which instantiation it is. An instantiated class is struct nish_gen_Box_i32 with methods such as nish_gen_Box_i32_get. A user name cannot start with nish_, so no declaration of the program can take one of these names — a user identity_i32 beside identity<i32> is two C functions (tests/cases/interop_generic_fn, interop_generic_collision). The sidecars declare instantiations only: identity<T> itself has no symbol, so nothing declares it.
  • An exported generic that nothing instantiates is refused when a sidecar is asked for. It has no symbol, so a header, a .d.ts or an N-API shim would leave it out without a word. With any of --emit-header, --emit-dts, --emit-napi or --emit-napi-async it is `identity` is generic, so it has no single C signature: a generic function becomes a symbol only where it is instantiated, and this program instantiates none (NL4007), and for a class `Box` is generic, so it has no single C layout: a generic class becomes a struct only where it is instantiated, and this program instantiates none (tests/cases/reject_interop_generic_unused); without a sidecar flag the same program compiles. The same flags refuse two functions whose C names collapse to one, which only a method's .-joined name can still cause — Point.shifted beside a function Point_shifted: `Point.shifted` and `Point_shifted` are both `Point_shifted` in C, so a header that declared both would not compile: rename one of them (NL4008, tests/cases/reject_interop_generic_clash). They also refuse a signature or an exported struct field that holds a discriminated union, which has no C spelling yet (NL4009, tests/cases/reject_union_export_param), and a signature that holds a string enum, alone or in an array, a | null or a Result: `rank`'s parameter `r` is `Role`, which holds a string enum: a string enum has no C, N-API or wasm spelling in this version, so take or return a `string` and convert with `String(r)` and a `switch` (NL4019, tests/cases/reject_enum_string_export, reject_enum_string_export_napi).
  • A template may be exported and instantiated from another module, and each instantiation is defined exactly once, in the module that declares the template. Everywhere else it is a declare carrying that definition's attributes, which is what an imported plain function gets — the difference is that one imported name becomes one symbol per distinct type-argument tuple rather than one symbol (tests/link/generic_import). Two modules asking for the same tuple share the one definition (tests/link/generic_two_importers), and the symbol carries the declaring package's prefix, so a package that exports pick<T> owns @pkg_lib.pick$i32 however many packages call it (tests/link/package_generic_import). A type argument may be a class the declaring module has never heard of: the layout travels with the request, and the instantiation is emitted where the template is (identity<Point> in tests/link/generic_import). A template that is not exported cannot be imported, in the words every other private declaration is refused in (tests/link/generic_import_private), and an imported generic may be renamed — import { Box as B } — without moving the symbol, because the instantiation is named after the template rather than after whatever the caller called it.
  • A type-argument list belongs to a template and to nothing else. An imported name that is not one is refused when the import binds, rather than where the annotation is written, because a module's signatures are resolved as soon as it is parsed and before any import is bound: `Point` in `./lib` takes no type arguments: only a generic class or interface is written with them (tests/link/generic_import_plain).
  • A type parameter has no members until a constraint names them. A template may pass a T on, return it, store it, compare it and hand it to an operator that works at the concrete type, but it may not read, write or call a member through one, even when every instantiation it has would have that member: Cannot read `x` of `T`: an unconstrained type parameter has no members; say which ones it has with `<T extends Point>` (tests/cases/reject_generic_member_unconstrained), and the call form Cannot call `bump` of `T`: ... (reject_generic_member_unconstrained_method). A value is T-typed however it reached the receiver: a local inferred from a T, an element of a T[], a T | null narrowed to T, a field of type T read through this, and the result of a generic call that returns its argument (reject_generic_member_through_local); a field of new Box<T>(t), an element of [t] (reject_generic_member_through_expression); and a Result<T, E>'s value, pop() on a T[], an assignment's value and this copied into a local (reject_generic_member_through_payload). What decides it is where the value came from, never its type — at T = Point a T and a Point are one type, and only the T is refused (reject_generic_member_same_type_local).
  • <T extends Shape> constrains a type parameter, and then a T has exactly Shape's members: its fields, and its methods when Shape is a class. Each instantiation still reads its own struct directly — T at Circle reads field 0 of %struct.Circle and calls @Circle.bump, with no bitcast and no vtable (tests/cases/gen_constraint, gen_constraint_method). A field may be written through a T on the same terms (gen_constraint_write), and each parameter answers from its own constraint, so <T extends Shape, U extends Counter> reads area through a T and calls bump through a U (gen_constraint_two). A member the concrete type has and the constraint does not is refused, once for the template however many instantiations it has, or the constraint would be decoration: Unknown field `radius` on interface `Shape`, the constraint of `T`: a constrained type parameter has only the members its constraint declares (tests/cases/reject_generic_member_not_in_constraint).
  • A constraint is a declared class or interface, or an instantiation of a generic one written with concrete type arguments (T extends Container<i32>, tests/cases/gen_constraint_generic_bound). It is resolved once, in the template's own module, so a template imported from another module carries its constraint with it (tests/link/generic_constraint_import). Anything else is refused at the constraint — a scalar, string, an array or a Result: `T extends i32` is not supported: a constraint must be a declared class or interface, because the members a type parameter has are its constraint's — and so is one that mentions a type parameter, which would mean something different at every instantiation: `U extends T` is not supported: a constraint cannot mention a type parameter, ... (tests/cases/reject_generic_constraint).
  • A type argument must satisfy its constraint: it is the constraint itself, or a class that implements it. Having fields of the same names is not enough, because satisfying a constraint is a declaration, and a class constraint is satisfied by that class alone because a class can only implements an interface. It is checked at each request, where the argument was inferred or written, and reported in that module: `T` of `areaOf` requires `T extends Shape`, and `i32` does not implement it; pass a class or interface that declares `implements Shape` (tests/cases/reject_generic_unsatisfied_constraint, tests/link/generic_constraint_import_unsatisfied). The constraint is a declaration, not a name: a module that declares a Shape of its own has not satisfied another module's <T extends Shape>, whatever fields the two share — neither with that Shape itself nor with a class that implements it — and a same-named class does not satisfy a class constraint either. The request is refused in those words at the call in the asking module (tests/link/generic_constraint_same_name, …_same_name_fields, …_same_name_itself, …_same_name_class). The constraint may still be the asking module's own declaration when the template's module imports that very declaration (tests/link/generic_constraint_import_back). Two modules can no longer declare a Shape each at all (A class or interface name is program-wide, under Generic classes and interfaces), so the four programs above are refused at the second declaration now, before any call is checked; this rule is still what decides a constraint. Every position a type argument can be written in is a request of its own — a field, an array element, a Result arm, inside another instantiation, an implements clause (reject_generic_unsatisfied_constraint_positions).
  • A type parameter list names each parameter once, on a function, an arrow, a method, a class and an interface alike: Duplicate type parameter `T` (NL2302). The second T could never be bound to anything the first was not, so the list is refused on the first name that repeats, and only there: <T, T, T> is one diagnostic, on the second T, because a declaration reports one. The rule needs no type, so a template nothing instantiates is refused too (tests/cases/reject_generic_duplicate_type_parameter, …_arrow, …_method, …_class, …_interface, …_template, …_triple). A type alias's list is refused before it is read, by NL1054.
  • Not supported yet: a default type argument (<T = string>, a default type argument (`T = ...`) is not supported: a type argument is inferred from the arguments, NL2292, refused on a template nothing instantiates too, tests/cases/reject_decl_sweep_together) and type parameters on a constructor, each with its own message. Type parameters on a type alias are forbidden rather than unsupported (NL1054, Type aliases). A generic main is refused too, in words of its own: the entry point is called by the C runtime, which has no type arguments to give it. Multiple bounds (T extends A & B), a bound that mentions the parameter it constrains, and a bound that is "any type with a method compare" are not planned on this path. The design and the milestones are in wp18-generics.md.

Generic classes and interfaces

A class or an interface may declare type parameters too, and an instantiation is an ordinary struct: Box<i32> is %struct.Box$i32 with Box's fields at T = i32, its methods are @Box$i32.get and @Box$i32.constructor, and everything the language already does to a struct — the layout, implements, new, field access, the C header, the debug info — works on it unchanged (tests/cases/gen_box_i32, gen_box_string).

class Box<T> {
  value: T;
  constructor(v: T) { this.value = v; }
  get(): T { return this.value; }
}

interface Pair<A, B> { first: A; second: B; }

export const main = (): i32 => {
  const b = new Box<i32>(7);                            // %struct.Box$i32
  const p: Pair<i32, string> = { first: 1, second: "" }; // %struct.Pair$i32$str
  return b.get();
};
  • The type arguments are written out, never inferred. Box on its own is not a type, because there is no layout until T is bound: `Holder` is generic: it must be written with its type arguments, e.g. `Holder<number>` (tests/cases/reject_generic_class). The two positions that take them are an annotation (const b: Box<i32>) and new (new Box<i32>(7)) — the two a one-token parser can read a type-argument list in, which is the same reason a call infers instead (§2a of wp18-generics.md). A class that is not generic takes none: `Point` is not generic, so `new Point` takes no type arguments (tests/cases/reject_cls_new_generic).
  • A type argument may be anything a type may be, including another instantiation: Box<Point> is %struct.Box$$Point and Box<Box<i32>> is %struct.Box$$Box$i32, the >> split back into two by the parser (tests/cases/gen_box_struct, gen_nested). A type parameter takes none, though: T is whatever the instantiation bound it to, so const y: T<i32> is Unsupported type reference `T<i32>` (tests/cases/reject_generic_type_param_args).
  • A template obeys every rule a declared class obeys. Type parameters change what a declaration means, not what it may be written with, so export default class Box<T>, abstract class Box<T> and interface Box<T> extends Base are refused in the same words as their non-generic spellings, and declare class Box<T> is A generic `declare class` is not supported (a C struct has one layout; tests/cases/reject_generic_declare_class, reject_generic_export_default, reject_generic_abstract, reject_generic_interface_extends).
  • A template claims its name in the one declaration namespace. class Box<T> is a declaration of Box, so a function, a constant, an alias, an enum or a second class of that name is refused whichever was written first — the structs pass runs before the functions pass, so the template is always the one already there (tests/cases/reject_generic_class_fn_clash, reject_generic_class_after_fn, reject_generic_class_redeclared).
  • A class or interface name is program-wide, because a struct type is identified by its name alone: %struct.Base, and a type id interned by that name. So two modules may not both declare a class or interface of one name, whether or not either is exported, whether it has fields only, and however different the two field lists are. The later declaration in load order is refused, once per name per module, naming the module that declared it first: Class `Base` is also declared in main.ts; a class or interface name must be unique across the program whether or not it is exported, because a struct type is identified by its name alone (tests/link/class_clash_fields, tests/link/iface_clash_private; NL3028). Left alone, a value of one was accepted where the other was expected and its fields were read at the other's offsets, with no diagnostic (#193). Two classes that share a name and a constructor or method are refused in the same words, once (tests/link/class_clash_constructor, …_method, …_package), and across two packages the wording is the one tests/link/two_packages_struct pins. A declaration is compared with the first of its name in its own package before any other, so when a third package declared the name first, the first of the two is refused across packages and the second in these words (tests/link/class_clash_after_package, …_package_after_root). Importing the one declaration is how two modules share a struct (tests/link/iface_same_name_imported).
  • An instantiation's name is program-wide, because %struct.Box$i32 and @Box$i32.constructor are. So two modules may not both declare a generic class or interface of one name once both instantiate it: Generic class `Holder` is also declared in helper.ts; a class or interface name must be unique across the program, and an instantiation is named after its template (tests/link/generic_class_clash), once per template even when signatures have made both instantiations, constructor and methods included, before any body is checked (tests/link/generic_class_clash_signature), and across two packages the wording is the one tests/link/package_generic_class_clash pins. As for a declared name, a template is compared with the first of its instantiation's name in its own package before any other, so when a third package instantiated the name first, the second module of the package is refused in these words (tests/link/generic_class_clash_after_package). Nothing about the two declarations is visible at a call, so left unreported the second layout would simply replace the first and a field read would land on the wrong offset.
  • implements may name an instantiation. class Box<T> implements Container<T> is checked once per instantiation, against Container$i32, by the ordinary field-prefix rule — the interface's fields are the class's first fields, so the conversion is one bitcast (tests/cases/gen_implements).
  • Each instantiation gets its own facts, exactly as a generic function's do: one new Box<T>(v) in a body that does not escape is an entry-block alloca at T = i32 and an arena bump at T = string when the box is returned — one source line, two memory behaviours (tests/cases/gen_stack).
  • It is named as written, and linked as mangled. A diagnostic says Unknown field `nope` on class `Box<i32>` and Argument 1 of `Box<i32>.set`: expected i32, got string (tests/cases/reject_generic_member_display, reject_generic_implements_display, reject_generic_ctor_display), and -g names the composite type Box<i32> and its methods Box<i32>.get, with Box$i32.get as the linkageName (tests/cases/dbg_generic). In a header it is struct nish_gen_Box_i32; the rule for both is the generic function's (Generic functions).
  • A class or interface may constrain its parameters with the rules a generic function's follow. Its own methods read the constraint's members through a field of that type — this.item.area in class Holder<T extends Shape> is two loads, item and then area of %struct.Circle (tests/cases/gen_constraint_class) — and every instantiation is checked against the constraint where its type arguments are written, after new and in an annotation (tests/cases/reject_generic_unsatisfied_constraint_class, reject_generic_unsatisfied_constraint_annotation).
  • A field may not grow its own type argument. class Nest<T> { inner: Nest<T[]> | null } asks for a strictly larger instantiation every time it is laid out: Monomorphising `Nest` would not terminate: `Nest<i32>` names `Nest<i32[]>`, which puts `i32` under a type constructor instead of passing it on, so the chain has no end; name `T` itself, or a type that does not mention it (tests/cases/reject_generic_expanding_field). Nest<T> and a type that mentions no parameter are both fine — it is the same rule a generic function's recursive call is held to, stated over fields.
  • A generic class or interface may be imported, and the whole-program rule is the one a generic function's is: %struct.Box$i32 and every @Box$i32.* symbol are defined once, by the module that declares Box<T>, and declared by every module that holds one (tests/link/generic_import, tests/link/package_generic_import). Unlike a declared class it may be renamed on import, because the name that crosses the ABI is the template's and not the caller's. Named without its type arguments it is refused in the same words a locally declared template is (tests/link/generic_import_no_args).

Generic methods

A method of a class — generic or not — may declare type parameters of its own, and it is compiled the way a generic function is: one define per (receiver struct, method type-argument tuple), named by appending $ and each of the method's mangled type arguments to the receiver's method symbol. The IR of an instantiation is the IR of the method somebody would have written for that type, with this first and the symbol renamed (tests/cases/gen_method).

class Chooser {
  flip: boolean = false;
  pick<T>(a: T, b: T): T { return this.flip ? b : a; }
}

class Box<T> {
  value: T;
  constructor(value: T) { this.value = value; }
  keep<U>(other: U): T { return this.value; }
}

const c = new Chooser();
c.pick(1, 2);                    // @Chooser.pick$i32
c.pick("a", "b");                // @Chooser.pick$str
new Box<i32>(7).keep("seven");   // @Box$i32.keep$str
  • The type arguments are inferred from the arguments, exactly as a generic function's are, and never written at the call: h.get<i32>(7) is Type arguments are not written at a call site in Nish: `T` is inferred from the arguments, so write `h.get(...)` (tests/cases/reject_generic_method, and through a field path, reject_generic_method_path_type_args). A method type parameter no parameter mentions is refused at the declaration: Cannot infer `U` for `Holder.make`: a type parameter is inferred from the arguments, and `U` appears in none of them; give `Holder.make` a parameter that mentions `U` (reject_generic_method_uninferable).
  • The receiver is part of the key. On a generic class, Box<i32>.pair<string> and Box<string>.pair<string> are two instantiations, and a tuple asked for twice is one: two receivers times two method tuples are four defines and no more (tests/cases/gen_method_generic_class). Inside the body the class's parameters and the method's are both bound.
  • A method's type parameter may not reuse a name of its class's, because both are in scope in its body and every T there would have two meanings: Type parameter `T` of `Box.map` shadows `Box`'s own `T`: a class's type parameters are in scope in its methods, and a diagnostic that names `T` has to mean one of them; give the method's another name (reject_generic_method_shadow, once per class however many times it is instantiated).
  • A method type parameter may be constrained, with a generic function's rules: a U extends Shape has exactly Shape's members, read straight out of the concrete struct (tests/cases/gen_method_constraint); an unconstrained one has none, through a local, a field or another generic call alike (reject_generic_method_member, reject_generic_method_member_through_call); a type argument is held to the constraint at each call (reject_generic_method_unsatisfied); and the constraint may mention no type parameter, the class's included, because it is resolved once for every receiver (reject_generic_method_constraint_class_param).
  • Recursion terminates or is refused by name, with a generic function's rule and sentence: Monomorphising `Walker.walk` would not terminate: `Walker.walk<i32>` asks for `Walker.walk<i32[]>`, ... (reject_generic_method_recursion).
  • -g names an instantiation as written — Box<i32>.pair<string> — and links it as mangled, Box$i32.pair$str (tests/cases/dbg_generic_method).
  • The whole-program rule holds: a generic method of an exported class is instantiated, and defined once, in the class's module, and every other module that calls it declares it (tests/link/generic_method_import).
  • In the C header an instantiation is declared under the nish_gen_ spelling a generic function's takes, bound to its symbol, with the object first: int32_t nish_gen_Holder_pick_i32(struct Holder *this_, int32_t a, int32_t b) NISH_SYMBOL("Holder.pick$i32"); (tests/cases/gen_method_export). The wasm and N-API sidecars bridge no method. An exported class's generic method that nothing calls has no symbol, so with a sidecar flag it is refused: `Holder.pick` is generic, so it has no single C signature: a generic method becomes a symbol only where it is instantiated, and this program instantiates none (reject_generic_method_unused).
  • A constructor has no type parameters of its own: it takes its class's, written after new. constructor<T>() is a syntax error (reject_generic_method_constructor), as it is in TypeScript.
  • A generic method is a member like any other, so a field or another method of the same name is Duplicate member `pick` in class `Chooser` (reject_generic_method_field_clash).
  • $ is refused in a method name where it could spell an instantiation: in any generic method's own name, which every instantiation's symbol is built on (pick$i32<V> at i32 would be pick<U1, U2> at i32, i32; reject_generic_method_dollar_generic, …_generic_first, …_generic_class), and in a plain method's name when the part before the $ is a generic sibling's — pick$i32 beside a generic pick would be pick<i32>'s symbol (reject_generic_method_dollar).

Function parameters

A parameter of a top-level function may have a function type written inline in its annotation, and its argument is then a function the checker can name at the call: a top-level function written by name, an arrow written right there, or a local holding a function reference, which the call wraps in an arrow that calls it (Function references). Such a function is a template, with or without type parameters, and each callee it is given is an instantiation of its own, exactly as each type-argument tuple is: one define per (template, type arguments, function arguments), whose body calls its callee directly (tests/cases/fnarg_named). There is no function pointer and no indirect call; the parameter has no LLVM parameter at all, and the whole-program fixpoint sees an ordinary direct call it can prove things through (wp23-language-surface.md §6, and wp28-compatibility-mode.md §7.4 for what the other kind of callee would cost). A parameter whose function type comes through an alias (type F = (x: i32) => i32, then f: F) is not a function parameter but an ordinary parameter holding a function reference.

const apply = (f: (x: i32) => i32, x: i32): i32 => f(x) + 1;
const square = (x: i32): i32 => x * x;

export const main = (): i32 => {
  const a = apply(square, 3);        // @apply$fn.6.square, calling @square
  const b = apply((x) => x - 1, 3);  // @apply$fn.16.nish_main$arrow0
  return a + b;
};
  • The callee is part of the symbol. An instantiation's name is the template's, then its type arguments, then $fn.<length>.<symbol> for each function argument, which is self-delimiting however many $ the callee's own symbol holds; a person reads it as apply<square> or map<i32, str, (x) => ...> in every diagnostic, the --emit-checked dump and the -g DWARF name. Two callees are two defines; the same callee twice is one (tests/cases/fnarg_named).

  • An arrow argument is lifted into a function of its own, named after the function it is written in (nish_main$arrow0) — a symbol no declaration can spell, because a declared name may not hold a $. It may be written x => ..., (x) => ... or with annotations, with a concise or a block body (tests/cases/fnarg_arrow), and under -g it has a DISubprogram of its own, placed where the arrow starts (tests/cases/dbg_fnarg_arrow).

  • Inference runs in two phases. The value arguments bind what they can first; then each function argument is resolved against its parameter's function type. An arrow's unannotated parameters take the types the value arguments bound, a named function's signature binds any type parameter it mentions, and an arrow with a concise body binds the result type from its body, so map(xs, (n) => n > 1) over an i32[] is map<i32, boolean, (n) => ...>. A numeric literal argument is checked after the others, against what they bound, so a fold's identity 0.0 over an f64[] is an f64 in either number mode (tests/cases/fnarg_generic).

  • A function parameter may be called and passed on, and nothing else. Passing it to another function-typed parameter forwards the callee as part of the key, so the innermost call is still direct (tests/cases/fnarg_forward). Storing, returning, comparing or putting it in an array is `f` is a function parameter and can only be called or passed on as a function argument: it names the callee each call is compiled for, not a value, so it cannot be stored, returned, compared or put in an array (tests/cases/reject_fnarg_store, reject_fnarg_return).

  • An arrow may read the const locals and the parameters of the function it is written in (WP29 capture). Each name it reads is handed to it as an extra argument when the call is made: the lifted arrow gains a parameter per capture, the instantiation that runs it gains one too and forwards it, and every call stays direct — no environment object, no indirect call. The captures are not part of the instantiation's key, so one arrow is one define however many values it reads. A narrowing in force at the call holds in the body (if (name !== null) { each(xs, (x) => use(name)) } reads name as a string). The arrow may write through what it captured — infos.push(x), box.n = v — because the call runs before the function goes on (tests/cases/fnarg_capture, fnarg_capture_narrow, fnarg_capture_local, fnarg_capture_escape). What it may not do:

    • read this (NL2590, reject_fnarg_capture_this): bind it, or what the arrow needs of it, to a const before the call;
    • read a let (NL2585, reject_fnarg_capture_let), whose value could change after the arrow is handed it; nish --fix copies it into a const just before the call (tests/fix/fnarg-capture-let);
    • assign to a name it captured (NL2586, reject_fnarg_capture_write): the assignment would change only its own copy;
    • name a function parameter of the function it is written in (NL2587, reject_fnarg_capture_function_param), which is bound at instantiation and is no value;
    • capture a Secret, a scope, a lock, a guard, a channel, a foreign handle or a CPtr (NL2588, reject_fnarg_capture_secret, reject_mutex_guard_capture, reject_ffi_handle_closure_capture, reject_ffi_handle_field_capture), which their own rules keep in the block that made them;
    • read anything from outside it when it is written inside another arrow (NL2501, reject_fnarg_capture_nested).

    An arrow given to expose or exposeWith still captures nothing (NL2335, NL2589; reject_secret_expose_capture, _forwarded), an arrow handed on to a parallel form or spawn meets their rules where it runs (reject_fnarg_capture_parallel_write, _spawn_write, tests/link/fnarg_capture_forward), and a recursive function that passes itself a new arrow on every call meets the instantiation limit (reject_fnarg_capture_recursion). Escape analysis and the parameter attributes read a capture as an argument to the arrow, at the arrow's own parameter for it: what it names escapes only when the arrow keeps it, or hands it to something that does, so an object an arrow only reads may be an alloca and a parameter it only reads readonly nocapture (mem_capture_read_stack, fnarg_capture_readonly; the negatives are mem_capture_forward_keep, mem_capture_callee_keep). An element of a captured array is kept unless the arrow proved it keeps none, and a captured array is never one whose strings go to the caller with it (mem_capture_element_keep). The capture stays an escape, as it was before, when anything but an ordinary function stands between the call and the arrow (a data-parallel form, spawn, a task, an async function: tests/link/mem_capture_parallel_forward, mem_capture_async), and a captured parts array is not released by join, whose release depends on order rather than escape (mem_join_parts_captured). An arrow anywhere but as such an argument or as a top-level const is an arrow in value position is not yet a function reference: declare it as a top-level function and name it (tests/cases/reject_fnarg_arrow_value), or NL2731 when it captures (Function references).

  • A function argument has exactly its parameter's type, once the type parameters are bound, because the body calls it with those types and reads that result: Argument 1 of `apply`: `widen` is `(x: i32) => i64`, and `f` is `(x: i32) => i32`; a function argument must have exactly its parameter's type (tests/cases/reject_fnarg_mismatch). The argument must be the name of a top-level function, an arrow, or a local that holds a function reference (tests/cases/fnref_arg): the call passes an arrow that captures the local by value, so the callee calls the function that was passed even if it reassigns where that came from (fnref_arg_reassign). An arrow passed there may call a reference it captures, narrowed first if it is nullable (fnref_arg_capture, fnref_arg_capture_param). A field or element would be read again at each call the callee makes, where JavaScript reads it once, so it is refused, ops[next(b)] and b.op alike: `ops[next(b)]` is read again at each call `twice` makes, where JavaScript reads it once at the call: read it into a `const` first and pass the `const` (NL2759, reject_fnref_arg_effect, reject_fnref_arg_field, reject_fnref_arg_element_nullable). A nullable local is passed once a test has narrowed it: `h` may be `null`, and `apply` calls it: pass it once a test such as `h !== null` has narrowed it (NL2760, tests/wordings/nl2760_fnarg_nullable_reference). A local that a later argument of the same call assigns is refused, since the call reads it after its arguments where JavaScript reads it before them: `h` is assigned by a later argument of `apply`, and the call reads it after its arguments where JavaScript reads it before them: read it into a `const` first and pass the `const` (NL2761, reject_fnarg_ref_assigned_later). A reference of another signature is refused as any other function argument is (NL2340, tests/wordings/nl2340_fnarg_reference_arity); any other value is refused and named by its type, `n` is `i32`, which is not a function (reject_fnarg_not_name). The builtins that check the body they are handed — parallelMap, parallelMapInto, parallelReduce, a scope's spawn, expose, exposeWith, region and enterWith — take only a function or an arrow named at the call, so a reference there is `h` is a function reference: `parallelMap` takes a top-level function or an arrow named at the call (NL2734, tests/wordings/nl2734_fnref_to_function_parameter). A generic function names no one function and is refused with the arrow to write instead (reject_fnarg_generic_name), and an arrow takes exactly its type's parameters (reject_fnarg_arity). An unannotated arrow parameter whose type no value argument binds (reject_fnarg_infer) and a block-bodied arrow whose result only it could bind (reject_fnarg_block_return) must be annotated.

  • Only a top-level function's parameter makes a function type a function parameter. Written inline on a parameter of a method or a constructor, or of a function type that is itself a function parameter's, it is `f` cannot have a function type here: a function type may only annotate a parameter of a top-level function, which is monomorphised for each function it is given, and this is a parameter of a method or a constructor, with a parameter of a function type for the second (tests/cases/reject_fnarg_method, reject_fnarg_nested); a method or a constructor takes a reference through an alias instead. Anywhere else — a field, a local, an array element, a return type, an alias — a function type is the type of a function reference, and what is stored there is checked against it: Field `run` of class `Handler` is (i32) => i32 but its initializer is i32 (reject_fnarg_field), Return type mismatch: function returns (i32) => i32 but expression is i32 (reject_fnarg_return_type), and a reference local is initialised where it is declared, Variable `g` must be initialized (reject_fnarg_local).

  • Across modules, an instantiation is made in the template's module (WP18 G7), so it may call a private function, or an arrow, of the module that asked. That callee is hidden rather than internal — in the final link, exported from nothing — and the template's module declares it with the attributes of its define (tests/link/fnarg_private). A hidden symbol is still global in the executable, so one the program's own package declared, which carries no prefix, ends with a $ (@square$): C cannot spell it, so a task called read, write or free is never the definition the runtime's calls into the C library reach (tests/link/thread_scope_libc_named_tasks, #610). The interop sidecars describe no instantiation that was given a function, and no lifted arrow: neither has a name a host could call.

Function references

A name bound to a top-level function, in either spelling, written anywhere but a call's callee or a function argument is a function reference (a local holding one may itself be passed as a function argument, below): a value of the function's type, (x: i32) => i32. A function type names a reference everywhere but inline on a parameter of a top-level function (Function parameters): on a local, a field, an array element, a return type, a type alias, and a parameter typed through an alias.

type Op = (x: i32) => i32;

const double = (x: i32): i32 => x * 2;
const negate = (x: i32): i32 => 0 - x;

class Stage {
  op: Op;
  constructor(op: Op) {
    this.op = op;
  }
  run(x: i32): i32 {
    return this.op(x);   // a direct call of the dispatcher for `Op`
  }
}

export const main = (): void => {
  const s = new Stage(double);
  console.log(`${s.run(5)}`);  // 10
  s.op = negate;
  console.log(`${s.run(5)}`);  // -5
};
  • What a reference can hold. A function this program defines, at the top level of a module, that is not generic, not async, has no rest parameter and takes no function parameter, and whose arrow, if it is one, captures nothing. Each other kind is refused where it is named:
    • `id` is generic: a function reference names one non-generic top-level function (NL2725, tests/cases/reject_fnref_generic);
    • `later` is async: a function reference cannot hold it, because its call is a promise (NL2727, reject_fnref_async);
    • `abs` is declared in C: a function reference names a function this program defines (NL2728, reject_fnref_ffi);
    • `sum` has a rest parameter: a function reference names a function whose every parameter the caller passes (NL2729, reject_fnref_rest; a defaulted parameter the same, nl2729_fnref_default), and a written function type with a rest, optional or defaulted parameter, `(x: i32, y?: i32) => i32` has an optional parameter: a function reference names a function whose every parameter the caller passes (NL2729, reject_fnref_type_optional, reject_fnref_type_rest);
    • `bump` takes a `MutexGuard`: a guard is handed only to a function named at the call, so a function reference cannot name it; call the helper by name (NL2739, tests/wordings/nl2739_fnref_guard);
    • `apply` takes a function parameter, so it has no single body to refer to (NL2726, tests/wordings/nl2726_fnref_template), and a function type whose own parameter is written as a function type, `(g: (x: i32) => i32) => i32` takes a function parameter, so no function it could name has a single body to refer to (NL2735, tests/wordings/nl2735_fnref_type_function_param);
    • a method, `c.step` is a method: a function reference names a top-level function, and a method needs its receiver (NL2732, tests/cases/reject_fnarg_alias);
    • an arrow that captures, this arrow captures `k`: a function reference names a top-level function and carries no environment (NL2731, reject_arrow_as_value, reject_fnref_capture), and one that does not but is not a top-level const, an arrow in value position is not yet a function reference: declare it as a top-level function and name it (NL2336, reject_fnarg_arrow_value).
  • Where a reference goes. It is held in a local, a field (of a class or an interface), an array element, a Map value or a return value, and passed as an argument to a parameter typed through an alias, of a function, a method or a constructor (tests/cases/fnref_field, fnref_array, fnref_stores, fnref_generic, fnref_map), or to an inline function-typed parameter (Function parameters, fnref_arg): that call passes an arrow that calls the reference, so the callee is instantiated for the call and calls through the dispatcher of the argument's place. A Map may hold an array of references or a nullable of either as its value (fnref_map_array, fnref_map_nullable); ?? on a nullable reference tests it against null, tag 0 (fnref_nullable_coalesce); a Map key may not be a reference, `(i32) => i32` cannot be the key type of `Map<(i32) => i32, i32>` (reject_fnref_map_key). A reference has no zero value: a local is initialised where it is declared (reject_fnarg_local), and new Array<(i32) => i32>(n) is `new Array<(i32) => i32>(n)` would zero-fill with 0, which names no function; build it with `[]` and `push` instead (NL2730, reject_fnref_zero_fill); new Array<F | null>(n) fills with null and is allowed, except as module storage, which holds no | null (NL2686, reject_fnref_storage_nullable). For the same reason an array of references has no fill, unshift or splice: fill is refused as a byte copy (reject_fnref_row_fill), the other two as an unknown method (reject_fnref_row_unshift, reject_fnref_row_splice). A module constant may not hold one (Module constant `alias` must be a number, boolean, string, or a `readonly` array of numbers written out as a literal, not (i32) => i32; there is no top-level code to build anything else). An arrow argument may capture a const reference local like any other value (fnref_capture).
  • What a reference does. It is called — h(3), this.op(4), or an element called in place — and compared with === and !==, which are true when the two name the same function (fnref_eq). Nothing else: printing it, putting it in a template string or any other use is `f` is a function reference: it can be called, compared, stored and passed, nothing else (NL2737, reject_fnref_print, and as JSON.stringify's argument, reject_fnref_stringify; a field, an element or a nullable reference alike, reject_fnref_stringify_place); jsonEncode refuses a type that holds one (`jsonEncode` cannot handle `Stage.op`, which is `(i32) => i32`, reject_fnref_json); < and the other orderings take numbers only (reject_fnref_order, reject_fnref_order_gt), as + and the other arithmetic does (reject_fnref_arith); and a function type inside another type prints in parentheses, ((i32) => i32)[] (reject_fnref_type_name).
  • A reference may be null. F | null holds a function or nothing: a field of that type starts null, new Array<F | null>(n) holds n nulls, and in memory null is tag 0, a tag no function has. A call goes through it only once a test proves it holds a function, h !== null or an early return on h === null; a call before one is `h` may be `null`: a nullable function reference is called only after a test such as `h !== null` proves it holds a function (NL2758, reject_fnref_nullable_call). A field is not narrowed: read it into a const and test that (reject_fnref_nullable_field_call). A number is no reference, so a nullable field initialised with one is refused as a non-null one is (NL2013, reject_fnref_nullable_field_literal, reject_fnref_nullable_field_negative). The rules otherwise follow Nullable types (tests/cases/fnref_nullable).
  • In memory a reference is an i32 tag, the function's position, from 1, among the program's functions of that signature: never an address. A call through it is a direct call of an internal dispatcher, @$ref.<module>.<signature>.<n>, that takes the tag and the arguments, switches on the tag and tail-calls each function directly, the last one as the default arm; there is no function pointer and no indirect call (tests/cases/fnref_field). Each place — a local or a parameter, a function's return, a field, and the elements of every array of one reference type (a Map's values are such elements) — has a set, the functions that can flow into it, and a call dispatches over its place's set only: a set of one's dispatcher is one tail call with no switch, and an empty set's dispatcher traps (fnref_empty_set, fnref_place_set). The dispatcher carries the attributes its members share, so a call through a set whose members all leave an argument alone leaves it on the stack. A set that mixes members that keep a pointer argument with members that do not is refused, this call through `h` can reach `u3`, which keeps `p`, and `u0`, which does not, so every call here would allocate `p` in the arena; ... (NL2736, reject_fnref_escape_join, fnref_escape_join). Map.get reads the place a Map's values share with the arrays of that type, so a call through what it returns dispatches over the functions stored into any of them (fnref_map, fnref_place_set_map); where one of those keeps a pointer argument the others leave alone, that call is the mixed set above and refused (reject_fnref_place_set_map). A Result arm is not a place, so a reference directly in one — Result<F, E>, Result<F | null, E>, Result<T, F>, including through a generic's type parameter — is refused, `Result<(i32) => i32, string>` cannot hold a function reference in an arm: ... (NL2762, reject_fnref_result, reject_fnref_result_generic); an array of references in an arm holds the element place and compiles (fnref_result_array). An assignment used as a value, k((r = inc)), stores into its target and hands the same function on (fnref_assign_value). A set past eight members is warning NL9017, a switch where a direct call would not branch (tests/wordings/nl9017_fnref_wide_set). Under --profile debug the dispatcher first checks that the tag is in its set and panics with function reference holds a function outside its place's set if it is not; the speed build checks nothing (fnref_debug_check). The IR is in IR_COOKBOOK.md.
  • A reference does not cross to a host. A tag means something only to this program's dispatchers, so in an open-world build (-o) an exported function whose parameter or return type holds a reference, or a class an exported signature reaches with such a field, is Parameter `f` of `apply` holds a function reference that code outside this program can reach: a reference is a tag this program's dispatcher reads, not an address, and a host could hand back one no dispatcher knows (NL2733, reject_fnref_host, reject_fnref_host_field, reject_fnref_no_strict, tests/wordings/nl2733_fnref_host_return). A closed-world --link build exports such a function between its own modules (tests/link/fnref_cross_module).

Still refused: a closure (an arrow that captures), a reference to a method, a generic or an async function, and a reference as a Map key (reject_fnref_map_key).

Async functions: async and await

An async function is a task the compiler can size: it may wait, and every wait is written await on a direct call (wp49-async-server.md §2). No expression ever has a promise type, so there is no promise to store, pass or chain, and an awaited call's frame nests in its caller's. An async function compiles to a frame and a resume switch (async_lowered, async_await_single, async_await_chain, async_await_all, async_spawn, async_main, async_result): one that awaits becomes i1 @f.resume(i8*), which picks up at the state its frame records and answers whether it finished; one that awaits nothing is a plain function, and awaiting it is a plain call. The frames of a program are sized when it is linked, and export async function main is resumed from the process entry until it finishes and its spawned tasks are done (async_main). A task waits on the outside world with await sleep(ms), await readable(fd) or await writable(fd): it is parked on the executor's kernel wait (epoll, kqueue on Darwin) and the others run until it is ready (below; wp49-async-server.md, "Stage 5, part 1"). A program with no async function compiles byte for byte as before, and links no executor; one whose async functions are never called keeps them internal, as it would a plain function (async_unused). A function that awaits loses willreturn and readonly; one that does not keeps its attributes (async_attributes). A failure is the answer's Result, Promise<Result<T, E>>, read after the await as any other Result (async_result).

import { awaitAll, spawn } from "nish:async"

const double = async (x: i32): Promise<i32> => {
  return x * 2
}

const log = async (x: i32): Promise<void> => {
  console.log(x)
}

const run = async (): Promise<void> => {
  const a = await double(3)
  let b: i32 = 0
  b = await double(a)
  await log(b)
  const [c, d] = await awaitAll(double(1), double(2))
  spawn(log(c + d))
}
  • Who may be async. A top-level function or an arrow, top-level or passed to a function-typed parameter. A method, a field or a constructor is Only a top-level function or an arrow may be `async` (NL1063, reject_async_method), because a task never keeps a this.
  • Promise<T> is only an async function's return type. An async function without it is An `async` function's return type is `Promise<T>` (NL2655, reject_async_function, reject_async_arrow_declaration, reject_async_return_missing); T is written inside it, Promise<void> for none. Anywhere else — a parameter, a field, a local, a type alias, an interface member — Promise<T> is `Promise<T>` is only an `async` function's return type (NL2654, reject_async_promise_parameter, _field, _local, _alias, _interface_member).
  • await is written inside an async function (`await` is only written inside an `async` function, NL2641, reject_await, reject_await_constant), on a direct call to one: await x on a value is NL2643 (reject_async_await_value), await on a plain function's call NL2644 (reject_async_await_plain_call). Inside an async function await (f(x)) is the operator, as TypeScript reads it; outside one await(f(x)) is a call to a function named await, NL2651 (reject_async_await_parenthesised).
  • await stands at the top of a statement: an expression statement, a declaration's initializer, a return or the right side of a plain =. Inside another expression — an argument, a condition, an operand, a compound assignment — it would hold a value across the wait, and is NL2642 (reject_async_await_nested, _condition, _compound); a for initializer is not a declaration statement, for awaitAll too (reject_async_await_all_for). Bind the answer to a const first.
  • A call to an async function is awaited where it is made, or handed to spawn or awaitAll. Anything else would make a promise a value: `load` is `async`: a call to it is awaited where it is made (NL2645, reject_async_unawaited_call, reject_async_unawaited_argument). .then is NL2650 (reject_async_then), Promise.all NL2649 (reject_async_promise_all), and for await is `for await` is not supported here (NL2133, reject_for_await_template, tests/wordings/nl2133_for_await).
  • nish:async exports spawn and awaitAll, reached only through the import. spawn(f(x)) starts f as a task of its own and does not wait for it; it takes a direct call to an async function (NL2644), and since nothing reads its answer, f answers void or a Result it reports itself (NL2646, reject_async_spawn_answer). awaitAll(f(x), g(y)) waits for two to eight calls at once and is awaited into one name per call, const [a, b] = await awaitAll(f(x), g(y)); one call, or an awaitAll not so destructured, is NL2647 (reject_async_await_all_one, reject_async_await_all_bare). So is a pattern over it that is not one plain name per call, each with the reason: an object pattern (the answers are read by position), a hole (it would drop an answer; name it _b), or a default, rest or nested slot (reject_async_await_all_object, reject_async_await_all_hole, reject_async_await_all_default).
  • An async arrow is passed only where it is awaited. The parameter's function type answers Promise<T>, and the callee awaits each call to it. An async arrow where the type answers a plain T is NL2639 (reject_async_arrow); a plain function where it answers Promise<T> is NL2640 (reject_async_plain_to_promise_parameter).
  • No recursion through await. An awaited call's frame is nested in its caller's, so a cycle in the call graph that runs through an await has no size: recursion through `await` is refused at the await that closes the cycle, direct or mutual (NL2648, reject_async_await_self, reject_async_await_mutual). Write it as a loop.
  • A spawned task's arguments outlive the spawner. spawn(f(x)) may return, and its caller with it, while f still waits. A number, a boolean or an enum is copied by value, and a string is copied into the task's own region before the task first runs (async_sleep_spawn_string, tests/link/async_spawn_args). An object, an array, a Map or a Set is shared: the task holds the spawner's own, and sees and updates its numbers (async_spawn_shared). A function that feeds a task a shared argument, itself or through a callee, releases no arena scope over what it allocated, so the object outlives every task that holds it (tests/link/async_spawn_shared_outlives). An argument of any other type — T | null, a Region, a Secret — is `spawn` hands `f` an argument of type `T`, which lives in the spawner's region and is freed when the spawner returns (NL2673, reject_async_spawn_array_arg). A shared object or array that holds, in any field or element, a Region, a Secret, a handle class a using block closes, or a scope, lock, guard or channel of nish/threads is refused too, with what it holds named: its spawner may free, wipe or close that part while the task still waits (NL2691, reject_async_spawn_region_field, reject_async_spawn_secret_field, reject_async_spawn_handle, reject_async_spawn_threads_field).
  • A spawned task keeps its own memory out of what it shares. The task's region is freed when the task ends, so the region rules hold it to a shared argument, and everything reachable from it, as memory outside its region (Regions): storing an object or a string it made into a field or an element is NL2609, directly, through an alias, a method or a callee (reject_async_spawn_store_region, reject_async_spawn_store_string, reject_async_spawn_store_alias, reject_async_spawn_store_method, reject_async_spawn_store_callee, reject_async_spawn_store_reachable, reject_async_spawn_store_element, reject_async_spawn_store_bound, reject_async_spawn_await_all). A call that hands a function or method both a shared object it writes into and a string or object of the task's is NL2609 as well, since nothing says the one is not kept in the other (reject_async_spawn_store_string_callee, reject_async_spawn_store_string_method, reject_async_spawn_store_copied), and growing a shared array, Map or Set — push, set, add, spawnPiped or spawnPipedWithin appending to its output arrays, on the argument or on what a method answers out of it — is NL2615 (reject_async_spawn_grow, reject_async_spawn_grow_method, reject_async_spawn_grow_map, reject_async_spawn_grow_piped), and so is a builtin with no body of its own that does either, reserve on a shared Map (reject_async_spawn_grow_reserve) or jsonDecode into a shared object (reject_async_spawn_store_json). Copy numbers into it instead. The task keeps none of what it shares in memory of its own: a shared object, or anything reachable from it, put into a constructor argument, an array or object literal, a field or element store, push, a Map's set or a Set's add, a method that may keep its argument in its receiver, or a callee that stores one argument into another is NL2610, as in a region(...) body (reject_async_spawn_keep_holder, reject_async_spawn_keep_literal), and so is an element read out of a shared array (for (const t of shared), reject_async_spawn_keep_element). An object a helper makes may hold what the helper was handed, so in a task a fresh helper's answer other than a string, awaited or not, carries its arguments' origins too, and a store through a holder a helper built from the shared object is NL2609 (reject_async_spawn_keep_returned). An argument is left out of that join only when the answer's type cannot reach it: no class, array or string type the answer may hold (through fields, elements, a nullable or a union) is one the argument may hold, and neither side holds a class that implements an interface. So a PgAsyncResult awaited from a query on a shared PgAsyncPool is the task's own, while a Holder of a shared Tally or a Badge of an interface Tally implements is still the shared object's (reject_async_spawn_answer_reach). Read and update the shared object in place instead. Three refusals are conservative, and known to refuse sound programs: a call that hands a callee both a shared object it writes into and any string or object of the task's, even when the callee writes only numbers; a store into the shared object of what an await answers (const x = await helper(tally); tally.child = x), since an awaited answer is unknown to the walk; and a store of the task's memory into a holder a helper answered from the shared object (wrap(tally).note = s), which may be the task's alone. A task may not itself spawn a task with an object or an array, directly or through a callee, since what it hands on may be its own region's (NL2689, reject_async_spawn_nested); a task with only copied arguments may.
  • A spawned task's function argument captures only what the task may share. A top-level function, or an arrow written at the call, is a compile-time argument: the call is an instance with the function in its body (async_spawn_fn_arg). The arrow may read locals of the function it is written in when each is a number, a boolean or an enum, copied into the task's frame, or an object or array that holds, through any field or element, no Region, no Secret, no handle class and no nish/threads scope, lock, guard or channel. Its strings and nested objects are fine (a PgAsyncPool holds both). Any other capture is NL2690: a string, a nullable value or a union itself, or an object holding a Region (reject_async_spawn_block_arg, reject_async_spawn_handler_holds_region). A capture of a handle, or of an object holding a Secret, is refused sooner, by the rule for any arrow argument (NL2588, reject_async_spawn_handler_holds_handle, reject_async_spawn_handler_holds_secret); the capture rule refuses them as well, a check behind that one. A captured object is shared with the task as if handed to it: the spawner releases no arena scope over it, and the arrow's body is walked with the task's rules, the capture as memory the task does not own, so the arrow may update its numbers and call its methods (a serve handler awaiting a query on a captured PgAsyncPool) but may not store the task's memory into it or grow it (NL2609, NL2615, async_spawn_handler_reach, reject_async_spawn_handler_*). The arrow's parameters carry what the task passes at each direct call; one it reaches through a further function parameter is unknown, so storing it into a capture is refused. Each arrow is an instance of its own, so a refused arrow at one call site is not hidden by an admitted one at another.
  • Each spawned task has a region of its own, entered whenever the task runs and freed when it finishes, so what a task allocates is given back when it ends, not when main does (tests/link/async_region_per_task).
  • Each awaitAll call has a region of its own, made before it first runs and entered whenever it runs. When the call finishes, what it allocated joins the caller's memory, so an answer that points into it (a string, an array, a class) stays readable, and is given back when the caller's would be. So two calls that each hold memory across an await never free or overwrite each other's, whatever its size or whichever finishes first (async_await_all_lanes_large, tests/link/async_await_all_lanes). What a call allocates and does not answer is given back when it finishes, so its temporaries never reach the caller's memory (tests/link/async_await_all_retain). What the caller does keep is the answer and about 1.3 KB more per call, the unused end of the region's first block, until the caller's own memory is given back (ASYNC-17 in docs/security/async.md).
  • A main that never waits runs the loop too. An async main that awaits nothing compiles to a plain call, as any such function does; one that spawns tasks returns first, then the executor runs every task it spawned, and the program exits with main's answer once they are done (tests/link/async_sync_main_loop).
  • No async in a wasm build. A task is resumed by the native event loop, which a wasm module does not have: under a wasm32 target or --profile wasi an async function is NL2652 (reject_async_wasm, reject_async_wasi_profile).

Waiting on a timer or a descriptor: sleep, readable and writable

nish:async exports three waits on the executor. Each takes one i32 and answers a boolean, and each is written only where it is awaited:

Call Waits until Answers false at once when
await sleep(ms) ms milliseconds have passed ms is negative
await readable(fd) fd has bytes to read, or its peer closed it fd is not an open descriptor
await writable(fd) fd can take bytes fd is not an open descriptor

A wait that is answered at once does not suspend the task. A regular file never blocks, so readable and writable answer true for one at once. While a task waits, the others run; when none can run, the executor blocks in the kernel until a descriptor is ready or the first timer is due, and resumes the tasks it woke: the descriptors in the order the kernel reported them, then the expired timers, earliest deadline first, ties in the order they were made (async_sleep, tests/link/async_timer_order, tests/link/async_readable). A wait holds a duplicate of its descriptor, so closing the descriptor while a task waits on it does not end the wait, and the number's reuse cannot answer it (docs/security/async.md, ASYNC-4).

import { readable, sleep, spawn } from "nish:async"
import { netRead } from "nish:net"

const echo = async (fd: i32): Promise<void> => {
  const ready = await readable(fd)
  if (!ready) {
    return
  }
  const buf: u8[] = new Array<u8>(4)
  console.log(netRead(fd, buf, 0, 4))
}

const tick = async (): Promise<void> => {
  await sleep(10)
  console.log("tick")
}

export const main = async (): Promise<i32> => {
  spawn(echo(0))
  spawn(tick())
  return 0
}
  • A wait is awaited where it is written. sleep(10) as a statement, as spawn's argument or anywhere else a value would be kept is `sleep` is a wait on the executor, written only where it is awaited (NL2671, reject_async_sleep_bare, reject_async_sleep_spawned), and so are readable and writable.
  • It takes one i32: `readable` takes one `i32` (NL2672, reject_async_wait_argument).
  • readable and writable have no deadline of their own. A descriptor that never becomes ready parks its task until it does, or until the task is cancelled; readableWithin and writableWithin, below, take one (docs/security/async.md, ASYNC-6).

Bounding a wait and cancelling a task

nish:async bounds a descriptor wait, and cancels a task, with six more calls:

Call Answers
await readableWithin(fd, ms), await writableWithin(fd, ms) true once fd is ready; false when ms milliseconds passed first, when the wait could not be made, or when the task is cancelled. A negative ms waits without a limit
taskId() the running task's handle, an i64; main's is 0
cancel(task) true when the task was still running, so it is now cancelled; false for a finished task or main
cancelled() whether the running task has been cancelled
cancelAfter(task, ms) as cancel, ms milliseconds from now unless the task finishes first; a later call replaces the deadline, a negative ms removes it

A cancel is cooperative: it never stops a task in the middle of its code. Every sleep, readable, writable or bounded wait the task has parked answers false at once, and every one it makes later answers false without parking, so the task runs on to its end, its using bindings dispose and its region is freed. A task that does not await is not reached until it does. A handle names its task and no later one: a handle kept past its task's end, once the executor reuses the task's slot, cancels nothing (docs/security/async.md, ASYNC-18).

import { cancelAfter, cancelled, readableWithin, taskId } from "nish:async"

const serveOne = async (fd: i32): Promise<void> => {
  // The whole task gets two seconds; this one read gets half of one.
  cancelAfter(taskId(), 2000)
  const ready = await readableWithin(fd, 500)
  if (!ready) {
    console.log(cancelled() ? "cancelled" : "timed out")
    return
  }
}
  • The bounded waits are waits. Each is written only where it is awaited (NL2671, reject_async_within_bare) and takes two i32s: `readableWithin` takes two `i32`s: the descriptor it waits on, and the milliseconds it waits at most before it answers false (NL2692, reject_async_within_argument).
  • The cancellation calls are plain calls, written anywhere, the task's handle an i64: `cancel` from `nish:async` takes one `i64`, a task's handle from `taskId()` (NL2693, reject_async_cancel_argument).
  • spawn runs its task to its first wait before it returns, so a cancel written after spawn(f(x)) lands after f has run that far.
  • std/net/serve cancels a handler's task when its client hangs up, when its deadline passes and when the server stops; the async fetch and the async Postgres pool answer a cancelled wait as a failure of their own (tests/link/async_cancel, tests/link/net_serve_hangup, tests/link/std_postgres_async_cancel).

Where it is not Node. The source is TypeScript and runs under Node, but the resume order is not Node's, and these are declared differences:

Node Nish
Every await yields to the microtask queue, even on a value that is already there An await on a call that finishes without waiting returns at once; nothing else runs in between
Microtasks run before I/O callbacks, in a defined order Tasks resume in the order the kernel reports readiness, then by timer deadline
A rejected promise throws at the await There is no throw: a failure is the call's Result or null, as every Nish call reports it
spawn(f(x)) runs f to its first await, and the rest of f after the current task yields spawn(f(x)) runs f at once until it first waits on the outside world, then returns; a task that never waits has finished, its output printed, before spawn returns (async_spawn)
setTimeout and socket callbacks run on libuv's loop, which keeps the process alive while anything is registered sleep, readable and writable park a task on the executor; the program exits once main has returned and no task waits, a synchronous main included
A string passed to a spawned call is a shared reference It is copied into the task's region; an object, array, Map or Set is shared, and the task may store only numbers into it (NL2609, NL2615) and keep none of it in memory of its own (NL2610); one holding a Region, Secret, handle or nish/threads value is refused (NL2691), as is any other argument (NL2673)
A task that never awaits holds the thread, as here The same: the executor is cooperative and does not preempt a task

A program whose output does not depend on how its tasks interleave prints the same under both.

Under --emit-ts. An async function, Promise<T>, await, an async main, spawn(f(x)) and const [a, b] = await awaitAll(f(), g()) are written as the TypeScript they already are, so the output runs on Node's own async/await and resumes in Node's order, the left column above (tests/emit-ts/async/). nish:async is nish/runtime/async.mjs: awaitAll is Promise.all over its two to eight calls, and spawn starts its task without awaiting it and keeps it. The launcher of a program that declares an async function awaits main, then settleTasks(), which waits for every task spawn started, those tasks' own included, before it exits, as the native driver drains its queue. await sleep(ms) is a setTimeout that answers true, or false at once for a negative ms (tests/emit-ts/async/sleep); readable and writable have no Node counterpart a task can await, so a program that calls either is --emit-ts cannot write readable or writable from nish:async (NL4017, tests/emit-ts/async/refused-readable), and so is one that calls readableWithin, writableWithin, taskId, cancel, cancelled or cancelAfter, since a Node promise is not a task the executor can cancel (NL4018, tests/emit-ts/async/refused-cancel). A program whose spawned tasks print in an interleaving prints the same lines in another order (async_spawn, the order row of tests/differential/emit-ts-known.txt).

Data parallelism: nish/threads

parallelMapInto, parallelMap, parallelFrom and parallelReduce run a function over every element of an array, or every index of a new one, on as many threads as the machine has and the length is worth (wp29-thread-surface.md §4.1). They are imported from the standard library, and the function is a function parameter: a top-level function by name, or an arrow written at the call — which may read the const locals of the function it is written in (below).

import { parallelFrom, parallelMap, parallelMapInto, parallelReduce } from "nish/threads";

const square = (x: f64): f64 => x * x;

export const main = (): i32 => {
  const src: f64[] = [1.0, 2.0, 3.0];
  const dst: f64[] = [0.0, 0.0, 0.0];
  parallelMapInto(src, dst, square);                         // dst[i] = square(src[i])
  parallelMapInto(dst, dst, (x) => x + 1.0);                 // in place
  const k: f64 = 2.0;
  const scaled = parallelMap(src, (x) => x * k);             // a new array: 2, 4, 6
  const halves = parallelFrom(3, (i) => toF64(i) / k);       // a new array: 0, 0.5, 1
  const total = parallelReduce(dst, (a, b) => a + b, 0.0);   // 17
  return toI32(total + scaled[2] + halves[2]);
};
  • The module is ordinary Nish, and its bodies are the meaning. std/threads.ts writes each function as the sequential loop, which is what runs under Node and what npm run check type-checks. The compiler recognises the two templates by module and name and lowers one call in each instance — the loop over the whole range — onto nish_parallel_range (runtime/runtime-parallel.c), which cuts the range into contiguous pieces, up to eight per thread, that the threads claim one at a time as they finish the last, and joins them all before it returns. Everything else in the instance is emitted as written (tests/link/par_map, whose threads.ll is the whole lowering).
  • parallelMapInto(src, dst, f) writes f(src[i]) into dst[i] for every index of src. dst must be at least as long, which is checked once, before any element is written or any thread started: a shorter one panics with parallelMapInto: dst has 2 elements and src has 3 and exits 1 (tests/link/par_dst_short). src and dst may be the same array (tests/link/par_map).
  • parallelMap(src, f) answers a new array of f(src[i]) for every index of src, and parallelFrom(n, f) a new array of n elements, f(i) at index i — Array.from({ length: n }, (_, i) => f(i)). Each allocates its result on the calling thread before any element is computed, so nothing but the region can hold it while the threads write it, and then runs the map: parallelMap is parallelMapInto into that array, and parallelFrom the same region over the indices (tests/link/par_map_new, tests/link/par_from). parallelFrom(0, f) answers an empty array. The rules below hold of their f as they hold of parallelMapInto's, except the one about dst, which nothing else can reach.
  • A short array is the loop it would have been. A map is divided only once every thread's share carries enough work to pay for handing it to another thread (wp20-threads.md §8e). How many elements that is — the grain — is sized per call from a static estimate of what f costs: 2^21 over the estimate, which counts one per operation, 32 per allocation, 4 per call, and a loop's body as many times as its literal bound says or 64 times when the bound is not a literal. A map longer than the grain is divided into ceil(length / grain) chunks, at most eight per thread with one thread per core, so the first division is two chunks of half the grain: 2^20 units each. That is where a measured body, (x) => x + 1 at an estimate of 3, beat its loop by 1.31x on the minimum and 1.19x on the median, more than any difference between the loop and a byte-identical copy of it (at most 3.3% and 9.1%); at two chunks of 2^19 units the minimum won by 1.16x and the median by only 1.04x, within that 9.1% (wp29-thread-surface.md §8a). So a body estimated at 3, such as (x) => x + 1 or (x) => x * x, is divided past 699,050 elements (tests/link/par_map), (x * 31 + 7) % 1000 past 262,144 (tests/link/par_map_large), a body with a loop far sooner, and a cheaper body later: (x) => x, estimated at 1, past 2,097,152. Up to the grain the map calls its loop directly on the calling thread, where it inlines; there is no hand-off and no indirect call. The estimate counts a call as a call and not as the callee's body, so a body that hides its work behind a call is divided too little rather than too much. There is no knob for the grain (wp29-thread-surface.md §9).
  • The threads are a pool, as many as the cores the process may use. The first region or scope join that divides starts one worker per core but the caller's, which is the last of them, and at most 63 workers; they park between calls (wp20-threads.md §8e). The core count is the smallest of the CPUs online, the process's affinity mask (taskset, a container's cpuset) and the cgroup v2 cpu.max quota rounded up, so a program in a container limited to two cores divides two ways, not as many as the host has. NISH_NUM_THREADS, when it is set to a positive number, lowers that count and never raises it: NISH_NUM_THREADS=1 runs every region and every scope's tasks on the calling thread, in spawn order, and anything that is not a positive number is ignored. It is read once, by the first region or join that asks, and it decides only how the work is divided: a map's answer and a reduce's bits are the same at any count (tests/run.js runs every tests/link/thread_* and par_* program under NISH_NUM_THREADS=1 and =2 and requires the output it gave without). A Node addon (--emit-napi) keeps a thread per chunk or task for each call instead of a pool, because Node can unload it and a parked worker would outlive its code, and on WASI every region and task runs on the one thread there is.
  • parallelReduce(src, f, identity) is deterministic. src is split into min(64, ceil(n / 2^20)) blocks, each block is folded from identity, and the block results are combined left to right. std/threads.ts does the same blocking on one thread, so an f64 sum is the same bits on one core or sixty-four, compiled or under Node — which is not the bits of a plain left fold over the array (tests/link/par_reduce). The blocks are part of the answer, so where block k starts, floor(n * k / blocks), is computed exactly even where n * k passes 2^31 (tests/link/par_reduce_blocks, 64 blocks over 70,000,001 elements). An empty src answers identity. For that to be the fold it looks like, f has to be associative and identity its identity. An arrow whose whole body is one operator on its two parameters is held to both: a non-associative one is `(a, b) => ...` combines with `-`, which is not associative: `parallelReduce` folds each block from the identity and then combines the blocks, which is a left fold only for an associative operator (-, /, %, ** and the shifts; tests/cases/reject_par_reduce_operator), and a recognised one given anything but its identity — 0 for +, | and ^, 1 for *, true for &&, false for ||, written as a literal — is `parallelReduce` folds every block from its identity, and the identity of `+` is `0`, not `1`: any other value would be counted once per block (tests/cases/reject_par_reduce_identity). A named function is opaque: its associativity, and the identity it is given, are its author's promise, and a wrong one gives a deterministic answer that is not the fold.
  • Importing nish/threads compiles the program with --threads, because every worker bumps an arena of its own: @nish_arena is declared thread_local in every module and the runtime is built with -DNISH_THREADS (tests/link/par_map). A program that does not import it is compiled exactly as before.

What makes a region safe to run on several threads is checked at the call, from the whole-program facts, and each rule is reported there:

  • The function writes nothing its caller can see. A store through a parameter, a call that makes one, or a runtime call that writes — console.log included — is `bump` writes memory its caller can see at main.ts:10:3, and `parallelMapInto` runs it on several threads at once: a parallel body may read what its caller owns and write nothing but its result, naming the write, or the callee that makes it (tests/cases/reject_par_shared_write, reject_par_shared_write_via, and reject_par_console_log, which names the runtime's nish_print). A method of an atomic — add, max and the rest — is the one shared write a body may make (tests/link/par_atomic). A bounds check, an integer division, the length check of new Array(n) and panic are not writes: each prints and exits, and nothing runs again to see memory. When several threads panic at once, the first claims the program: it prints its whole message and exits 1, and every later one waits for that exit without printing, so a failing run prints one message, never two or a mix of them (tests/link/par_panic_once, thread_panic_once). Output is a write — console.error and process.exit as much as console.log (reject_par_console_error, reject_par_process_exit). Reading a shared Map or Set that an element or a capture reaches, with get, has or size, is a read (tests/link/par_map_table_reads); set, delete, clear and iterating one, whose walk counts itself on the table, are writes (tests/cases/reject_par_map_set, _delete, _clear, _values). Handing a helper a number, a boolean, an enum or a string — a string element of the caller's array included, verify(hashes[k]) — gives it no slot to store into, so it is a read (tests/link/thread_bcrypt_batch, whose task verifies against the parent's hashes). The instance performs every store itself, into slots the partitioner has made disjoint, so a function that writes nothing cannot race.
  • A store into an array only this call reaches is not a shared write. A const the function binds to its own array literal or new Array(n) — one whose elements are not stored inline — is private while every use of it is an element, a store into a slot, .length, an array method, a for...of, or an argument of a function that keeps nothing of it. Such a function may fill it, push onto it, or hand it to a helper that writes through its parameter, and still write nothing its caller can see: the helper's writes through that parameter land in the private array (tests/link/thread_task_private, whose body takes its element apart into a scratch array). Returned, stored, pushed, bound to a second name (in any declarator), named in an arrow, or handed to a function or a constructor that keeps it, the array is shared, and so is every store into it (tests/cases/reject_par_private_kept; the scoped-task twins below). What a private array's slots hold is not private: a store into rows[0][0], or a helper that makes one, writes what rows[0] reaches (reject_par_private_helper_deep).
  • Nor is a write into an object's own slots, when the object is one this call made. A const bound to the function's own new or object literal is memory no caller, thread or task can see while the call runs, so a method or a function that writes only that object's own fields — scalar state such as an Rng's — writes nothing shared: const r = new Rng(seed); r.nextI32() is a body's own business (tests/link/std_random). That is the whole of what is admitted. A write below the object's own slots, into a field or an element of it, counts as a write into whatever it may hold, and once a constructor, a store, a push, a Map or a Set has put the caller's memory there, or memory of unknown origin, that write is refused as any shared write is: hs.push(box); hs[0].bump(), h.b = box; h.b.bump(), a constructor that keeps box, a method that forwards to a field's method, o.a.b = box, and the same with a captured Rng (tests/cases/reject_thread_task_private_object_element, _field, _ctor_alias, _capture, _map, _set, _nested, _array_own, _forward, _forward_store, _array_rng, and reject_par_private_object_field_rng, _array_rng, _for_of, _held, _kept). Freshness is not followed through a call: an object a helper made and returned is not one this call made.
  • What the function allocates dies with its element. A function may build strings, arrays and objects on the way to its result. They are given back after every element: the function gets an arena scope of its own, so each call marks the arena of the thread it runs on and releases it before it returns, and a map over a million elements holds one element's temporaries at a time (tests/link/par_alloc). Under glibc, a program that has divided a region or joined a scope on other threads keeps the memory a release gives back rather than returning it to the kernel, so an element or a task that builds a 1 MB temporary does not fault it in again on every call: 200 tasks that each build one took 48 ms on 1 thread, against 314 ms when every release trimmed the heap. A worker that then finds no work for a second gives the memory back to the kernel before it parks, its own heap included: after ten rounds of 8 tasks that each build a 16 MB temporary, an idle program holds 2.3 MB rather than 68 MB, and after four maps of 1.2 million strings on four threads, 2.3 MB rather than 82 MB (tests/link/par_map_idle). With no worker (one CPU, or NISH_NUM_THREADS=1) nothing parks, so nothing gives memory back early: what a join freed is kept up to the 64 MB at which glibc returns the top of the heap by itself. A region below its grain runs as the loop it would have been, on the calling thread, and keeps that loop's costs: 200 elements that each build a 1 MB temporary take some 320 ms, as the loop does. It compiles with a performance warning, NL9012, because every element pays for the allocation and the release: the body of this `parallelMapInto` allocates per element: `label` answers `i32` but allocates on every call, so each thread marks and releases its arena around every element. Compute the answer without building a string, an array or an object to save both. Two things make that release unsound, and each is refused. An allocation stored into memory, which the escape analysis stops following, is `width` allocates at main.ts:21:21 and stores the allocation into memory, and `parallelMapInto` releases what a body allocates after every element: a parallel body may allocate only temporaries it drops before it returns, and this analysis stops following a value once it is stored (tests/cases/reject_par_alloc_escapes). And a function that reads or moves the arena would see the arena of whichever thread ran the element: `used` reads or moves the arena, and `parallelMapInto` runs it on several threads that each have an arena of their own: a parallel body may not call `Arena.mark` or `Arena.used`, nor open a `using a = arena()` block, nor call `region`, `regionWith`, `child`, `childWith`, `enter`, `enterWith` or `free` (`nish:region`) (tests/cases/reject_par_arena).
  • A map's result U is a number, a boolean, an enum, a string, or an array of numbers, booleans or enums, because the element's memory is released before the next element runs and a worker's arena is freed when its thread exits. A string or a scalar array crosses by copy-at-join, and the caller then adopts it: as the chunk loop rewinds its arena after each element, it moves the answer to the end of the chunk's transfer block, a malloc'd block of 4 KB doubling to 64 KB, or of the answer's size when that is larger (nish_cross_keep). Once every chunk has finished, blocks holding 64 KB or more in all are linked into the caller's arena's chunk list (nish_cross_adopt), so the answers stay where they are and the join costs one link per block rather than a copy per element; less than that is copied into the caller's arena and the blocks freed, so a small map run in a loop does not leave a chunk behind on every run (tests/link/par_map_join_small). Adopted blocks are then the caller's arena memory like any other, given back when the caller's scope releases what the result was made in, exactly when a copied value would have been (tests/link/thread_adopt_lifetime); a result the body did not allocate — a literal, a string it was handed — is not moved at all (tests/link/par_map_string: parallelMap, parallelFrom and parallelMapInto into a string[], under churn()). Anything else — an array of strings, of arrays or of objects, a Map, a Set, a class — is `name` answers `string[]`, and `parallelMapInto` hands back only a number, a `boolean`, an enum, a `string` or an array of numbers, booleans or enums, which the join copies out of the worker's arena: anything else would point into memory the worker frees or reuses (NL2348, tests/cases/reject_par_result_type; reject_par_map_result_strings for parallelMap), because its copy would have to follow every element. parallelMap and parallelFrom allocate their result before the region, so theirs is refused there, before std/threads.ts is instantiated (reject_par_map_new_result_class, reject_par_from_result_type). A map of 1M short strings, `s${x % 100000}`, took a median of 23 ms on 4 threads and 24 ms on 8, against 38 ms for the same loop written sequentially, and peaked at 32 MB against the loop's 55 MB. On 1 thread it is the loop's time: 54.9 ms against the loop's 57.6 ms and its copy's 56.4 ms, minima of 9 interleaved rounds on one machine. It used to take 72.8 ms there, because the mark sat at the end of the full chunk the result had filled, so every element's body allocated a fresh 64 KB chunk and the rewind freed it; the rewind now keeps that chunk, emptied (tests/runtime-test.c's test_cross_keep_rewind). Before adoption, when the caller copied every string into its arena one at a time, the map took 56 ms on 4 threads and peaked at 67 MB; two byte-identical copies of that binary differed by 5 ms in the same 7 interleaved rounds. What the body allocates for an element is released with that element, also when the body stores what it builds into an array literal on the way, which the arena scope above does not cover: the chunk loop marks the arena before each call and keeps only the answer (nish_cross_keep moves it to the transfer block and rewinds to the mark; on the sequential path, below a map's grain, it copies it out and back). A body that builds a 2 KB string, stores it in an array literal and answers a 4-byte slice of it, over 1M elements, peaks at 32 MB on 1 thread and on 8 (63 and 67 MB before adoption), where it used to hold every element's temporaries until its chunk ended and peaked at 2.1 GB (tests/link/par_map_release).
  • A frame that runs a map gives its result back when it returns. A function that calls parallelMap, parallelFrom or parallelMapInto and lets nothing the map made outlive it frees the result array, and the blocks a string map adopted from its workers, when it returns, as the same loop written sequentially does. A result that is returned, stored into a field or an outer array, captured by a later task, or nested in an array of arrays is kept with whatever holds it (tests/link/par_map_reclaim_escapes). A function that maps 2^22 elements and answers one of them, called 40 times on 4 threads, takes 140.3 ms and peaks at 33.8 MB, where it took 493.3 ms and peaked at 658 MB: 1.87x its sequential loop (262.3 ms) instead of 0.56x (main against a byte-identical copy of itself: about 2%). One call over 2^25 elements is unchanged, 251.3 against 252.5 ms, inside that band (tests/link/par_map_reclaim). A string map's result handed to a function whose string[] parameter keeps none of it, reading it or handing it on only to another such parameter, is still freed with the frame: 72 string maps over 2^17 whose frames each pass the array to such a callee peak at 20 MB on 1 thread and 22 MB on 8, where they peaked at 237 MB. The parameter keeps it, and the frame keeps the strings, when the callee stores the array or one of its strings into a field, an outer array or a Map, returns either, passes it to new or to a constructor that hands this out, stores it through any declarator of a statement or through a local alias, or hands it to a callee that keeps it, and whenever the callee holds an arrow, such as a map's body; the proof covers a parameter typed string[] exactly, so one typed string[][] keeps the strings it reaches (tests/link/par_map_reclaim_strings).
  • A reduce's T stays a number, a boolean or an enum. It is the element, the identity and every partial as well as the answer, folded on several threads, so a string would be copied at every combine; a reduce over strings or objects is refused at the call: `(a, b) => ...` answers `string`, and `parallelReduce` hands back only a number, a `boolean` or an enum: a worker's arena is freed when its thread exits, so anything else would point into freed memory; answer a number and fold those, or run it on the thread that called it, `let joined: string = ""; for (const b of words) { const a = joined; joined = a + b }` (NL2348, tests/cases/reject_par_reduce_result_type, and reject_par_reduce_result_class for a class). Map the strings to numbers first — parallelReduce(parallelMap(names, (s) => s.length), ...).
  • dst is not reachable from an element of src. A function that only reads can still read dst through its argument while another thread writes it, so an array of dst's element type reachable from T, through fields, elements and | null, is `(r) => ...` can reach a `f64[]` through `r.cells`, which could be the array `parallelMapInto` is writing: another thread would be writing it while this one reads, so `dst` may not be reachable from an element of `src` (tests/cases/reject_par_reaches_dst). It is judged by type, so such an array is refused whether or not it is dst at run time. A global Map or Set reaches only what it stores: its own arrays are private to it, so a Map<string, i32> beside a dst: i32[] is not refused (tests/link/par_map_read_into, par_map_read_into_keys, par_map_read_capture), while a Map<string, i32[]> is, through x.index.get(k) (tests/cases/reject_par_map_value_dst). A program's own class named Map is walked like any class (tests/cases/reject_par_map_user_class). A reduce writes no array of its caller's and has no such rule.

A data-parallel arrow reads its function's const locals

An arrow passed to one of the four forms may read the const locals and the parameters of the function it is written in, and so may an arrow given to a scope's spawn, by the rules of scoped tasks. An arrow given to any other function-typed parameter of a top-level function captures too, by the sequential rules of function parameters (tests/cases/fnarg_capture_local); the rules below are the stricter ones a body that runs on other threads keeps.

import { parallelMapInto } from "nish/threads";

export const main = (): i32 => {
  const k: i32 = 3;
  const table: i32[] = [10, 20, 30, 40];
  const xs: i32[] = [1, 2, 3, 4];
  const dst: i32[] = [0, 0, 0, 0];
  parallelMapInto(xs, dst, (x) => table[x % 4] * k);   // 60 90 120 30
  return dst[0];
};
  • Each name the body reads is a parameter of the body. The checker appends it to the arrow's parameters after the ones it wrote, and the call passes the local's value through every instance on the way to the body and through the region's context with the other arguments, so a scalar is copied and a reference is passed as the pointer it is (tests/link/par_capture, and tests/link/par_capture_large, whose regions divide). A narrowing in force at the call holds in the body, because a const cannot change: if (w !== null) around the call lets the body read w.scale. No closure is made: the arrow is still a function of its own, with a direct call to it.
  • A captured reference is read-only. A store through it — an element, a field, push, pop, fill or set, through a second name the body binds to it, or through an object or an array of the body's own that holds it — is a store into memory the caller owns, and is refused as any shared write is (reject_par_capture_element_write, _field_write, _push, _pop, _fill, _set, _map_set, _alias, _alias_object, _alias_literal, _nested_array, _param_write, and _reduce_write and _into_write for the other forms). So is handing it to a function or a method that writes it (_helper, _method). Assigning to the name itself is The arrow assigns to `k`, which belongs to the function it is written in: a data-parallel body reads a captured value and never writes it, because every thread reads the same one; answer the value from the body instead, and read the answers from the array the call gives back, `const kValues = parallelMap([1, 2], (x): i32 => x)` (reject_par_capture_assign, and reject_par_capture_update for a k++ after a read). A reducer that assigns to one is shown the same parallelReduce without the write, answer the value from the reducer instead, and let `parallelReduce` fold it, `const callsTotal = parallelReduce([1, 2, 3], (a, b): i32 => a + b, 0)`.
  • A let is not captured, since it could change between the call and a thread's read of it: The arrow reads `m`, a `let` of the function it is written in: a data-parallel body may read only a `const` or a parameter of that function, whose value no thread can change while the body runs; bind the value to a `const` before the call, `const mNow = m`, and read `mNow` in the arrow (reject_par_capture_let).
  • Only an arrow written directly in a function's body captures. An arrow inside another arrow reads nothing from outside itself, neither the function's locals nor the outer arrow's parameters: The arrow reads `k` from outside the arrow it is written in: only an arrow written directly in a function's body may read that function's `const` locals and parameters, and an arrow inside another arrow reads only its own parameters and the module's top-level names; move the inner call into a top-level function that takes the value as a parameter (reject_par_capture_nested_arrow, reject_par_capture_nested_param).
  • this is not captured by that name, nor a field through it; bind it, or what the body needs of it, to a const first: The arrow reads `this`: a data-parallel body may read the `const` locals and the parameters of the function it is written in, and `this` is neither; bind it, or what the body needs of it, to a `const` before the call (`const self = this`, `const k = this.k`) (reject_par_capture_this). This is a matter of spelling, not of safety: a capture becomes a parameter of the arrow, and a parameter named this is what makes a function a method to every analysis that reads one. A const bound to this is captured as the object reference it is, read-only like any other (const self = this, in tests/link/par_capture).
  • A captured name is read at the call. The body runs where the call is, so the rules that follow a local of the function through its reads count a capture as a read of that local there. Three refuse: a destination of a scope's task read through a capture before the scope joins, by any of the four forms and on a loop's later pass (reject_par_capture_scope_dst_index, _indexof, _for_of, _large, _map, _map_into, _reduce, _loop), an element reference read through a capture after a push or a pop may have moved its storage (reject_par_capture_stale_record, _into, _loop, _pop, _for_of, _helper), and an array read through a capture after secret(raw) took it (reject_par_capture_secret_moved), each with the rule's own message. Two warn under --warn-portability, at the read inside the arrow as at a direct one: a record copied into an array and written through one copy (NL8003, tests/link/port_rec_capture_copy, and port_rec_capture_quiet for a read before the write), and a byte offset compared with a count or answered by the arrow, which stores it in the form's array (NL8001, tests/link/port_str_capture_offset, port_str_capture_result). A captured name is not a parameter of the arrow to NL8009, which reports it where its function declares it (tests/link/port_num_capture_wide).
  • Some values are never captured: a Mutex, its guard, a Channel or a scope, which are shared only by their own block's rules, a Result, a value that may be absent, a Secret, or anything that reaches one of them: The arrow reads `m`, which is `Mutex<i32>`, and a data-parallel body cannot capture it: a scope, a lock, a guard or a channel is shared only by the rules of its own block (reject_par_capture_mutex, _guard, _channel, _scope, _result, _maybe, _secret; _mutex_field for a lock in a field of a captured object, _mutex_map for one in a Map's values). A body that runs a data-parallel form of its own is refused as a shared write, captures or not (reject_par_capture_nested_parallel).
  • dst is not reachable from a capture either. A captured array of dst's element type, or a captured value that reaches one, could be dst, which other threads are writing while the body reads it: `(x) => ...` reads `dst`, and can reach a `i32[]` through `dst`, which could be the array `parallelMapInto` is writing: another thread would be writing it while this one reads; a captured array is known not to be `dst` only when one of the two is a `const` bound to an array of its own, such as a literal or a `new Array`, and the other is another such `const` or a parameter (reject_par_capture_dst, _dst_alias, _dst_field, _dst_param; _self_field_dst for self.out while this.out is dst, and _generic_same_dst, refused in each of two instantiations). A capture is a binding the call can name, so one case is admitted by identity: a captured array and a dst that are two different bindings, one of them a const bound to an array of its own and the other another such const or a parameter of the function, as table and dst are above. A fresh array is memory no binding held before it was made, and a parameter's value was fixed before the function ran (tests/link/par_capture, where a parameter table, and two fresh consts in a generic function, are admitted). Two parameters may be one array and are refused. parallelMap and parallelFrom allocate their own dst, which no capture can reach.
  • A fresh dst nothing was handed is apart from every capture (#612). When dst is a const bound to an array of its own, and nothing before the call — or in a loop around the call that does not also declare it, or in a function written inside — names it except to index it or read a member, then no capture can reach it: the only way into a value is through a binding, and none was given this one. So the body may read a parallelMap result, a config object or a const self = this beside it (tests/link/par_into_fresh_pipeline, par_into_fresh_object); indexing it, reading d.length, calling d.fill(0) or walking it with for...of first hands it to no one (par_into_fresh_reads). Anything that hands dst on first is refused with the message above: a store p.arr = d or this.tab = d (reject_par_into_store_field, _this_field), an argument keep(h, d) or ts.push(d) (_helper, _push), an alias const e = d (_alias), a branch c ? d : w (_ternary), a holder new Holder(d) (_holder), a store after the call inside a loop around it (_loop_store, and _loop_store_while, _do, _for_of for the other loops) — the outermost such loop, so a store after an inner loop inside an outer one counts (_nested_loop) — a hand-on on one branch of an if (_if_branch), an earlier call writing the same dst (_twice), dst passed as the call's own src (_as_src), and the arrow reading dst itself (_captured). A dst bound to a captured array is no fresh const (_from_capture). The rule asks for the one binding of that name in its function, so a dst whose name is declared twice there keeps the stricter rule above (_shadow).

Scoped tasks: using s = scope()

A scope runs a handful of different functions at once, each on a thread of its own, and joins them all when the block that declares it ends (wp29-thread-surface.md §4.2). It is the shape parallelMapInto cannot take: work that is not one function over one array.

import { scope } from "nish/threads";

const sumOf = (xs: f64[]): f64 => { /* ... */ return 7.0; };
const primesBelow = (n: i32): i32 => { /* ... */ return 168; };

export const main = (): i32 => {
  const xs: f64[] = [1.5, 2.5, 3.0];
  const sums: f64[] = [0.0];
  const counts: i32[] = [0];
  {
    using s = scope();
    s.spawn(sumOf, xs, sums, 0);           // sums[0] = sumOf(xs)
    s.spawn(primesBelow, 1000, counts, 0); // counts[0] = primesBelow(1000)
  }                                        // both have run, and both answers are stored
  console.log(`${sums[0]} ${counts[0]}`);  // 7 168
  return 0;
};
  • s.spawn(entry, arg, dst, at) runs entry(arg) on a thread of its own and stores the answer in dst[at] (tests/link/thread_scope_basic). entry is a function parameter: a top-level function named at the call, or an arrow written there, which may read its function's const locals (below). at is checked at the spawn and again when the answer is stored; out of range is spawn: destination index 3 is out of range for an array of 1 elements and exit 1.

  • s.spawn(entry, arg) and s.spawn(() => ...) have no destination. The task runs as any other does and its scope stores nothing when it joins, so entry answers void: it is seen through the guard of a Mutex its argument or a capture reaches, or through a Channel it sends on (tests/link/thread_task_void, thread_task_arrow). A block body is void without an annotation. An entry that answers a value is `sumOf` answers `i32`, and a `spawn` without a destination stores nothing: a task's answer is kept only by `s.spawn(entry, arg, dst, at)`, which stores it in `dst[at]` when the scope joins, so make the task answer `void` or give it a destination, `const answer2 = new Array<i32>(1)` above the scope's block and `s.spawn(sumOf, xs, answer2, 0)` (NL2511, tests/cases/reject_thread_task_void_result, and _arrow for () => sumOf(xs)). TypeScript would spell the three forms as overloads; the checker picks ThreadScope's method by the number of arguments, spawnWith for two and spawnThunk for one, and under Node every form reaches spawn, which runs a task without a slot at once (RUN_UNDER_NODE.md).

  • An arrow task reads its function's const locals and parameters, which is the whole spelling of a task that needs more than one value:

    {
      using s = scope();
      s.spawn(() => binInto(xs0, hist));   // xs0: a `const` array; hist: a `Mutex`
      s.spawn(() => binInto(xs1, hist));
    }
    

    Each name the arrow reads is appended to its parameters, as a data-parallel arrow's is, and copied into the task's payload at the spawn beside its argument; the task reads it on its own thread when the scope joins. A capture is the task's argument to every rule of this section: the parent may not write what it reaches before the join, however it does so — a store, a second name for it, a helper, a new whose constructor writes it or keeps this in it, in a scope bound by any declarator of its using (tests/cases/reject_thread_task_capture_parent_write, _alias, _helper, _new, _this_escape, _declarator); a destination is never captured (_dst); the task writes nothing its caller can see except through a guard, a constructor it runs included (_task_write, _task_new, _task_this_escape); a Mutex it captures is one the parent may not lock before the join (R7, _mutex_lock) and that it may reach only as a capture or a field of one (R8, The task's arrow reads `o`, which is `Outer` and reaches a `Mutex` through `o.inner.m`: a captured value is the task's argument, …, NL2510, _mutex_deep); the parent's orReturn before the join is refused (NL2394, _or_return); a captured element reference is read at the join, so a push that may move its storage before the end of the block is NL2290 (_stale_record, _stale_join, and reject_thread_task_arg_stale_join for a task's argument, read at the join the same way); and an array moved into a Secret is not read again (NL2445, _secret_moved). A let is not captured, since it could change before the task runs (NL2506, _let), nor this (NL2507, _this; a const self = this is captured as the reference it is, tests/link/thread_task_arrow), and a captured name is never assigned (NL2508, _assign). A guard, a channel, a scope, a Result, a value that may be absent and a Secret are never captured: The arrow reads `ch`, which is `Channel<i32>`, and a task cannot capture it: a guard, a channel or a scope is shared only by the rules of its own block; a task reaches a channel through its argument, `s.spawn(entry, arg)`, where the scope counts its senders (NL2509, _guard, _channel, _secret). An arrow inside a task's arrow reads only its own parameters and the module's top-level names (NL2501, _nested). The arrow is lifted into a function named after the one it is written in and a counter (main$arrow0), which is the name a debugger or a profiler shows for the task's thread.

  • A task runs when its scope joins, not when it is spawned. Joining a scope runs every task it was given at once — the first on the thread that opened the scope and each other on a thread of its own, so N tasks start N - 1 threads — waits for all of them, and then stores each answer in the order the tasks were spawned, on the thread that opened the scope. That is what makes a scope free of races: while the tasks run, the thread that opened the scope is waiting for them, the tasks write nothing another can see (below), and every store into the program's memory is made after the last task has finished. So a task may be handed any argument — an array, an object.

  • scopeWithStack(bytes) is a scope whose tasks have a stack of at least bytes bytes, for a task that recurses deeper than a thread's default stack allows (8 MiB on Linux): using s = scopeWithStack(64 * 1024 * 1024) (tests/link/thread_stack_deep, a million calls deep on 256 MiB; under an 8 MiB ulimit -s the same program faults with scope(), and with scopeWithStack(131072) on its guard page, tests/link/thread_stack_fixtures). It covers a scope's tasks only: parallelMap, parallelFrom and the other data-parallel calls run on the pool's workers, whose stack is the platform's default. It is a scope in every other rule. Its tasks all run on new threads, a thread each, none on the thread that opened it; each thread's stack is bytes rounded up to the page, with a guard page below it, so a task that still overflows faults rather than writing past it. If one cannot be started the program ends with exit 1 and nish: cannot start a thread for scopeWithStack: out of threads or of memory for its stack when the system is out of either, or nish: cannot start a thread with the stack scopeWithStack asked for when it refuses the size. The smallest size is 131072 (128 KiB, the largest PTHREAD_STACK_MIN among the targets): a number written at the call below it is refused, `scopeWithStack(4096)` asks for a stack below the minimum of 131072 bytes: no target can start a thread on less, and a task that ran on it would overflow before it did any work; ask for at least 131072, `scopeWithStack(131072)` (NL2584, tests/cases/reject_thread_stack_small, _negative), and a computed one panics when the scope is made, scopeWithStack: a stack of 65536 bytes is below the minimum of 131072 and exit 1 (tests/link/thread_stack_computed_small). Under Node, on WASI and wasm32 (no threads: every task runs on the one thread, in spawn order) and in a build without --threads, the size is checked the same way and has no other effect, so a program that needs the stack needs it from Node too (node --stack-size).

  • A destination is a fresh const array that is never handed on, and until the block ends the scope's thread neither reads a destination nor writes what a task may read. Natively a task runs, and its answer is stored, when the scope joins; under Node each runs and stores where it is spawned. Two rules keep the program's meaning the same either way:

    • A spawn's destination is a const local bound to a literal array or new Array(...) in the same function, used only by index, .length, its methods, a for...of and as a destination — never an argument, a task's argument, a stored value or a returned one — so nothing else can reach it. Anything else is The destination of this `spawn` must be a `const` local bound to a fresh array (a literal or `new Array`), used only by index, `.length`, its methods and as a destination: a scope's tasks run, and store their answers, when its block ends, and under Node each runs and stores where it is spawned, so a destination is a fresh `const` array that is never handed on, and between a scope's first `spawn` and the end of its block its thread neither reads a destination nor writes memory a task may read (tests/cases/reject_thread_region_dst_not_fresh, and reject_thread_region_dst_as_arg for one handed to a later task).
    • From the first statement of the block that holds a spawn on the scope to the block's end — a return's value aside, computed after the join — the program may not read a destination a task of the scope has been given by then (This reads `sums`, a destination of a task of `s`, before the scope joins: …, tests/cases/reject_thread_region_read), and may not write memory a task could read. A task reads only what its argument reaches, so a store is fine into the own slots of a const bound to a fresh array or object literal that is never handed on, and a call is fine unless it writes through a pointer it is handed; a function that only fills an array it allocates, such as one that builds and returns a row, writes nothing a task can see. Every other write is refused, whatever alias it goes through — a store, push or pop, Arena, a call that writes through its argument, a task of another scope, whose join stores (reject_thread_region_write, reject_thread_region_alias_write, reject_thread_region_call_write). A constructor is a call: a new whose constructor writes through an argument, directly or through a constructor it calls, is `new Bump` writes memory a task of `s` may read, before the scope joins: … (tests/cases/reject_thread_region_new_write, and _declarator for the second declarator of a statement, _nested for a constructor that calls one, _this for one that hands this to its argument, which is a store through that argument). A write counts however deep it lands in what the argument reaches: a field of a field, a field of an element, through a const that names one, a for...of variable over one, a helper, or a method of what a field holds (reject_thread_region_new_deep, _new_element, _new_helper, _new_method, _call_deep, _call_element, _call_alias, _call_loop). Its stores into the fresh object itself are not counted, so new Point(3, 4) is fine there (tests/link/thread_task_private). An object a function or a constructor made itself is its own memory, and a write into it counts against nothing, until memory an argument reaches may have got into it: stored into a field or an element at any depth, pushed, put there by a method or a function it was handed to, kept by its constructor from an argument, or any non-number value rooted at it — the object, a field or an element of it at any depth, a constructor's this or this.f — used as a value anywhere but a read, a comparison or an argument the checker follows into its callee: the right side of an assignment, a const's initialiser, an element of an array literal, a property of an object literal, { k: v } or { k }, an argument of push, of a Map or Set method or of a call, an arm of a ?:. Each position is refused for each form of the value, a local, a field, an element and this.f (reject_thread_region_escape_<position>_<form>, with positions assign, alias, array, object, shorthand, push, call, ternary and element, forms local, member, index and this_field, and _this for the bare this in every position but shorthand, which cannot name it). A Map or a Set keeps exactly what its methods are handed — set's key and value, add's element, getOrInsert's from nish/map — so each is a store into the table, as a push is (reject_thread_region_map_value_param, _map_fused_value_param, _map_own_subpath, _map_route_subpath, _map_route_value_param, _set_own_subpath), and its methods write only the table's own storage, never what it holds. What a container gives back is its element: what get or getOrInsert answers, and the const variable of a for...of over an array, a Set, or a Map's keys() or values(), is memory below that container, so a write through it counts against what was stored into the container, and a store through it is a store into the container (_readback_get, _readback_get_or_insert, _readback_nested, _readback_store_alias, _table_param_store, _walk_array_param, _walk_array_twice, _walk_values, _walk_keys, _walk_set_param, _walk_array_store, _walk_set_store). So a call that answers a table or what one holds — getOrInsert(m, …), m.set(…), s.add(…) — is a value rooted at that table, and escapes it wherever a name would (reject_thread_region_call_escape_<position>_<form>, with positions push, field, index, array and object, forms goi, set_chain and add_chain, and _ternary_goi, _nullish_get, _let_init_goi, _let_assign_goi, _table_arg_set_chain). Only a call that keeps what it is handed stores it — set its key and value, add its element, getOrInsert its key and default — so a lookup, has, get or delete, keeps nothing (reject_thread_region_table_key_set, _table_key_get_or_insert); and a number, a string or a boolean read back through | undefined, as m.get(k) ?? 0, names no memory. A lookup in an argument's table, has or get, reads it and writes nothing (tests/link/thread_scope_region_param_lookup). A ?: or a ?? stored somewhere may be either of its sides. Two values may be anything where they are written through: an x ?? y, and the variable of a walk of anything else — a Channel's messages are another task's. An object literal whose property is an argument's memory carries it however it is written (reject_thread_region_literal_object_param, _shorthand_param, _array_object_param, _field_object_param, _push_object_param, _new_object_param, _object_const_param, _object_method_param). A write below such a slot counts against the arguments whose memory may be there, or against all of them when that cannot be told (reject_thread_region_new_alias_field, _alias_push, _alias_element, _alias_const, _alias_helper, _alias_method, _alias_item, _call_wrap, _fresh_field, _fresh_method_bind, _fresh_attach, _fresh_push, _fresh_element, _fresh_array_field, _fresh_nested, _fresh_bump_method, _fresh_const, _fresh_ctor_field, _fresh_literal, _fresh_escape, _fresh_held, _fresh_held_temp, _fresh_captured, _this_local, _this_ternary, _this_literal, _this_object). A constructor that keeps what it is handed keeps it per argument, so one that keeps the caller's own array, or pushes an argument's object onto an array it made, writes nothing of the task's (tests/link/thread_scope_region_fresh_object, thread_scope_region_kept_own), and a call that only writes its argument, or returns an object of its own, keeps nothing of the caller's in it: w = build(h) on a function that returns a fresh object, and c.reset(h.nums[0]) on a fresh counter followed by writes into c (thread_scope_region_own_handoff). So a Map or a Set a function makes and fills itself writes nothing of the task's (thread_scope_region_map_set, thread_scope_region_map_get_or_insert, thread_scope_region_set_add), and so does one that walks, memoises or indexes a table of its own, or writes through what it reads back out of a container of fresh objects it made (thread_scope_region_own_tables, thread_scope_region_read_back). A map handed to set as a named const escapes, as any own object handed to a table does, so a write into it counts against every argument: build a nested table from what get reads back instead. A task argument read out of a variable — s.spawn(f, ps[0], out, 0), or either arm of a ?: or ?? — hands that variable on, so a store into its elements or fields before the join is refused too: a record element is the address of its slot, and the task reads it at the join (tests/cases/cg_sec_scope_record_arg). And r.orReturn() on a Result is refused in the same stretch, between the first spawn and the end of the block, because its early return would leave the scope unjoined (NL2394, tests/cases/cg_sec_scope_or_return).

    The rules are local because memory is reached only through parameters and what a function allocates: a module constant is a number, a boolean or a string, and nothing is static. N tasks over N arrays of the destination's own type, built before the scope, in the spawning loop or after the first spawn, all compile (tests/link/thread_scope_many_arrays).

  • The join is emitted at every exit of the block: its end, a return — before the returned value is computed, so the value can read what the tasks stored — and a break or continue that leaves the block (tests/link/thread_scope_exit_paths). It sits inside every arena scope around it, which releases only after the join, and an argument a task is handed is never given back by a loop's pass before the join reads it (tests/link/thread_scope_nested_arena, which also nests one scope inside another). Since the join comes first, spawning gives up no memory: a function, a loop's pass or a using a = arena() block that hands a task an array it made still gives that array back when it ends, after the join, unless the task keeps it — answers it, or stores it where it outlives the task — or the frame keeps it past its end by any other route (tests/link/thread_scope_reclaim, whose peak resident memory tests/run.js bounds at every thread count, and thread_scope_reclaim_escapes, which reads each route after the memory is reused). A panic ends the process and joins nothing (throw is refused by Phase 0, reject_throw).

  • A scope is introduced by using, and using takes only a scope, a Mutex's lock or an arena(). A scope bound any other way is `scope()` must be the initialiser of a `using` declaration: a scope joins its tasks when the block that declares it ends, so a scope bound any other way would be one nobody joins; declare it as `using s = scope()` (tests/cases/reject_thread_scope_not_using), and a using of anything else is `using` takes only `scope()` or a `Mutex`'s `lock()` from `nish/threads`, or the builtin `arena()`, in this version: they are the values whose disposal the language defines — a scope joins its tasks, a guard releases its lock, an arena releases what its block allocated — so a `using` of anything else would promise a disposal nothing performs (tests/cases/reject_thread_using_not_scope). A using declaration is a statement of a block; as the body of an if or a loop it is A `using` declaration must be a statement of a block, `{ ... }`: its scope joins when that block ends, and a single-statement body is not a block (tests/cases/reject_thread_using_placement), and in a case clause it is refused as every declaration there is. Outside a declaration using is an ordinary identifier, as in TypeScript (tests/parser/using.ts).

  • A scope never leaves its block. It is only ever the receiver of a spawn statement; passed, stored, returned or copied it is A `ThreadScope` can only be the receiver of a `spawn` statement: passed, stored, returned or copied, it could be given a task after its block has joined it (tests/cases/reject_thread_scope_escapes). That, and the join at every exit, is the whole proof that every task is joined: there is no handle to forget.

  • [Symbol.dispose] is nish/threads's alone. ThreadScope and MutexGuard declare one, so TypeScript and Node accept using s = scope() and using g = m.lock(); the compiler emits the join and the release itself and never calls it. Channel's [Symbol.iterator], Node's way into a for...of, is the one other computed name, and is nish/threads's alone too: anywhere else it is refused as every computed name is (NL1041). Any other class's is `Handle` cannot declare `[Symbol.dispose]`: in this version `using` takes only `scope()` or a `Mutex`'s `lock()` from `nish/threads`, or the builtin `arena()`, whose disposal the compiler emits itself, so a disposal method of any other class would never be called (tests/cases/reject_thread_dispose_elsewhere).

  • There is no thread count. Each spawn is one task and each task one thread; a scope with more tasks than cores leaves the scheduler to share them (wp29-thread-surface.md §9 declines the knob until a measurement asks for it).

What a task may do is what a data-parallel body may, less the rules about an element's arena, which a task does not share, and each rule is reported at the spawn:

  • The task writes nothing another can see, except through the guard of a Mutex, the one shared write a task may make, by a method of an atomic (tests/link/thread_atomic_tasks), or by a send or a receive on a Channel, neither of which is a write the rule counts: `bump` writes memory its caller can see at main.ts:10:3, and `spawn` runs it beside the scope's other tasks: make scratch only this task uses inside it, because an object handed in as the argument is shared memory even when the `spawn` made it, since where an argument was made is not tracked. Write memory the tasks share only through the guard of a `Mutex`; hand a value the parent needs back as the task's answer, or run it on the thread that called it, `out[0] = bump(c)` (NL2471; tests/cases/reject_thread_task_shared_write, and reject_mutex_task_unguarded for a task that also writes through a guard). A wipe is a write through its argument like any other, so it is shared exactly when its target is: an array the task, or a function it calls, allocated itself is private, and wiping it is allowed (tests/link/thread_task_wipe). So is a Secret the function makes with secret(v) and wipes, when v is allocated in the call — new, an array literal or an object literal — and the Secret is held by a const (tests/link/thread_task_wipe_secret). Wiping the task's argument, a captured local, a field, an element, an array a call answered, a Secret held by a let, made from a call's answer, from a local moved into it or from a record whose constructor is handed an object, a Secret a call answers (wiped, or written by a helper first), or one a holder keeps in a field, is NL2471 (reject_thread_wipe_param, _capture, _field, _element, _helper, _call_result, reject_thread_wipe_secret_call_result, _moved, _holder, _let, _call_secret, _call_secret_helper, _ctor_param). A helper may write the task's own Secret before the wipe. A borrowed Secret parameter is refused before it (reject_thread_wipe_secret_borrowed). The wipe itself is unchanged, one volatile llvm.memset. A lookup in a shared Map or Set — get, has, size — is a read, so tasks of one scope may look keys up in the same table (tests/link/thread_scope_task_map_reads). A mutator — set, add, delete, clear, getOrInsert — writes it, a walk counts itself into it (walkOpen), and what a lookup answers is the table's memory, so each is refused (reject_thread_task_map_set, _map_delete, _map_clear, _map_get_or_insert, _set_add, _set_delete, _set_clear, _map_walk_keys, _map_walk_values, _set_walk, _map_readback_write). A task may panic, as a parallel body may: the first panic of the program prints and exits 1, and a task that panics after it prints nothing. Output stays a write (reject_thread_task_console_error, reject_thread_task_process_exit), and so does the parent's: between the first spawn and the join it may neither change a Map or Set it holds nor walk one — for (const k of s), m.keys(), m.values(), or a helper that does (reject_thread_region_parent_table_set, _set_walk, _map_keys_walk, _map_values_walk, _helper_walk). The rule is conservative: it judges the call by what it writes through its receiver, not by which tables a task holds, so a walk of a table no task can reach is refused too (reject_thread_region_parent_unreached_walk), and so is a mutator or a walk called straight on a table made in the block. Proving that no task reaches one table would mean following every pointer each task is handed; the walk moves before the first spawn, after the block, or into a helper that makes and fills its own table (thread_scope_region_own_tables). A store into an array the task allocated and hands to nothing that keeps it is not counted, as for a parallel body: a sieve may fill its own table, and a helper may fill the array it is handed (tests/link/thread_task_private). Returned, stored into its argument's field, pushed onto its argument's array, kept by a helper or a constructor, or bound to a second name in a later declarator, the array is shared (tests/cases/reject_thread_task_private_returned, _field, _pushed, _kept, _new, _declarator). A constructor that hands this out can lead to the array only by keeping it, so that route is _new's. A store through a ?: with the array in one arm is a store into either (_ternary), and an element of a private array of arrays is the array it holds (reject_thread_task_private_nested). The same holds for the task's other own memory: a method that writes only the fields of an object the task made, const h = new Holder(); h.poke(); set, add and delete on a Map or a Set it made; a store into an array field of an object it made, whose constructor filled it with a fresh array; a second name for its array bound in the same statement, xs = new Array<i32>(n), ys = xs; and a store through what a helper answers when the helper answers its argument, the task's own array, all write nothing shared (tests/link/thread_task_own_writes). What the argument reaches is not the task's own, even when the spawn made it there, because where an argument was made is not tracked: an object the task makes that holds the argument's array from its constructor on, has a field re-pointed at it, or is handed it by a helper or a method, and a helper that may answer either its own array or the caller's, all write the caller's memory (reject_thread_task_own_holds_param, _repointed, _helper_stores_param, _method_adopts_param, _either_returned), and so does a second name bound in the same statement as a fresh array when it names the argument's (reject_thread_task_own_declarator_binds_arg). A helper's fresh answer that holds the caller's memory, [ys] or new H(ys), is the task's at its top level only, so a write below it is refused (reject_thread_task_own_fresh_holds_array, _new, _object). That refusal starts with what to write: make scratch only this task uses inside it, because an object handed in as the argument is shared memory even when the `spawn` made it, since where an argument was made is not tracked. Write memory the tasks share only through the guard of a `Mutex`; hand a value the parent needs back as the task's answer, or run it on the thread that called it, `out[0] = bump(c)`. A task inside a task, or inside a data-parallel body, is refused by this rule, because a scope's join stores into memory (tests/cases/reject_thread_task_nested_spawn); and a data-parallel call inside a task is refused by it too (reject_thread_task_nested_map).
  • It answers a number, a boolean, an enum, a string, or an array of numbers, booleans or enums, because a task's thread frees its arena when it exits. A string or a scalar array crosses by copy-at-join: the worker copies its answer into a malloc'd transfer block before its arena goes (nish_cross_out), and the parent, once the scope has joined, copies each block into its own arena and frees it (nish_cross_in), so the block lives only between the two and is freed on every path out of the scope. An answer the task did not allocate — a literal, or a string its argument already held — passes through uncopied (tests/link/thread_task_string: a built label, a field of a made object, a substring and a slice, a returned fresh array, and a task that also stores into a Mutex, all read after churn()). Anything else is `words` answers `string[]`, and `spawn` hands back only a number, a `boolean`, an enum, a `string` or an array of numbers, booleans or enums, which the join copies out of the worker's arena: anything else would point into memory the worker frees or reuses (NL2348, tests/cases/reject_thread_task_result_strings; reject_thread_task_result_type for a class, reject_thread_task_result_nested for i32[][] and reject_thread_task_result_map for a Map). What a task allocates on the way is freed with its thread.
  • It leaves the arena alone, because the first task runs in the arena of the thread that opened the scope: `wipe` reads or moves the arena, and `spawn` runs it on several threads that each have an arena of their own: a parallel body may not call `Arena.mark` or `Arena.used`, nor open a `using a = arena()` block, nor call `region`, `regionWith`, `child`, `childWith`, `enter`, `enterWith` or `free` (`nish:region`) (tests/cases/reject_thread_task_arena).
  • Its argument is not a Result small enough to travel as its parts: The argument of `spawn` is `Result<i32, i32>`, which a task cannot be handed: a `Result` is passed as its parts rather than as one value, so hand the task the value it holds (tests/cases/reject_thread_task_result_arg).

Under Node, scope() is an object whose spawn runs its task at once and stores the answer there; the rule above is what makes that print the same as the compiled program (RUN_UNDER_NODE.md). using needs --js-explicit-resource-management on Node 22 and is native from Node 24.

parallelFor, non-scalar results and scope(n) are not in this stage (wp29-thread-surface.md §7); the lock is the next section, atomics the one after it, and channels the one after that.

A lock that owns its data: Mutex<T>

A Mutex lets a scope's tasks share one value they all update: the data lives inside the lock, and the only way to it is the guard lock() answers, bound by using, which holds the lock until its block ends (wp29-thread-surface.md §4.3). It is Rust's Mutex<T>, narrowed until no borrow checker is needed.

import { Mutex, scope } from "nish/threads";

class Counter {
  n: i32 = 0;
}

const addMany = (total: Mutex<Counter>): i32 => {
  for (let k: i32 = 0; k < 100000; k++) {
    using g = total.lock();          // waits for the lock, and holds it to the block's end
    g.value.n = g.value.n + 1;
  }                                  // released here, on every pass
  return 0;
};

export const main = (): i32 => {
  const total = new Mutex<Counter>(new Counter());
  const done: i32[] = [0, 0, 0, 0];
  {
    using s = scope();
    for (let t: i32 = 0; t < 4; t++) {
      s.spawn(addMany, total, done, t);
    }
  }
  using g = total.lock();
  console.log(`${g.value.n}`);       // 400000
  return 0;
};
  • Natively the lock is one word inside the guard, taken with an acquire compare-and-swap in lock() and given back with a release store, so a lock nobody holds costs one atomic instruction and allocates nothing; a lock() that finds the word held calls the runtime's nish_mutex_wait, which spins briefly and then yields until it can take it (tests/link/thread_mutex_counter, whose threads.ll is the lowering, and tests/link/thread_mutex_histogram). On wasm32 a scope's tasks run one after another, as they do for every scope there, so no lock is ever found held: the same program links for --profile wasi, its atomics lowered for a module without shared memory, and prints the same total (tests/run.js, "the --profile wasi build").
  • The release is emitted at every exit of the guard's block: its end, a break or continue that leaves it, and a return or an orReturn — after the returned value is computed and before control leaves, unlike a scope's join, which runs before the value, so return g.value.n reads under the lock (tests/link/thread_mutex_or_return, whose main.ll pins the order) — and inside an arena() block it is given back before the arena releases (tests/link/thread_mutex_exit_paths, which locks again after each exit and would wait on itself if one kept the lock).

Eight rules keep every path to the data behind the lock, and a program that breaks one is refused with that rule's own code:

  • R1: the data is fresh all the way down, and only the guard reaches it. new Mutex<T>(init) takes a scalar, or a new expression or a literal whose every argument, element or value is a number, a boolean, an enum, a string literal — immutable, and never stored over — or itself fresh by this rule, because a value the program already holds would be a path to the data that skips the lock: `new Mutex` takes a value that is fresh all the way down — a `new` expression or a literal whose every argument or element is a number, a `boolean`, an enum, a string literal or itself fresh — because the data a `Mutex` owns is reachable only through its guard, and a value the program already holds would be a path to it that skips the lock (NL2463, tests/cases/reject_mutex_not_fresh, reject_mutex_not_fresh_deep for new Hist(bins) with a named bins, and reject_mutex_not_fresh_literal for [row] with a named row; new Mutex<Label>(new Label("total", 0)) compiles, tests/link/thread_mutex_scalar). And lock is the only member of a Mutex a program may name: `guard` is not a member a program may name: the data a `Mutex` owns is reachable only through the guard its `lock()` answers, `using g = m.lock()`, so every path to it takes the lock (NL2464, reject_mutex_member).

  • R2: m.lock() is only the initialiser of a using declaration, the third disposable beside scope() and arena(), and one using takes one lock (a second declarator's lock() is R5's, below). A lock may share a using with a scope opened before it or with an arena on either side, and the declarators close in the reverse of the order they opened, each opening before the next declarator's initialiser runs (an orReturn() there closes the ones before it), so in using a = arena(), g = m.lock() the lock is given back before the arena releases (tests/link/thread_mutex_using_mixed); a scope opened after the lock, using g = m.lock(), s = scope(), is R5's (reject_mutex_using_lock_then_scope). A lock() anywhere but a using initialiser is `lock()` must be the initialiser of a `using` declaration: the lock is released when the block that declares it ends, so a guard bound any other way would be one nobody releases; declare it as `using g = m.lock()` (NL2465, reject_mutex_lock_not_using; reject_mutex_lock_expr_body for an arrow whose expression body is m.lock()).

  • R3: a guard is used only as g.value, and what it reaches only as the base of another access. Bound, stored, returned or walked, a guard is A guard is used only as `g.value`, and `g.value` only as the base of a field or element read or store: passed, bound, stored, returned or walked, what it reaches would outlive the block that holds the lock; write `g.value.n` where the value is used, or hand `g` to a helper whose parameter is a `MutexGuard<Counter>` (NL2466, reject_mutex_guard_helper_return, reject_mutex_guard_helper_field, reject_mutex_guard_helper_ternary). The one place a guard may go is to a guard helper: a function named at the call, neither a method, a constructor nor a generic, whose parameter in that position is declared as the guard is, MutexGuard<T>, with the guard named once among the call's arguments:

    const bump = (g: MutexGuard<Counter>): void => {
      g.value.n = g.value.n + 1;
    };
    
    {
      using g = m.lock();
      bump(g);
    }
    

    A guard helper is named, never held: naming it as a value is `bump` takes a `MutexGuard`: a guard is handed only to a function named at the call, so a function reference cannot name it; call the helper by name (NL2739, reject_mutex_guard_helper_ternary_alias, reject_mutex_guard_helper_nullish), because the rules below follow the function a call names.

    The helper's parameter is held to R3–R5 in its body exactly as g is in the block, so it can only read and store through it, and R5 follows the call, so a helper that locks, opens a scope, spawns or receives is refused at the call (reject_mutex_guard_helper_spawn, reject_mutex_guard_helper_receive). A helper that passes the guard on is judged by the same rule at that call (reject_mutex_guard_helper_forward), through helpers that call each other (reject_mutex_guard_helper_mutual) and across an import (link/reject_mutex_guard_helper_import); a helper that locks a Mutex it is handed is refused by R5 at the call (reject_mutex_guard_helper_second_lock), and g.value handed beside the guard by NL2467 (reject_mutex_guard_helper_value_arg). A helper has no default, rest or destructured parameter, which Nish has none of (reject_mutex_guard_helper_default, _rest, _destructured). link/thread_mutex_helper is three helpers deep on four threads. Any other call is `g` is passed to `push`, a method: a guard is handed only to a function named at the call, neither a method, a constructor nor a generic, whose parameter is declared as the guard is, `MutexGuard<T>`, once per call, because that parameter is held to the rules of the guard's block; write the helper as `const useGuard = (g: MutexGuard<Counter>): void => { console.log(g.value.n) }` and call it as `useGuard(g)` (NL2536; reject_mutex_guard_helper_array, _map, _method, _new, _ctor_this, _generic, _twice, _nullable, and _callback for a helper that takes a function, which makes it a generic), because a method or a constructor has this as a second way out, a generic's body treats T as any value, and a guard passed twice is two names for one lock. Anything read through it that is not a number, a boolean, an enum or a string — const c = g.value, g.value.xs as a value, a for...of over it — is This reads `Counter` through a guard, whose chains are only the base of a further field or element read or store: a value that is not a number, a `boolean`, an enum or a `string`, which the read copies, would outlive the block that holds the lock; do the work inside the guard's block, or read the numbers and strings it holds one at a time, `const n = g.value.n` (NL2467, reject_mutex_guard_alias; reject_mutex_guard_alias_array for const b = g.value.bins, which would race a guarded write once the block ends; reject_mutex_guard_return and reject_mutex_guard_return_array for a return; reject_mutex_guard_store_field and reject_mutex_guard_store_element for a store; reject_mutex_guard_for_of for a walk). An arrow's expression body is its answer and is judged as a return of it is, so a helper written (g: MutexGuard<Box>): Box => g.value is refused as one that writes return g.value (reject_mutex_guard_helper_expr_value, _field, _paren, _index, link/reject_mutex_guard_helper_expr_import; _call, _new and _cond for g.value handed to a call or a new, or written as an arm of ? :, inside the body), and reject_mutex_guard_helper_expr_churn is the use-after-free it would open: the object kept past the lock, its string replaced under a second lock, and churn() reusing the memory before the old one is printed. An arrow argument, which may read its function's const locals, may not read a guard among them, whether it runs on this thread (reject_mutex_guard_capture) or is passed to a data-parallel form (reject_par_capture_guard). A scalar read through the guard is a value like any other, .length of an array it reaches included. When T is itself a scalar, g.value is read and stored directly — g.value = g.value + 1 on a Mutex<i32> — and the store reaches the Mutex under Node as natively, because the guard is the Mutex's own storage rather than a copy (tests/link/thread_mutex_scalar).

  • R4: a store through a guard stores a scalar or a string, and no method is called through it. A number, a boolean, an enum or a string, into a field or an element of an array that already exists: This stores `string[]` through a guard, which stores only a number, a `boolean`, an enum or a `string`, which the store copies, into a field or an element that already exists: anything else is a pointer, which may point into a task's arena, freed when the task is joined; build the value into the one the `Mutex` is made with, or store what it holds one at a time into slots that already exist, `g.value.names[i] = v[i]` (NL2468, reject_mutex_guard_store; reject_mutex_guard_store_class for a class). A null is stored through a guard only into a string | null slot — a field, an element or a Mutex<string | null> — where the store frees the copy the slot held and keeps nothing, so a later read through ?. answers null (tests/link/thread_mutex_null_store, natively and under Node, read back after churn()). Into any other nullable slot, a Link | null field, element or Mutex<Link | null>, it is refused, since such a slot holds a pointer, which a guard stores in no form: This stores `null` through a guard into a slot that holds an object, and a guard stores only a number, a `boolean`, an enum or a `string`, and a `null` only into a `string | null` slot, where the store frees the copy the slot held; keep a `boolean` field beside the slot, `g.value.hasNext = false`, and test it before reading the object (NL2537, reject_mutex_guard_null_class, _link_element, _link_value, _link_nested for g.value.inner.next = null, and _link_paren for g.value.next = (null)). Only the literal null is stored: a string | null value, a variable or a ??, is refused as NL2468 above, since the store copies what it is given and a value that may be null would need a test the store does not make. A string stored through a guard is copied into memory the mutex owns: nish_guard_keep mallocs the copy, records it, and frees the copy the slot held before, if that one was its own; a string read through a guard is copied out into the reader's arena (.length excepted, which keeps nothing), so no reader holds the block the next store frees. The block a slot holds when the Mutex itself is dropped is not freed — the arena that held the Mutex does not say when it goes — and is reported by valgrind as still reachable, one block per guarded string slot (tests/link/thread_mutex_string). A program that makes and drops many such Mutexes keeps all of them: 100,000, each holding one 850-byte string, peaked at 89 MB against 11 MB for the same loop without the store. The record of those copies is one for the process, behind a lock of its own, so stores through two different Mutexes wait on each other while it is held. push — which can move the array into the task's arena — and every other method are `push` is called through a guard, which reads and stores fields and elements and calls no method: a method could keep what it reaches, or grow an array into a task's arena, which is freed when it is joined; write what it does as reads and stores through the guard, into slots that already exist, `g.value.xs[0] = 1` (NL2469, reject_mutex_guard_push). So a Mutex<i32[]> or a class with an i32[] field holds a histogram, sized when it is made.

  • R5: one lock at a time, and nothing waited on while it is held. Inside a guard's block — directly, or through any function it calls — a second lock(), scope(), spawn, parallelMapInto or parallelReduce is A second `lock()` is reached while `ga` holds its lock: inside a guard's block, directly or through any function it calls, there is no second `lock()`, no `scope()` or `spawn`, and no `parallelMapInto` or `parallelReduce`, so one lock is held at a time and nothing is waited on while it is held; move `b.lock()` below the end of the block that opens with `using ga = a.lock()`, which releases the lock (NL2470; reject_mutex_nested_lock, and reject_mutex_using_two_locks and reject_mutex_using_two_mutexes for a second declarator of the same using; reject_mutex_lock_through_call for a callee that locks, and reject_mutex_held_new for a constructor that does, reached by new; reject_mutex_held_scope and reject_mutex_held_scope_call for a scope; reject_mutex_held_map and reject_mutex_held_reduce for the data-parallel calls). That is the whole argument that a program cannot deadlock: a task holding a lock never waits for anything, so every lock is given back.

  • R6: a store through a guard is the one shared write a task may make. P2's rule is otherwise unchanged (above, NL2471).

  • R7: from the statement that holds a scope's first spawn to the end of its block, the parent does not lock what the scope's tasks share, directly or through any function or constructor it calls — P2's own window, so a lock before the spawn in the same loop body is inside it: on the loop's second pass that lock runs after the first pass's spawn (reject_mutex_region_lock_loop). Natively the tasks have not run yet there, and under Node each has run at its spawn, so the parent would see a different value: This `lock()` locks a `Mutex` a task of `s` can reach through `arg`, before the scope joins: natively the scope's tasks run when its block ends and under Node each runs where it is spawned, so the parent locks what they share only before the statement that holds the first `spawn` or after the join; move `m.lock()` above the block that opens with `using s = scope()`, or below the end of that block (NL2472, reject_mutex_region_lock; reject_mutex_region_lock_field for a Mutex the task reaches as a field of its argument; reject_mutex_region_lock_call for a call that takes the lock, and reject_mutex_region_lock_new for a new whose constructor does). It is asked by type: a Mutex<T> of the type a task's argument reaches, or, for a call that takes a lock, any Mutex a task reaches. Before the first spawn and after the join the parent locks freely, as tests/link/thread_mutex_counter does.

  • R8: a Mutex reaches a task as its argument, or as a field of the class its argument is. An array of them, a deeper field, or a Mutex inside another's data is The argument of `spawn` is `Mutex<Counter>[]`, which reaches a `Mutex` through `ms[i]`: a `Mutex` reaches a task as its argument or as a field of the class its argument is, never deeper, so which tasks share a lock is a question the checker can answer; hand the task the `Mutex` itself, `s.spawn(first, ms[i], done, 0)`, and declare its parameter a `Mutex` of the same data (NL2473, reject_mutex_task_array; reject_mutex_task_nested for a Mutex inside another's data, reject_mutex_task_deep for one two classes deep, reject_mutex_task_nullable for a nullable carrier, reject_mutex_task_map for a Map holding them). A task's destination holds numbers, so a Mutex is never one.

And a lock is a scope's: the body of parallelMapInto or parallelReduce, which runs on many threads at once, takes none (`tally` takes a `Mutex`'s lock, and `parallelMapInto` runs it on several threads at once: a lock is taken by a scope's task or by the thread that opened the scope, never by a parallel body; run it as a scope's tasks instead, `{ using s2 = scope(); for (let i = 0; i < src.length; i++) { s2.spawn(tally, src[i], dst, i) } }`, NL2474, reject_mutex_parallel_body; reject_mutex_parallel_body_call and reject_mutex_parallel_body_new for a lock taken in a callee or a constructor).

What a guarded program means. The tasks' critical sections run one at a time, in some order. Natively the order is the scheduler's; under Node, where lock() answers the guard at once and its [Symbol.dispose] does nothing, it is spawn order, which is one of the native orders (RUN_UNDER_NODE.md). A program prints the same both ways when its guarded updates commute exactly — integer sums and counts, a minimum, a histogram; an f64 sum does not, because the order changes its bits — and no task's answer or control flow depends on guarded state it read. Anything else is its author's obligation, as an associative operator is for a named function given to parallelReduce; the checker does not try to prove it. It is the one place the lock weakens the scope's promise above, that the checker refuses anything that could tell the two orders apart.

Atomics: AtomicI32 and AtomicI64

An atomic is one integer a scope's tasks, and the bodies of the data-parallel forms, update at once without a lock: each method is a single indivisible operation on the word, so no update is lost to a race and none waits for another. Where a Mutex guards a whole value, an atomic guards one counter. The fast way to use one is the example's: each task counts into a local tally no other core touches and adds it to the shared counters once, at its end, so the atomics see one update per bin per task rather than one per value ("Measured", below).

import { AtomicI32, parallelMapInto, scope } from "nish/threads";

class Shard {
  xs: i32[];
  bins: AtomicI32[];
  constructor(xs: i32[], bins: AtomicI32[]) {
    this.xs = xs;
    this.bins = bins;
  }
}

const binShard = (s: Shard): i32 => {
  const local: i32[] = [0, 0, 0, 0]; // this task's own counts: no core shares them
  for (const x of s.xs) {
    const bin: i32 = x & 3;
    if (bin < local.length) {
      local[bin] += 1;
    }
  }
  for (let b = 0; b < local.length; b++) {
    s.bins[b].add(local[b]);         // one indivisible add per bin, no lock
  }
  return 0;
};

export const main = (): i32 => {
  const bins: AtomicI32[] = [new AtomicI32(0), new AtomicI32(0), new AtomicI32(0), new AtomicI32(0)];
  const done: i32[] = [0, 0];
  {
    using s = scope();
    s.spawn(binShard, new Shard([1, 2, 3, 4, 5], bins), done, 0);
    s.spawn(binShard, new Shard([6, 7, 8], bins), done, 1);
  }
  const big = new AtomicI32(0);
  const xs: i32[] = [3, 9, 4];
  const ys: i32[] = [0, 0, 0];
  parallelMapInto(xs, ys, (x) => big.max(x));  // the one shared write a body may make
  console.log(`${bins[0].load()} ${bins[1].load()} ${big.load()}`); // 2 2 9
  return 0;
};
  • The methods. load() and store(v); add(n), sub(n), min(v), max(v) and exchange(v), each answering the value the word held before it; and compareExchange(expected, desired), which stores desired when the word holds expected and answers the value it saw either way, so a loop that retries until the answer is its expected builds any other update (tests/link/thread_atomic_tasks). add and sub wrap at the width; min and max compare signed. AtomicI64 is the same over i64.
  • Natively each method is one instruction, sequentially consistent: a load atomic or store atomic ... seq_cst, an atomicrmw add, sub, min, max or xchg ... seq_cst, or a cmpxchg ... seq_cst seq_cst, aligned to the width (align 4, align 8), the whole body of the method's instance in nish/threads's module (tests/link/thread_atomic_tasks, whose threads.ll is the lowering; docs/IR_COOKBOOK.md, "an atomic"). On wasm32 a scope's tasks and a parallel call's chunks run one after another, and the same IR is lowered for a module without shared memory.
  • Under Node each method is the plain operation on an ordinary field, which is exact because Node runs every task at its spawn and every parallel body in order. Node holds an AtomicI64 as a number, and it agrees with native while values stay within ±2^53; wrapping at 2^63 is native-only, the same limit as every i64 (RUN_UNDER_NODE.md), so a program meant to run under Node as well passes it i64-typed values, not toI64 of a number, which is a BigInt there.

Three rules keep every touch of the word indivisible and every program's answer the same natively and under Node:

  • R1: an atomic is reached only by calling its methods. Its word is a field, value, that no program may name, read or store: `value` is not a member a program may name: an atomic is read and updated only by calling its methods, `load()`, `add(n)` and the rest, each one indivisible: a plain read or store of its word would race with the update a task makes at the same moment (NL2514, tests/cases/reject_atomic_member_read, reject_atomic_member_store, and reject_atomic_member_nested for h.a.b.value). A method named without a call is refused by the rule every class's methods keep, not by NL2514: Unknown field `add` on class `AtomicI64` (it is a method; call it) (reject_atomic_member_unbound). new AtomicI32(v) takes a number, so an atomic is fresh when it is made, and new is the only way one is made. An object literal of any class of nish/threads — an atomic, a Mutex, its MutexGuard, a Channel or a ThreadScope — would be a plain record under Node, without the methods, and is refused with `AtomicI32` is a class of `nish/threads`, and its objects are made only as the module makes them: `new AtomicI32(v)`, `new AtomicI64(v)`, `new AtomicI32Array(n)`, `new AtomicI64Array(n)`, `new Mutex(v)`, `new Channel()`, `m.lock()` and `scope()`. An object literal would be a plain record under Node, with none of the methods the class is used through (NL2515, tests/cases/reject_atomic_literal, reject_atomic_literal_i64, reject_thread_literal_channel, reject_thread_literal_mutex and reject_thread_literal_mutex_only, whose Mutex literal is refused by this rule rather than by new Mutex's fresh-value one, and reject_thread_literal_scope).

  • R7: from the statement that holds a scope's first spawn to the end of its block, the parent does not touch an atomic the scope's tasks can reach — with a method, or through any function, method or constructor it calls, or by a data-parallel call that does — Mutex's R7, for the same reason: natively the tasks have not run there, and under Node each has run at its spawn, so a load() would answer differently: This `load()` reaches an atomic a task of `s` can reach through `arg`, before the scope joins: natively the scope's tasks run when its block ends and under Node each runs where it is spawned, so the parent touches what they share only before the statement that holds the first `spawn` or after the join; move `a.load()` above the block that opens with `using s = scope()`, or below the end of that block (NL2513, reject_atomic_region_direct; reject_atomic_region_capture for an arrow task that captures it; reject_atomic_region_field, reject_atomic_region_array, reject_atomic_region_map and reject_atomic_region_generic for one the task's argument reaches through a field, an element, a Map's value or a generic's field; reject_atomic_region_call, reject_atomic_region_method and reject_atomic_region_new for a function, a method and a constructor that touch it; reject_atomic_region_this for one reached through this handed out of a constructor; reject_atomic_region_declarators for the second declarator of a declaration; reject_atomic_region_spawn_arg for a second spawn's argument; reject_atomic_region_parallel for a parallel call). It is asked by type, as Mutex's is: an atomic of the type a task's argument or capture reaches, at any depth, so an alias through ?: or a function's answer is the same atomic (reject_atomic_region_ternary, reject_atomic_region_return), and so is any other atomic of that type the task cannot in fact reach. An atomic of a type no task reaches is the parent's alone, and its methods are not counted as writes the scope's tasks may read (tests/link/thread_atomic_unreached). Before the first spawn and after the join the parent touches it freely.

    The scopes of one using are judged against each other too: natively they join in reverse order, so a task of t in using s = scope(), t = scope() runs before every task of s, and under Node each runs at its spawn. A spawn on one of them whose task shares a destination, an atomic or a lock with a task of another scope of the same statement is refused, naming the other scope and the path it shares: This `spawn` on `t` hands its task an atomic through `a`, which a task of `s` also reaches: the scopes one `using` statement opens join one after the other natively, the last declared first, while under Node each task runs where it is spawned, so the tasks of sibling scopes share no lock, no atomic and no destination; open each scope in a block of its own, so the first joins before the second opens, `{ using s = scope(); s.spawn(() => { a.add(1); }) } { using t = scope(); t.spawn(() => { b.store(a.load()); }) }` (NL2516). A destination is shared when two scopes' spawns name the same array, at the same slot or at different ones (reject_thread_sibling_destination, _slots, _arrow for arrow tasks, _three for three declarators). A task that reaches another scope's destination — to read the answer that scope's task returns, or to write it — hands the array on, which the destination rule refuses before this one: _void writes it, _read reads it through an arrow's capture, _new through a constructor reached by new, and _this through a field of an object whose constructor hands this out. An atomic or a lock is told apart by the const it was made in by new: a task handed that binding, as its argument or a capture, reaches that object and no other, and two such bindings are two objects. Reached any other way — through another binding, a field, an element, a function's answer — it is read as any object of its type, which meets every one (reject_atomic_sibling_load, reject_atomic_sibling_exchange, reject_atomic_sibling_arrow, reject_atomic_sibling_three, reject_thread_mutex_sibling, reject_thread_mutex_sibling_alias for a second name of one lock, reject_thread_mutex_sibling_field for one reached through a field). Every way a task reaches the object counts: through a field of a field (reject_atomic_sibling_field_field, reject_thread_mutex_sibling_field_field), through an object whose constructor hands this out (reject_atomic_sibling_this, reject_thread_mutex_sibling_this), in a constructor reached by new (reject_atomic_sibling_new, reject_thread_mutex_sibling_new), in a helper the task calls (reject_atomic_sibling_helper, reject_thread_mutex_sibling_helper), from an arrow's capture (reject_thread_mutex_sibling_arrow), and between any two declarators of the statement (reject_thread_mutex_sibling_three). Tasks that take no lock share none, and tasks that call no atomic method share no atomic. Sibling scopes whose tasks reach disjoint locks or atomics, of one type or of two, run as before (tests/link/thread_sibling_disjoint_mutex, _mutex_types, _atomics).

  • R8: an atomic reaches a task at any depth — as its argument, a field of it, an element of an AtomicI32[], or a capture — unlike a Mutex, which stops one field down: an atomic holds no data a task could keep, so which tasks share it is a question R7 answers by type, without the exact sharing set a lock's R5 needs.

An atomic update is the one shared write a task or a parallel body may make beside a guard. A task that calls add on an atomic its argument reaches writes nothing the rule above counts, and neither does the body of parallelMapInto, parallelMap, parallelFrom or parallelReduce, which takes no lock but may update an atomic it captures or an element reaches (tests/link/par_atomic). Any other write beside it is refused as before (reject_atomic_par_shared_write). A parallel body answers a number, never an atomic (reject_atomic_par_result).

What an atomic program means. The updates land in some order: natively the scheduler's, under Node spawn order and element order. A program prints the same both ways when its updates commute exactly — add and sub, min, max, a count, a histogram — and no task's answer or control flow depends on a value it read from a word another task updates. A store, an exchange or a compareExchange that wins against another task is order-dependent. A parallelReduce's blocks depend on n alone, the same on one core or sixty four, but natively they run at once and under Node one after another, so what a body does to an atomic must not depend on which block reaches it first. That is the author's obligation, as it is for a Mutex's critical sections; the checker does not try to prove it.

Measured. examples/histogram-atomic.ts bins 4 × 2,000,000 values per round into one AtomicI32Array of 256 bins, four rounds, on four tasks. On a four-core machine, over seven interleaved rounds whose seeds move every round, the medians are: each task counting into a local tally and adding it with one add per bin, 29 ms (a byte-identical copy of it, the noise band, 23 ms; the two copies' runs 22–49 ms); one thread binning into plain counters, 80 ms (2.8× slower); one atomic add per value, 554 ms (19× slower), because every increment contends for a cache line another core is writing; and one Mutex taken per value, 2,155 ms. An atomic is far cheaper than a lock and still far dearer than a counter no other core touches: update it once per batch, not once per value.

Atomic arrays: AtomicI32Array and AtomicI64Array

An atomic array is length atomic counters in one contiguous buffer. It is AtomicI32 (or AtomicI64) with the element's index first: every method of the one has a twin on the other, with the same meaning, the same ordering and the same rules, R1, R7 and R8 above, and the same one shared write a task or a parallel body may make. Where an AtomicI32[] is one object per counter, each its own allocation, so sixteen counters land on several cache lines that the cores writing them pass back and forth, an AtomicI32Array(16) is sixteen adjacent i32s in 64 contiguous bytes, which is the layout Rust's [AtomicI32; 16] and Go's [16]atomic.Int32 have. The buffer starts on a 64-byte boundary at least one line into its own allocation, which ends past the line the last element is on, so no other object shares a line with the elements, the lines the counters' writers fight over.

import { AtomicI32Array, AtomicI64Array, parallelMapInto, scope } from "nish/threads";

class Shard {
  xs: i32[];
  bins: AtomicI32Array;
  constructor(xs: i32[], bins: AtomicI32Array) {
    this.xs = xs;
    this.bins = bins;
  }
}

const binShard = (s: Shard): i32 => {
  for (const x of s.xs) {
    s.bins.add(x & 3, 1);            // one indivisible add on element x & 3
  }
  return 0;
};

export const main = (): i32 => {
  const bins = new AtomicI32Array(4);   // four counters in one buffer, each 0
  const done: i32[] = [0, 0];
  {
    using s = scope();
    s.spawn(binShard, new Shard([1, 2, 3, 4, 5], bins), done, 0);
    s.spawn(binShard, new Shard([6, 7, 8], bins), done, 1);
  }
  const bytes = new AtomicI64Array(2);
  const one: i64 = 1;
  const xs: i32[] = [3, 9, 4];
  const ys: i32[] = [0, 0, 0];
  parallelMapInto(xs, ys, (x) => {
    bytes.add(x & 1, one);             // the one shared write a body may make
    return x;
  });
  console.log(`${bins.load(0)} ${bins.load(1)} ${bins.length} ${bytes.load(1)}`); // 2 2 4 2
  return 0;
};
  • The methods. load(i) and store(i, v); add(i, n), sub(i, n), min(i, v), max(i, v) and exchange(i, v), each answering the value element i held before it; and compareExchange(i, expected, desired). length is a readonly field, fixed by new AtomicI32Array(n), which makes n elements, each 0; a negative n stops the program as new Array of one does. AtomicI64Array is the same over i64, its index still an i32.
  • An index is checked as an array index is: one out of range stops the program with index out of range: 16 >= 16 and exit status 1, and never touches memory outside the buffer (tests/link/atomic_array_out_of_range). The check is an ordinary one, which the prover may discharge.
  • Natively each method is the bounds check and then one instruction on a getelementptr into the buffer: a load atomic or store atomic ... seq_cst, an atomicrmw add, sub, min, max or xchg ... seq_cst, or a cmpxchg ... seq_cst seq_cst, aligned to the element (align 4, align 8); no element is an object of its own, so there is no per-element pointer to load (tests/link/thread_atomic_array, whose threads.ll is the lowering; docs/IR_COOKBOOK.md, "an atomic array").
  • Under Node each method is the plain operation on an element of an ordinary array, exact for the reason an AtomicI32's is, with the same ±2^53 limit on an AtomicI64Array's elements.
  • The rules are AtomicI32's. Its buffer, values, is a field no program may name (R1, NL2514, reject_atomic_array_member_read, reject_atomic_array_member_store), and a method named without a call is refused as any class's is (reject_atomic_array_member_unbound); length is assigned only by the constructor (reject_atomic_array_length_store); an object literal of either class is refused by NL2515 (reject_atomic_array_literal); the parent does not touch one a scope's tasks reach before the scope joins (R7, NL2513, reject_atomic_array_region_direct); the tasks of sibling scopes share none (NL2516, reject_atomic_array_sibling_load); and a parallel body may update one and write nothing else shared (reject_atomic_array_par_shared_write, tests/link/par_atomic_array).

Choose an atomic array over an AtomicI32[] when the counters are many or hot — a histogram's bins, a counter per route or per worker — and an AtomicI32 for one counter, or for counters that live in different objects. A counter no other core writes is still cheaper than either: count into a task's own tally and add it once per batch, as the example under "Atomics" does.

Channels: Channel<T>

A Channel passes scalars from a scope's tasks to one receiver: the tasks that produce call send, and the one that consumes walks the channel with for...of, which waits for values and ends once every task that may send on it has returned and it is drained (wp29-thread-surface.md §4.4). Nobody closes a channel, so nobody can close one too early, or twice.

import { Channel, scope } from "nish/threads";

class Pipe {
  ch: Channel<i32>;
  n: i32;
  constructor(ch: Channel<i32>, n: i32) {
    this.ch = ch;
    this.n = n;
  }
}

const produce = (p: Pipe): i32 => {
  for (let i: i32 = 1; i <= p.n; i++) {
    p.ch.send(i);                                 // never waits
  }
  return p.n;
};

const consume = (p: Pipe): i32 => {
  let sum: i32 = 0;
  for (const x of p.ch) {                         // waits; ends once the sender has returned
    sum = sum + x;
  }
  return sum;
};

export const main = (): i32 => {
  const sent: i32[] = [0];
  const sums: i32[] = [0];
  const ch = new Channel<i32>();                  // for one run of the scope below
  {
    using s = scope();
    s.spawn(produce, new Pipe(ch, 100), sent, 0); // every sender first
    s.spawn(consume, new Pipe(ch, 0), sums, 0);   // then the one receiver
  }
  console.log(`${sums[0]}`);                      // 5050
  return 0;
};
  • Natively a channel is a buffer of 8-byte slots outside every arena, behind the channel's state word, with a lock, a condition to wait on and a count that keeps it open (runtime/runtime-parallel.c). send widens the value into a slot and never waits; the loop takes a slot and narrows it back, and waits while the channel is empty and open. The count starts at one for the scope: a spawn whose task may send on the channel adds one before the task is filed and takes it away when the task returns, and the scope takes its own away at every exit of its block, just before the join — whether or not the join finds a task to run — so a channel whose only sender was never spawned closes too (tests/link/thread_channel_pipeline, whose main.ll and threads.ll are the lowering; tests/link/thread_channel_exit_paths for the end of the block, a return, a break and a scope that runs nothing). On wasm32 a scope's tasks run one after another in spawn order, every sender before the receiver, and the same programs print the same thing (tests/run.js, "the --profile wasi build").
  • The buffer is given back when a loop drains the channel closed, and the channel records that, so a second loop over it ends at once rather than reading what was freed; a loop that stops early with break leaves the rest for the next one (tests/link/thread_channel_parent_drain). Only that drain gives anything back. A channel whose last loop stopped early with break or return, or that no loop ever walks, keeps its bookkeeping — a few dozen bytes for its lock, its count and its buffer's bounds — and the values left in its buffer until the program exits, one such allocation per run of the declaration.
  • Under Node a channel is an array queue. send pushes, and the loop takes values off the front as it walks, so a second loop finds what the first left (RUN_UNDER_NODE.md). That is correct because a task runs at its spawn under Node and every sender is spawned before the receiver, so every value has been sent by the time the receiver's loop starts.

Seven rules keep "every sender has returned" a single moment and the one receiver from waiting on what never comes, and a program that breaks one is refused with a code of that rule's own: ten codes, NL2475 to NL2484, because C2 has one for where a channel is declared, one for how it is used and one for a channel inside an arrow, and C6 one for spawn order and one for a task that sends and receives:

  • C1: a channel carries a number, a boolean, an enum or a string. A sender's arena is freed when its scope joins, so a string is copied on the way: send copies it into a malloc'd block the buffer slot holds (nish_cross_send, a copy in time that grows with the string alone: tests/link/thread_channel_string_many sends a million it also keeps), and receive, or a for...of step, copies it into the receiver's arena and frees the block (nish_cross_in). A string a receiver never takes — the loop that left with break — stays in the buffer, which is itself given back only once it has been drained closed, so the two leak together. Anything else would be a pointer into the sender's arena: A `Channel` carries a number, a `boolean`, an enum or a `string`, and this one would carry `string[]`: a sender's arena is freed when its scope joins, and only a string is copied out of it on the way, so anything else would point into freed memory; keep the values in an array the receiver can read and send each one's index, `const words = new Channel<i32>()` (NL2475, tests/cases/reject_channel_element). A Channel<string> pipeline — two producers, a rewriting stage and a consumer that keeps what it receives in an array — matches Node (tests/link/thread_channel_string). An f64 channel (tests/link/thread_channel_two_producers), a boolean one (thread_channel_parent_drain), and an i64, an enum and a u8 one (thread_channel_kinds) compile and carry their values exactly.
  • C2: a channel is a const declared beside its scope, and used only as a channel. new Channel<T>() initialises a const in the block that holds the scope's using s = scope(), or in the block that has the scope's block as one of its statements, before it, with no loop, condition or function between, so each run of the declaration makes a channel for one run of one scope — a scope in a loop gets a fresh one every pass: `new Channel` must be the initialiser of a `const`: a channel is a `const` declared in the block that holds its scope's `using s = scope()`, or in the block that has the scope's block as one of its statements, before it, so each run of the declaration makes a channel for one run of one scope; write `const ch = new Channel<i32>()` on the line before `using s = scope()` (NL2476, reject_channel_not_const, reject_channel_let_second_declarator for a let that declares it beside something else, reject_channel_scope_conditional for a scope inside an if, and reject_channel_scope_loop for one inside a loop the declaration is outside; a function boundary between them is C4's, below). A channel is the receiver of send, the iterable of a for...of, an argument, or a field its class's constructor sets from a parameter — never bound to another name, stored, returned or compared, and no member but send is named: A `Channel` cannot be bound, stored, returned or compared here: a channel is used only as the receiver of `send`, the iterable of a `for...of`, an argument, or a field its class's constructor sets from a parameter, so it is never bound, stored, returned or captured, and which tasks reach it is a question the checker can answer (NL2477, reject_channel_bound, reject_channel_member, reject_channel_member_take, _ready, _head and _state, reject_channel_returned_expr for an arrow whose expression body is a channel, which is its answer, and reject_channel_stored for a class that holds one made anywhere but at the spawn; reject_channel_nullable for a task's argument whose channel may be null, which the spawn could not count as a sender). A type holds a channel only as exactly Channel<T>: never as one that may be null, an array's element, a Result's arm or another generic's type argument, wherever the type is written — a local, a parameter, a result or a field, a type alias read as the type it names (reject_channel_nullable, reject_channel_array, reject_channel_map_value, reject_channel_result_type, reject_channel_alias_type, reject_channel_field_generic). An argument is one only to a parameter its function declares a Channel, not to a generic's T (reject_channel_generic_argument). It reaches a task as R8 says a Mutex does: as the task's argument, or as a field of the object made for it at the spawn, new Pipe(ch, 100), and by one path only, so a task never holds it in two fields (reject_channel_two_paths). A constructor uses a channel it is handed only to set a field from it, this.ch = ch (reject_channel_ctor_bound); one it declares itself it uses as any function does. And a channel is used only in a top-level function's own body, never inside an arrow, which is lifted into a function of its own that these rules do not follow: A `Channel` is used inside an arrow here: an arrow is lifted into a function of its own that the channel rules do not follow, so a channel is declared, sent on and received from only in a top-level function's own body; write the arrow as a module function, `const relayBody = (q: Pipe): i32 => { const c = q.ch; c.send(1); return 1; }`, and pass `relayBody` where the arrow was (NL2484, reject_channel_arrow, and reject_channel_arrow_nullable for a channel type an arrow writes but never uses).
  • C3: a channel is unbounded, so send never waits, because under Node a scope's tasks run one at a time and nothing would drain a full one: `new Channel` takes no capacity: a channel is unbounded and `send` never waits, because under Node a scope's tasks run one at a time and nothing would drain a channel that was full; write `new Channel<i32>()` (NL2478, reject_channel_capacity).
  • C4: one scope's tasks reach a channel, and the parent sends on it only before that scope's block. After the join it only receives. "May send" is the call graph from a task's entry reaching send on a channel its argument holds, so an over-count only closes the channel later: This sends on `ch` once its scope's block has begun: a channel is reachable from the tasks of one scope, the parent sends on it only before the block that holds that scope's spawns, and nobody closes it: it closes once the scope has started to join and every task that may send on it has returned; move `ch.send(7)` above the block that opens with `using s = scope()` (NL2479, reject_channel_parent_send, reject_channel_send_after_join for a send through a call after the join, and reject_channel_two_scopes for the two scopes of one using). A spawn hands its task only a channel the function declared, never one it was handed, so no function boundary comes between a channel and its scope (reject_channel_spawn_handed, and reject_channel_spawn_handed_direct for the channel as the argument itself). In the constructor of a class that holds a channel, directly or through a field's class, this appears only as the base of its own field, this.f, used in one of three ways: as the target of a store (this.f = ..., this.f += ...); as any value, except as the callee of a call, when the field is a number, a boolean, an enum, a string or an array of those, from which nothing reaches a channel (this.total = this.a + this.b, this.width = this.name.length, this.items.push(x)); or as the base of a store into a field or element of what it holds, when that is no channel and holds none (this.span.lo = 1; tests/link/thread_channel_ctor_reads). A method called on it, an argument (to a function or to a constructor), a store elsewhere, a second name (in any declarator), an array element, a read of the channel or of an object that holds one, or this stored even into its own field would carry a send the count would not see: The constructor of a class that holds a channel uses `this` other than to read or set its own fields, `this.f`, where a send made through it could not be counted: … (NL2479; reject_channel_ctor_this, reject_channel_ctor_this_stored, reject_channel_ctor_this_bound, reject_channel_ctor_this_array, reject_channel_ctor_this_new, reject_channel_ctor_this_declarator, reject_channel_ctor_this_self, reject_channel_ctor_this_deep_carrier, reject_channel_ctor_this_send, reject_channel_ctor_this_carrier).
  • C5: a channel has one receiver — one task, from a spawn outside any loop, or the parent in a later statement of the block that has the scope's block as one of its statements, never both — and inside a scope's block the parent receives nothing, directly or through any function it calls, so its one wait is the join: `ch` has a second receiver here: a channel has one receiver, either one task spawned outside any loop or the parent in a later statement of the block that has the scope's block as one of its statements, and inside a scope's block the parent receives nothing, so the parent's one wait is the join (NL2480, reject_channel_two_receivers, reject_channel_parent_and_task, reject_channel_receiver_loop, reject_channel_parent_receive_inside for a call that receives inside the block, reject_channel_receive_in_spawn_argument for one in a spawn's argument, which is the parent's code, and reject_channel_receive_before_using for one before the using in the scope's own block). A call counts as a receive when its function may receive on a channel it is handed; one that receives only on a channel of its own has joined that channel's scope first, and is not one. One receiver is what makes "the receiver sees every value sent" true natively and under Node alike.
  • C6: every task that may send on a channel is spawned before the one that receives on it, in the block's statement order. Under Node a receiver spawned first would find the channel empty, and the same order makes the graph from senders to receivers acyclic, so channels cannot deadlock either: `produce` may send on `ch` and is spawned after its receiver: under Node a task runs where it is spawned, so every task that may send on a channel is spawned, in the block's statement order, before the one task that receives on it, which is also what keeps channels from deadlocking; move the `spawn` of the receiving task below every `spawn` that sends, `s.spawn(produce, new Pipe(ch, 3), out, 1)` above `s.spawn(consume, new Pipe(ch, 0), out, 0)` (NL2481, reject_channel_receiver_first). And no task both sends and receives on one channel: `echo` may both send and receive on `ch`: no task both sends and receives on one channel, because a channel closes only once every task that may send on it has returned, and this one would wait in its own loop for its own return; split it into a task that sends and a task that receives, or answer the result through a second channel (NL2483, reject_channel_send_and_receive, reject_channel_both_through_new for a send made by a constructor a new runs, and reject_channel_both_untraceable for a task that reaches a channel through a second name for what holds it, which counts as using every channel it holds, and says so).
  • C7: nothing receives while a lock is held, directly or through any function it calls — R5's "nothing is waited on while it is held". A send never waits and may stay: A loop over a channel is reached while `g` holds its lock: a receive waits, and nothing waits while a lock is held, so there is no receive inside a guard's block, directly or through any function it calls (a `send` never waits, and may stay) (NL2482, reject_channel_loop_held, and reject_channel_receive_held_call for a function that receives on a channel it is handed; reject_channel_receive_held_expr for one reached through an arrow whose expression body calls it). A function that receives only on a channel of its own opens a scope to fill it, which R5 refuses under a lock already.

What a channel program means. The receiver sees every value sent, each sender's in its send order. How two senders' values interleave is the scheduler's natively and spawn order under Node, so a receiver whose answer depends on it — not a count or an integer sum, but the first value, or an f64 sum whose partial sums are not exact — is its author's obligation, as for the lock. A task's channel is checked where it can be read, at the spawn; a parallel body has no way to reach one, because a task calls no parallelMapInto and a channel is made only beside a scope. select stays declined (wp29 §9).

What nish --fix writes for a thread program

The thread refusals end with the code to write instead (each pinned whole by its tests/wordings case), and six of them carry a fix that nish --fix applies, each only where the source alone decides the replacement (tests/fix/thread-*; the refused shapes are its thread-nofix-* cases):

Refusal The fix Left alone
NL2388, a scope() not bound by using const s = scope() becomes using s = scope() a let, which is assigned again
NL2465, a lock() not bound by using const g = m.lock() becomes using g = m.lock() a let
NL2511, a spawn with no destination, whose task answers a number or a boolean const answer = new Array<T>(1) above the scope's using, and , answer, 0 after the arguments a spawn in a loop, where every task would write one slot; an answer that is not a number or a boolean
NL2506, a task's arrow that captures a let const kNow = k just before the statement, and the arrow reads kNow: the value it would have read when it ran a let another argument names; a let a for header declares, where no statement before the call is in the loop
NL2500, the same in a parallelMap the same the same
NL2484, an arrow that uses a Channel the arrow moves to a const above the top-level statement it is written in, named for the function and Task (mainTask), and the call names it an arrow that captures, or leaves a parameter's or its result's type out, which would not check at the top level

Shared memory between processes: nish/sync/shm and nish/sync/ring

Threads share an address space; processes share nothing unless they map the same memory. nish/sync/shm maps a named region that every process of the same user can open, and nish/sync/ring builds lock-free queues of byte records in one, so prefork workers can hand work to an owner process without a socket:

import { ACQ_REL, ACQUIRE, createRegion, unlinkRegion } from "nish/sync/shm";
import { initSpscRing, spscRingBytes } from "nish/sync/ring";

export const main = (): i32 => {
  const region = createRegion("lang-demo", 64 + spscRingBytes(4096)).expect("create");
  region.addI64(0, toI64(1), ACQ_REL); // a counter every process may bump
  const ring = initSpscRing(region, 64, 4096, true); // a ring after it
  const job: u8[] = [1, 2, 3];
  ring.tryPush(job, 0, job.length); // false when full
  const got: u8[] = new Array<u8>(16);
  const n: i32 = ring.tryPop(got, 0); // 3, or RING_EMPTY
  console.log(`${region.loadI64(0, ACQUIRE)} ${n}`);
  unlinkRegion("lang-demo").expect("unlink");
  return 0;
};
  • A region is named, never inherited. A Nish program has no fork, and every descriptor is close-on-exec, so an anonymous mapping or a memfd cannot reach a child; a name can. createRegion(name, bytes) makes one, zeroed, mode 0600, and refuses a name that exists (SHM_EXISTS), so a name another user planted is never adopted. openRegion(name) maps one this user made, and refuses one owned by someone else, open to group or others, reached through a symbolic link, or (on Linux) not a regular file or with a second hard link (SHM_ACCESS). A name is 1 to 30 of A-Z a-z 0-9 . _ -, not starting with .; a size is 1 byte to 1 GiB. Errors are a Result with the errno (SHM_NOT_FOUND, SHM_INVALID, SHM_TOO_MANY, ...).
  • Lifetime. The name lasts until unlinkRegion, even past every process that mapped it, which is how a process started later finds it. A mapping lasts until close() or the process's exit, when the kernel unmaps it. A thread may hold 63 regions at once.
  • Every access is checked, as an array index is. loadI32, storeI32, addI32, exchangeI32, compareExchangeI32 and their I64 twins take an i32 byte offset; a slot must lie wholly inside the region and at a multiple of its width, or the access panics naming the offset (nish/sync/shm: slot offset 4 is not a multiple of 8, the slot's width). read and write copy a byte window, checked against both the region and the buffer. No offset reaches outside the mapping, whatever a peer wrote into it.
  • Windows: a u8[] that is the region's bytes. region.lend(at, len) answers a u8[] of length len whose data is bytes [at, at + len) of the mapping, checked as read's window is (nish/sync/shm: byte window at 60 is outside the region of 64 bytes). An index store, copyBytes into it or netRead into it writes the region itself, and every process mapping it sees the bytes, with no copy. The window is not atomic: order it with a slot (storeI32(.., RELEASE) after the bytes, loadI32(.., ACQUIRE) before reading them), as a ring does. region.giveBack(window) answers its length and empties it, and close() empties every window the region lent, so an index after either is index out of range: 0 >= 0, never a read of an unmapped page. A pop or a shorter length keeps a window; a push past its length moves it off the region into the program's memory, as any push past capacity does, and giveBack then panics (window of length 17 is no longer the region's bytes: a push moved it), as it does for a window given back twice or lent by another region.
  • The reuse rule. Each lend without reuse takes a header of 48 bytes that lasts until the process exits, since the program may hold the array past any giveBack or close. A loop that lends again passes the window it gave back, region.lend(at, len, window), which re-points that header and takes no memory; a reuse that is still lent, or not this region's, panics. Windows belong to the thread that lent them, as the region does.
  • Why an intrinsic, not FFI. lend and giveBack are the module's two compiler-known functions (src/shm-window.ts), whose bodies the compiler writes as a call of nishshm_lend / nishshm_give_back: a foreign function may take a u8[] but answers no array, and a window must be one the program indexes. Known by its package and path, as nish:region is, the call is marked as writing through and keeping its array and as possibly changing any array's length, so no length the bounds proof knew survives it, and a region's close() is marked the same way.
  • Orderings are C11's: RELAXED, ACQUIRE, RELEASE, ACQ_REL, SEQ_CST, plus fence(order). Each access is one LLVM load atomic, store atomic, atomicrmw or cmpxchg with that ordering; an ordering an access cannot take (a load with RELEASE) panics. A compare-exchange answers the value it found, so it succeeded when that equals expected.
  • Waking. waitI32(at, expected, timeoutMs) sleeps while the slot holds expected and wake(at, count) wakes sleepers, in any process: Linux's shared futex. Darwin and wasi answer SHM_UNSUPPORTED (-95) from both, and a program there polls.
  • Rings. SpscRing (one producer, one consumer, records of any length up to its capacity) and MpscRing (any number of producers, fixed slots) live at a 64-byte-aligned base in a region: initSpscRing / initMpscRing write the header once, attachSpscRing / attachMpscRing read it in another process. tryPush answers false when full; tryPop answers the record's length, RING_EMPTY or RING_TOO_SMALL; popBatch takes every record ready. A ring made with the doorbell lets popWait sleep, and a push rings it only when the consumer is asleep, so a busy ring makes no system call. Two consumers, or two producers on an SpscRing, lose or repeat records but never leave the region; a length no record can have panics as the ring is corrupt.
  • What a program reaches. The modules call C, so a program that imports them reaches ffi (and clock, for popWait's deadline) and --deny ffi refuses it; under --sandbox a region, being a file in /dev/shm, needs fs.read and fs.write as well. They are native only: --emit-ts refuses them, since Node's SharedArrayBuffer is shared between threads of one process, not between processes. A region belongs to the thread that mapped it. A program that imports neither module is unchanged. Tested by tests/link/sync_shm, tests/link/sync_shm_window, tests/link/sync_ring and tests/link/sync_shm_deny_ffi; the design, its measurements and the threads it was weighed against are docs/wp47-shared-memory.md, and the threat model is docs/security/shm.md.

Random numbers: Rng in nish/random

new Rng(seed) is a generator of pseudo-random numbers whose every answer depends on its i32 seed alone, and is the same natively and under Node:

import { Rng } from "nish/random";

export const main = (): i32 => {
  const r = new Rng(42);
  const die: i32 = r.nextInt(1, 6); // 1 to 6, each with equal chance
  const u: f64 = r.nextF64(); // in [0, 1)
  console.log(`${die} ${u < 1.0}`);
  return 0;
};
  • nextI32() answers the next 32 bits as an i32, and nextF64() a double in [0, 1) built from 53 bits of two steps. The state walks a Weyl sequence (an odd increment, wrapping) through all 2^32 values, and each answer is MurmurHash3's 32-bit finaliser of it, so every seed, 0 included, starts a full sequence, and seeds 1, 2 and 3 start unrelated ones. It is not for cryptography: a key comes from crypto.getRandomValues.
  • nextInt(lo, hi) answers an i32 in [lo, hi], both ends included, each with equal chance. It refuses the few nextI32() draws below 2^32 mod the span and answers lo plus the draw mod the span, so no answer is likelier than another; fewer than half the draws are refused, so a call takes under two steps on average. nextInt(-2147483648, 2147483647) is one nextI32(), and lo > hi panics with Rng.nextInt: lo is greater than hi.
  • The same bits under Node are the point of the design. The operations that wrap are wrappingAdd, wrappingSub and wrappingMul from nish:unsafe, every shift has a constant count, nextInt reads its unsigned counts through i32 compares and remainders of non-negative values, and nextF64's double is exact, so tests/link/std_random prints the same lines both ways.
  • A parallel body or a task steps an Rng it made itself, which is its own memory (const r = new Rng(i), "Nor is a write into an object's own slots" under data parallelism); a generator shared by several tasks would be a shared write, and is refused as one. Seed each task's own from its index.

Byte search: indexOfAny in nish/text

indexOfAny(text, bytes, from) answers the first index at or after from whose byte is one of bytes, or -1 (wp38-simd.md §3.1). It is the scan a parser spends its time in — the next " or \ in a JSON string, the next delimiter in a line — and natively it reads 16 or 32 bytes at a time.

import { indexOfAny } from "nish/text";

export const main = (): i32 => {
  const line = "key: \"value\", next";
  console.log(`${indexOfAny(line, "\":", 0)}`);   // 3, the colon
  console.log(`${indexOfAny(line, ",", 99)}`);    // -1: `from` is clamped to the length
  return 0;
};
  • The set is 1 to 16 ASCII bytes, or the call panics. An empty set (indexOfAny: the set must hold 1 to 16 bytes) matches nothing and is a mistake rather than a search; seventeen bytes is more than one vector of the kernel compares; and a byte of 128 or more (indexOfAny: every byte of the set must be ASCII) would be one byte of a multi-byte sequence natively and a UTF-16 unit under Node, which find different things (tests/link/std_text_index_of_any_empty_set, _long_set, _non_ascii_set).
  • from is clamped to [0, text.length], as substring clamps: a negative from searches from the start, and one at or past the length answers -1. The answer is a byte offset, like s.length; an ASCII byte never occurs inside a multi-byte UTF-8 sequence, so a non-ASCII text is searched as it is and only its ASCII bytes match. Under Node the offset is a UTF-16 one, so the two agree on what was found and, once a non-ASCII character precedes it, not on where (tests/link/std_text_index_of_any).
  • The module is ordinary Nish, and the walker is the meaning. std/text.ts writes indexOfAny as its checks and its clamp followed by one call to a private walker, a loop over charCodeAt, which is what runs under Node. The compiler recognises that call by module and name (src/kernels.ts) and emits the runtime's nish_str_index_of_any in its place; everything around it is emitted as written, so the kernel is only ever handed a clamped from and a set of 1 to 16 bytes. The package is part of the test: a root-package file that happens to sit at std/text.ts is an ordinary module, and its walker is called as written. The kernel picks a scalar, a 16-byte (SSE2 or NEON) or a 32-byte (AVX2) path on its first call, and NISH_SIMD=scalar|base|avx2 pins it.
  • It is variable-time. How long the search takes says where the first match is, so it is never for secret data: std/crypto does not call it on one, and nish:secret's expose lets it through only because it touches nothing outside its two strings.

On a 1 MiB text with a match every 4,099 bytes, the kernel scans at 14 GB/s with AVX2 and 7 GB/s on the 16-byte path, against 1.6 GB/s for the same walker compiled as written (bench/index-of-any.ts).

Fixed digits and a split limit: toFixed and splitLimit in nish/text

Nish has no methods on a number and no optional parameters, so JavaScript's x.toFixed(n) and text.split(sep, limit) are two functions of nish/text. Each is the method it stands for, read as TypeScript: under Node the module runs as written and answers what the method answers, and natively it answers the same bytes (tests/link/std_text_to_fixed, tests/link/std_text_split_limit, whose every line tests/run.js also holds to Node's own method).

import { splitLimit, toFixed } from "nish/text";

export const main = (): i32 => {
  console.log(toFixed(1.005, 2));                     // 1.00: 1.005 is stored a little below
  console.log(toFixed(2.5, 0));                       // 3: a tie goes away from zero
  console.log(splitLimit("a,b,c", ",", 2).join("|")); // a|b: the rest is dropped
  return 0;
};
  • toFixed(x, digits) rounds the exact binary value, as ECMA-262 does: 1.005 is stored as 1.00499999999999989…, so it gives "1.00", while 8.345 is stored a little above and gives "8.35"; an exact tie rounds away from zero, so toFixed(-2.5, 0) is "-3". digits is 0 to 100, and any other count panics (toFixed: digits must be 0 to 100) where JavaScript throws a RangeError. NaN, the infinities and any x of magnitude 1e21 or more answer ${x}, as JavaScript does; -0 gives "0.00", and a negative x that rounds to zero keeps its sign (toFixed(-0.0001, 2) is "-0.00"). It is written in Nish over an array of decimal digits, so it is exact for every f64, the subnormals included, and it needs no runtime C.
  • splitLimit(text, sep, limit) keeps the first limit pieces of split(text, sep) and drops the rest, rather than joining them into the last; a limit of 0 gives []. A negative limit keeps every piece, because JavaScript reads the limit through ToUint32 and -1 becomes 4294967295; that makes splitLimit(text, sep, -1) the same as split(text, sep). An empty separator panics (splitLimit: an empty separator) whatever the limit, as split's does, even where JavaScript would answer [].

Byte runs: skipBlank, skipDigits, skipJsonText, skipJsonPlain and copyBytes in nish/bytes

The four runs answer the first index in [from, to) of a u8[] whose byte is not in their class, or to when every byte there is; copyBytes copies a window of one array into another, or into itself (wp38-simd.md, S6a). They are the scans and the copy a parser spends its time in, and std/json is written over them.

import { copyBytes, skipBlank, skipDigits, skipJsonPlain, skipJsonText } from "nish/bytes";

export const main = (): i32 => {
  const text: u8[] = [32, 10, 52, 50, 44, 34, 104, 105, 34]; // " \n42,\"hi\""
  const start = skipBlank(text, 0, 9);
  console.log(`${start} ${skipDigits(text, start, 9)}`);   // 2 4
  console.log(`${skipJsonText(text, 6, 9)}`);              // 8, the closing quote
  console.log(`${skipJsonPlain(text, 0, 9)}`);             // 1, the newline
  const word: u8[] = [0, 0];
  copyBytes(word, 0, text, 6, 2);
  console.log(`${word[0]} ${word[1]}`);                    // 104 105
  return 0;
};
  • The classes. skipBlank skips JSON's four blanks (space, tab, newline, carriage return) and skipDigits the ASCII digits. skipJsonText skips what a JSON string holds as itself: printable ASCII other than " and \ (DEL included, which JSON does not escape), and every well-formed UTF-8 sequence (RFC 3629 §4) that ends before to. It stops at a quote, a backslash or a control byte, and at the lead byte of a sequence that is overlong, a surrogate, past U+10FFFF, a stray continuation byte or cut short by to, so where it stops is the byte a decoder has to look at. skipJsonPlain skips what a JSON writer copies without an escape: every byte from 0x20 up other than " and \, DEL and 0x80–0xff included, with no UTF-8 check. It stops at a quote, a backslash or a byte below 0x20.
  • The runs never panic. to is clamped into [0, b.length] and from into [0, to], as indexOfAny clamps: a negative from scans from the start, and a from at or past to answers it (tests/link/std_bytes_runs).
  • copyBytes panics on a range outside its array. A negative count, or a window that does not lie inside src or dst, panics copyBytes: the range lies outside an array before any byte is written (tests/link/std_bytes_copy_outside). Overlapping ranges of one array copy as if every byte were read before any was written, as copyWithin does.
  • The walkers are the meaning. Each public function clamps or checks its range, then makes one call to a private walker, a plain loop over the bytes, which is what runs under Node. Natively the compiler recognises that call by module and name (src/kernels.ts), as it does indexOfAny's, and emits the runtime's nish_bytes_skip_blank, nish_bytes_skip_digits or nish_bytes_skip_json_text or nish_bytes_skip_json_plain (runtime/runtime-bytes.c) in its place, or one llvm.memmove for the copy. The kernels pick a scalar, a 16-byte (SSE2, NEON or wasm SIMD) or a 32-byte (AVX2) path on their first call, and NISH_SIMD=scalar|base|avx2 pins it, except in a freestanding wasm32 build, which has no environment to read it from.
  • The runs are variable-time. How long one takes says how long the run is, so they are for text, never for secret data: std/crypto does not call them.

SQLite: nish/sqlite

nish/sqlite is the SQLite C library, linked into a program that imports the module, with a Database and a Statement to drive it. A statement is prepared once and bound, stepped and reset as often as wanted:

import { openDatabase, OPEN_READWRITE, OPEN_CREATE } from "nish/sqlite";

export const main = (): i32 => {
  const db = openDatabase(":memory:", OPEN_READWRITE | OPEN_CREATE).expect("open");
  db.exec("CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT)").expect("create");
  const insert = db.prepare("INSERT INTO t (name) VALUES (?1)").expect("prepare");
  insert.bindText(1, "ada").expect("bind");
  insert.step().expect("insert");
  insert.reset().expect("reset");
  console.log(`${db.lastInsertRowid()}`);
  db.close().expect("close");
  return 0;
};
  • Every failure is a Result. openDatabase and prepare answer a SqliteError, with SQLite's extended code (extendedCode, such as 2067 for SQLITE_CONSTRAINT_UNIQUE), its primary code (code, the low byte) and SQLite's message. Every other call answers the bare extended code, and error(code) on the Database or Statement that answered it builds the SqliteError, so a failure allocates only for the caller who asks. step() answers true with a row to read and false when the statement has finished.
  • A row allocates nothing. Binding, step, reset, columnI64, columnF64 and columnInto answer scalars; columnInto(c, buf, at) copies a text or blob column into the caller's u8[], and answers its length, copying only when all of it fits. columnText(c) is the call that makes a string, and it stops at the first NUL byte. Parameters count from 1 and columns from 0, as in SQLite.
  • exec(sql) runs every statement of sql and discards any rows, and begin(), commit() and rollback() are exec of BEGIN, COMMIT and ROLLBACK; pragmas, such as journal_mode=WAL, are SQL too. A binding is copied by SQLite, and reset keeps the bindings.
  • A handle is an i32 into tables the C adapter keeps per thread. close finalizes every statement the connection still has, and a handle used after close or finalize answers SQLITE_MISUSE rather than reaching freed memory. A connection and its statements belong to the thread that opened them.
  • It reaches ffi. The module calls C through declare function, so a program that imports it is reported with ffi and --deny ffi refuses it, naming the call. The files SQLite opens are opened by C, which the fs.read and fs.write capabilities do not see. It is tested natively and under --profile wasi (tests/link/std_sqlite*).

Interfaces and object literals

interface Pair { first: number; second: number; }

const swap = (p: Pair): Pair => ({ first: p.second, second: p.first });
  • A record type is always a declared name. There is no inline object type: (p: { x: i32 }) => p.x is syntax error: expected a type name, found `{` (NL0001), and so is the same type after type Point =, in a field, a return type or a type argument (tests/cases/reject_type_object_literal_param, reject_type_object_literal_alias, _field, _return, _type_arg). A record type is identified by its declaration, not by its fields: two interfaces that declare the same fields are two types, and a value of one is refused where the other is wanted, the only widening being a class to an interface it implements (Types). So a record's fields are declared once, in an interface or a class, and every annotation names it.

  • An interface is a struct type with the same layout rules as a class, with fields only: a method, or an accessor signature get x(): i32, is Interface `Shape` cannot declare methods (interfaces describe layout only) (tests/cases/reject_iface_method, reject_iface_accessor; a field may still be called get or set), extends is Interface inheritance (`extends`) is not supported; list every field (tests/cases/reject_iface_extends, reject_generic_interface_extends), an index signature is Index signatures are not supported in interface `Point` (object layout is fixed) (reject_iface_index_signature), a construct or call signature, new (): Point or (x: i32): i32, is Unsupported interface member in `Point`: ConstructSignature (or CallSignature, reject_iface_member_unsupported), declare interface is `declare interface` is not supported, and new on an interface is Cannot `new` interface `Pair` (tests/cases/reject_cls_new_interface).

  • An object literal allocates in the arena and stores every property in source order. It must set every field exactly once (is missing field `second`, tests/cases/reject_cls_literal_missing; `Pair` has no field `third`, reject_cls_literal_extra), with plain identifier keys and key: value or shorthand members: a key written as a string or a number is Object literal keys must be plain identifiers (reject_cls_literal_key), and a method or an accessor is Unsupported object literal member: MethodDeclaration (only `key: value`) (GetAccessor, SetAccessor; reject_cls_literal_member) — get, set and async stay keys and shorthands ({ get: 1 }, { get }). It takes its type from the context: a variable annotation, the enclosing function's return type, the parameter it is passed to, the field or variable it is assigned to, or the field of an enclosing literal; otherwise Object literal needs a contextual class or interface type (tests/cases/reject_cls_literal_no_context, cls_interface_literal, cls_nested). That context reaches a nested object literal and a null, and stops there: a numeric literal written for a field keeps the mode's default type (see "Numeric literals"), and [] has no element type at all in this position — { xs: [] } is Empty array literal needs a type annotation and wants a named array instead (tests/cases/reject_struct_field_empty_array). A value of the wrong type is Field `code` of `IoError` expects a value of type i32, got f64 (reject_struct_field_f64).

  • A nullable context gives an object literal the struct inside it: wherever one of the contexts above expects E | null — a declared local, a return, an arrow's concise body, a ternary arm whose other arm is null, a parameter, or a field — the literal is an E, checked against E's fields as above, and that E converts to E | null as any other E value does, the same pointer with no wrapper (tests/cases/obj_lit_nullable_ternary, obj_lit_nullable_return, obj_lit_nullable_arrow, obj_lit_nullable_local, obj_lit_nullable_field; reject_obj_lit_nullable_field). A ternary with no declared type around it gives neither arm a struct, so const z = c ? { v: 7 } : null is still Object literal needs a contextual class or interface type (reject_obj_lit_nullable_context).

  • A class with methods is made only by new. Natively a literal would take the class's layout and its methods, but under Node it is a plain object, and a call on it throws, so wherever a context above types a literal as such a class it is `Counter` has methods, so it is made only by `new`: an object literal would be a plain object under Node, with none of the methods the class is used through. Write `new Counter(...)` (NL2556; tests/cases/reject_class_literal_decl, …_return, …_arrow_body, …_call_arg, …_method_arg, …_new_arg, …_array_elem, …_push, …_index_assign, …_map_set, …_record_field, …_assign, …_field_assign, …_ternary, …_paren, …_alias, …_coalesce, …_coalesce_chain, …_coalesce_field, …_coalesce_call, …_coalesce_map, …_coalesce_optional, …_coalesce_ternary, …_equality, …_index_of, …_math_max, …_fn_param, …_unary, …_compound_bitwise, …_compound_arith, …_nullable, …_generic_class, …_generic_target, …_generic_method, …_imported). A generic method counts as a method; a constructor does not. A class with fields alone, and an interface, still take a literal (tests/cases/cls_literal_methodless). The nish/threads classes keep their own refusal, NL2515 (see "Atomics"), and a class holding a Secret keeps NL2545.

  • An object spread { ...s, x: 1 } copies s's fields into the literal, one store per field, and is otherwise the literal written out: the spread and { x: 1, y: s.y } emit the same IR, an alloca where the literal stays local and an arena allocation where it escapes (tests/run.js "object spread", tests/cases/obj_spread_twin). Each spread is evaluated once, in source order with the entries around it, and the entry written later wins: a field set after a spread overrides it, and of two spreads the later one supplies the fields both have; only the fields no later entry overrides are stored (tests/cases/obj_spread_order). A field holding an object is copied as the reference it is (obj_spread_nested), a class instance spreads its fields and none of its methods, as Node's spread does (obj_spread_class), and a nullable source spreads once narrowed (obj_spread_narrowed); a write to the source after the spread leaves the copy as it was (obj_spread_class). The literal must still set every field of its type, by the spread or by name: a spread that supplies only some of them leaves the rest Object literal for `Point` is missing field `z` (reject_object_spread_missing_field). Each form whose Node reading would differ from the copy is refused:

    Form Code Tests
    no contextual class or interface type, const o = { ...s }: An object literal with a spread needs a contextual class or interface type, whose layout the spread's fields are copied into: annotate the variable (`const p: P = { ...s, x: 1 }`) NL2709 reject_object_spread_no_target, reject_ffi_handle_walk_spread
    a source that is not a class or interface value, { ...n } for an i32 NL2705 reject_object_spread
    a source that may be null, which Node spreads as no field at all: narrow it first NL2703 reject_object_spread_nullable
    a union source, whose fields depend on the variant it holds NL2704 reject_object_spread_union
    a class instance with a field the target does not declare, which Node would carry on the object NL2706 reject_object_spread_class_extra
    an interface value with a field the target does not declare NL2707 reject_object_spread_extra_field
    a copied field whose type differs from the target's, i32 into i64: the copy converts nothing. A field a later entry sets by name is not copied, so its type is not checked (obj_spread_override_type); one a later spread supplies is still checked NL2708 reject_object_spread_field_type
    a field set by name before a spread that always overwrites it, TypeScript's TS2783 NL2710 reject_object_spread_overwritten

    An interface value may hold any class that implements it, and such a class may declare fields after the interface's prefix (see implements below): Node copies every own field of the instance, the native copy only the interface's. So once the whole program is checked, a spread of an interface I into a type that lacks a field of some class implementing I is Spread of `Named` into `Tagged` may spread an instance of `Item`, which implements it with field `weight` besides: … (NL3055, reject_object_spread_width); spread a value of the class type instead. A spread into the typed JSON options of a class is NL2711 (reject_json_options_spread). One gap remains: Nish accepts private on a field and ignores it, so a class instance with a private field spreads into a type that declares that field, where tsc refuses the spread; under Node the copy is the same object either way. A computed key is still refused (NL1041). NL1061, Phase 0's old refusal of every object spread, is retired and never reused.

  • readonly on an interface field forbids every assignment (i.a = 2 is Cannot assign to readonly field `a` of `Config`, tests/cases/reject_cls_readonly_interface); literals still set it. The message drops the class form's "outside its constructor", because an interface has none to point at.

  • class C implements I requires I's fields to be C's first fields, in the same order and with identical types; C may declare more after them (tests/cases/cls_implements_prefix). A missing one is does not implement `Point3`: it lacks field `z: i32` (reject_cls_implements_short) and a mismatched one is field 1 is `width: i32` in `Shape` but `height: i32` in `Square` (reject_cls_implements_mismatch); both end (the interface's fields must be the class's first fields, in order).

    A C then converts to I wherever an I is expected (initializer, return, argument, assignment, field store, ternary arm, I[] element) by one bitcast, because the two address the same bytes at the same offsets for every field I names; a class with the right fields that does not list I is not assignable (tests/cases/cls_implements, reject_cls_not_implements). Nothing converts back: there is no downcast, no as, no instanceof. The class's own fields after the prefix reuse its tail padding, exactly as clang lays out the same C struct (tests/layout/structs.ts, classes K, L, M).

    This is the only widening in the language. Since WP25 a class is a prefix of nothing but the interfaces it names, which is what lets two classes with different tails be held in one I[] and operated on by one function (tests/cases/cls_implements_prefix, tests/differential/corpus/class_prefix) — the job extends used to do, without a dispatch rule to get wrong.

Calling C

declare function name(params): T; declares a C function this program calls but does not define. The symbol is the identifier — no mangling, no prefix — and the call is an ordinary call:

declare function abs(n: i32): i32;
declare i32 @abs(i32)
  %0 = call i32 @abs(i32 %n)

The spelling is TypeScript's own ambient declaration, and declare is a contextual keyword: a variable may still be called declare.

A parameter is a scalar, a CPtr, a borrowed string, a u8[] buffer with its count or a borrowed declare class; a return is a scalar, a CPtr or a copied string — i32, i64, u8, u16, u32, u64, f32, f64, boolean, void, the foreign pointer below, string and string | null in either position, and a buffer and a C struct as a parameter. Any other class or an interface is rejected with and a declared C function takes scalars, `CPtr`, a `string` it borrows as `const char *` and a `u8[]` buffer followed by its `u64` count, or a `declare class` it borrows by pointer in a parameter (reject_ffi_class_param), any other array with the only array a declared C function takes is a `u8[]` or a `readonly u8[]` (tests/cases/reject_ffi_string_array_param: a string[] too, which would be a char ** the compiler built per call) and returns a scalar, a `CPtr` or a `string` it copies from a `const char *` in the return position (reject_ffi_string_array_return). This is not a stylistic limit: the only pointer this compiler allocated that crosses the boundary is a string C reads for the length of the call, and the only one C hands back that the program keeps is a copy it made, so there is nothing for a foreign function to capture or free, which is what lets the escape analysis stay correct without knowing anything about the callee.

A u8, u16 or boolean crosses as C's unsigned char, unsigned short or _Bool, and the caller widens it (#1755). The x86-64 SysV and Apple arm64 ABIs give the caller that job and a callee clang built reads the whole register, so the declare and every call carry zeroext on each such parameter and result; toU8(511) reaches C as 255 (tests/cases/ffi_narrow_zeroext):

declare i32 @widen_u8(i8 zeroext)
declare zeroext i1 @is_odd(i32)
  %3 = call i32 @widen_u8(i8 zeroext %2)

Nish has no signed integer narrower than i32, so no argument takes signext.

A string parameter: borrowed for the length of the call

A string parameter is C's NUL-terminated const char *, and C borrows it: a foreign function may read the bytes until it returns and may not keep the address. A function that keeps its argument — putenv, setvbuf's buffer, a callback's context — is declared to take a CPtr instead, and the program hands it a copy C made (tests/cases/ffi_string_param, ffi_string_param_arena, cookbook):

declare function strdup(s: string): CPtr | null;
declare function putenv(entry: CPtr): i32; // keeps the pointer: C's, not ours

const kept = strdup("MODE=fast");
if (kept !== null) {
  putenv(kept);
}
declare i8* @strdup(i8*)
  %0 = getelementptr inbounds i8, i8* bitcast ({ i64, [10 x i8] }* @.str.0 to i8*), i64 8
  %1 = call i8* @strdup(i8* %0)
  • The argument is the string's own bytes, with no copy. A nish_str stores them NUL-terminated, a literal included, so the call passes the address eight bytes into it. The escape analysis treats the argument as a read, the way it treats a string console.log prints: an arena scope that holds it is released after the call returns, and the call is never a tail call.
  • C may not write through it. A literal lives in read-only constant data and an arena string is shared freely, so a C function that writes into its argument (strtok, mkstemp) corrupts a value the program still holds, or faults. Such a function takes a CPtr buffer allocated through C. The compiler cannot check this rule; it is the declaration's author's word, like the borrow.
  • A string that holds a NUL byte panics before the call, because C would read the NUL as the end of the string — "secret\0.txt" would pass a .txt check in the program and name secret in C. The message names the callee, the argument's position and the parameter: `strlen`: argument 1 (s) holds a NUL byte, which C would read as the end of the string (tests/cases/panics_ffi_nul). That is the ffi-nul panic site. A literal with no NUL in it is proved at compile time, so puts("hello") has no check and no site; every other string is checked with one scan of the bytes C is about to read anyway.
  • string | null passes NULL for null, and checks only a string.
  • The declaration carries no attribute for a string parameter: nocapture and readonly would restate the borrow as an LLVM fact the C body need not honour, which would be a miscompile rather than a broken contract.

A string result: copied out of C's const char *

A declare function that returns string or string | null answers C's NUL-terminated const char *, and the program copies its bytes into the arena at the call. C keeps its buffer — strerror's table, getenv's environment, setlocale's static, which the next call may overwrite — and the copy is the program's own string from then on, placed, kept and released like any other allocation of the function that made the call (tests/cases/ffi_string_return, cookbook):

declare function strerror(errnum: i32): string;
declare function getenv(name: string): string | null;

console.log(strerror(2)); // No such file or directory
const home = getenv("HOME");
  %0 = call i8* @strerror(i32 2)
  %1 = icmp eq i8* %0, null
  br i1 %1, label %ffi.null, label %ffi.copy

ffi.copy:
  %2 = call i8* @nish.ffi.str(i8* %0)
  • The copy is one scan for the NUL and one nish_str_new, in a private helper the module defines once (@nish.ffi.str). C's pointer is never held past it, so a buffer C reuses or frees later cannot reach the program.
  • Every call's copy is new, and is the calling function's allocation, as s.substring(...) is: a loop that keeps one keeps it past the pass, and a using a = arena() block that would release one held outside it is refused, as it is for any fresh string (tests/cases/ffi_string_loops, reject_ffi_string_arena).
  • The copy follows the call, so a call with a string result is never a tail call: a C function that answers a pointer into its argument (strchr, strstr) is read before that argument's arena scope is released.
  • string | null reads NULL as null, tested before the copy but with no panic site.
  • A string result that is NULL panics after the call, because a string cannot hold it: `getenv` returned NULL for a `string` result; declare it `string | null` (tests/cases/panics_ffi_null). That is the ffi-null panic site. Declare the result string | null whenever the C function's manual says it may answer NULL.
  • A result C allocated for the caller to free (strdup, realpath with no buffer) is a CPtr, not a string: the copy would leave C's allocation with no one to free it. Declare it CPtr | null, copy it with readCString, and pass the pointer back to free.

CPtr: the pointer a C function hands back

CPtr is an address C owns. It is eight bytes and i8* in the IR, and that is the whole of what the compiler knows about it: it is not arena memory, not a %struct, not a nish_str, and its lifetime is the C library's business (tests/cases/ffi_pointer, cookbook).

declare function calloc(count: u64, size: u64): CPtr | null;
declare function free(block: CPtr): void;
  • It may be written in a declare function signature and on a local that holds one, and nowhere else. A field, an array element, a Result arm, a type argument, and the parameters or return type of a function this program defines are all `CPtr` cannot be ...: a foreign pointer may only appear in a `declare function` signature or on a local bound to one (reject_ffi_pointer_field, reject_ffi_pointer_array, reject_ffi_pointer_param, reject_ffi_pointer_return, reject_ffi_pointer_type_argument_fn). The first group would put a foreign address inside a value the arena owns and the escape analysis walks; the second would put one across a boundary --emit-header, --emit-dts and --emit-napi describe, and none of the three has a spelling for an address whose provenance and lifetime are unknown. The type-argument clause is stated where a generic function or class is instantiated (reject_ffi_pointer_type_argument), so new Empty<CPtr>() is refused even when no member of Empty<T> names T. The fields that may hold a CPtr are a handle class's and a declare class's, below.
  • A null reaches C only as the literal. A declare function may return CPtr | null and may take one, and a call passes a CPtr | null parameter either the literal null — realpath(path, null), getcwd(null, 0), the manual's "pass NULL to have it allocated" — or a pointer narrowed with if (p !== null). A nullable pointer the program holds is Argument 1 of `free` is a `CPtr | null` the program holds: a nullable foreign parameter takes the literal `null` or a pointer narrowed with `!== null`, so C is never handed a null by a path nobody tested (NL2539, reject_ffi_pointer_nullable_variable), wherever it is held: a local, a parenthesised one, a ternary or ?? that may answer it, inside a constructor reached by new or a method, and through a function parameter a declare function is passed to (reject_ffi_pointer_nullable_specialised). So every NULL C receives is one the source spells, and a pointer that may be NULL because C said so is narrowed before it goes back. The narrowing is the ordinary nullable one (WP6). The literal lowers to i8* null (tests/cases/ffi_owned_string).
  • No arithmetic, no dereference, no index, no member. p + 1 is Operator `+` requires two operands of the same numeric type (reject_ffi_pointer_arithmetic) and p[0] is Cannot index a value of type CPtr. What is left is === null, !== null, comparing two CPtrs, assignment, and passing one to another foreign function.
  • It gets no pointer attributes. dereferenceable, align, nonnull, nocapture and readonly are each a claim about memory this compiler laid out, and it laid out none of this; isPointerParam in src/attributes.ts is an allow-list so that a CPtr falls out of it by construction rather than by a case somebody remembered to write.

readCString: a string out of a buffer C owns

readCString(p: CPtr): string, from nish:unsafe, copies the NUL-terminated bytes at p into a new string of the program's own. It is the copy a string result makes, offered for the pointers a string result cannot be: the ones C allocated for the caller to free. The program copies, then frees C's buffer itself (tests/cases/ffi_owned_string, cookbook):

import { readCString } from "nish:unsafe";

declare function strdup(s: string): CPtr | null;
declare function free(p: CPtr): void;

const owned = (s: string): string => {
  const p = strdup(s);
  if (p === null) {
    return "";
  }
  const copy = readCString(p);
  free(p);
  return copy;
};
if.end:
  %6 = load i8*, i8** %p.addr, align 8
  %7 = icmp eq i8* %6, null
  br i1 %7, label %ffi.null, label %ffi.copy

ffi.null:
  call void @nish_panic(i8* bitcast ({ i64, [34 x i8] }* @.str.3 to i8*))
  unreachable

ffi.copy:
  %8 = call i8* @nish.ffi.str(i8* %6)
  store i8* %8, i8** %copy.addr, align 8
  %9 = load i8*, i8** %p.addr, align 8
  call void @free(i8* %9)
  • It is nish:unsafe because nothing proves the bytes end. The copy scans for the NUL with the @nish.ffi.str helper a string result uses, and a pointer that is dangling, already freed, or to a buffer with no NUL is undefined behaviour: the import is the author's word, as uncheckedGet's is. It is an unsafe call to --deny unsafe, and the CPtr it reads came from a declare function, which is ffi.
  • A NULL pointer panics before the copy, with the ffi-null kind, as readStruct's does: `readCString` read a NULL pointer, exit 1 (tests/cases/panics_ffi_read_cstring_null). The parameter is CPtr, so an unnarrowed pointer is `readCString` expects an argument of type CPtr, got CPtr | null (reject_unsafe_read_cstring) and if (p !== null) comes first, but a declare function declared to answer CPtr may still answer NULL, so the copy tests its pointer: the icmp eq and the branch to ffi.null above. No proof removes the test, so --deny-panics refuses a call in its scope; the copy's oom beside it is allowed. Declare a result that may be NULL as CPtr | null and narrow it, rather than lean on the panic.
  • The copy is new on every call and is the calling function's allocation, as a string result's is: kept past a loop's pass, returned, stored by a constructor and released with its owner, and refused inside a using a = arena() block that would release one held outside it (`readCString(p)` may be allocated inside the `using arena()` block, reject_ffi_owned_string_arena). C's buffer is never held past the call, so free(p) straight after it is safe.
  • On wasm32 it lowers to the same helper. @nish.ffi.str is a byte scan and a nish_str_new, which every runtime has, so readCString adds nothing to the link; whether the program links there is the declare function's question, answered by the libc the target has.

cErrno: the errno a C call left

cErrno(): i32, from nish:unsafe, answers the errno the most recent declare function call on this thread left, and 0 before any. In a program that names it anywhere, the compiler sets errno to 0 immediately before every declare function call and copies errno into a per-thread runtime slot immediately after it returns, before any result is marshalled; cErrno() reads the slot (tests/cases/ffi_errno, cookbook, design):

import { cErrno } from "nish:unsafe";

declare function strtol(s: string, end: CPtr | null, base: i32): i64;

const parse = (s: string): i64 => {
  const n = strtol(s, null, 10);
  return cErrno() === 0 ? n : -1;
};
ffi.ok:
  %2 = getelementptr inbounds i8, i8* %s, i64 8
  %3 = call i32* @nish_errno_ptr()
  store i32 0, i32* %3, align 4
  %4 = call i64 @strtol(i8* %2, i8* null, i32 10)
  %5 = load i32, i32* %3, align 4
  store i32 %5, i32* @nish_errno_slot, align 4
  store i64 %4, i64* %n.addr, align 8
  %6 = load i32, i32* @nish_errno_slot, align 4
  • Saved before anything else runs. The copy of a string result, an arena allocation or a console.log may call into libc and move errno, so the save is the first instruction after the call, ahead of the NULL check of a string result. A C function that sets errno and answers a string is read correctly after the copy.
  • Zeroed before, so a success reads 0. strtol reports an out-of-range number only through errno; with the zeroing, cErrno() after an in-range call is 0 unless the callee set it. No setter is exposed.
  • Per thread. errno is per thread in every libc, and the slot is _Thread_local under --threads (thread_local(initialexec) in the IR), so a thread reads what its own last call left (ffi_errno_threads).
  • Program-wide, and free when unnamed. One cErrno anywhere makes every declare function call in every module of the program pay the zero and the save, and none of them is then a tail call. A program that never names it emits none of it.
  • Not covered: a host-module import (--host) is not a C call; the close a using handle runs at scope exit is the compiler's call, not the program's, and is not captured.
  • Not a global. errno is a macro over a thread-local accessor in every libc, so there is no errno symbol to declare, and declare let errno: i32 is the parser's a module holds only `function`, `class`, `interface`, `const`, `type`, `enum` and `import`, found `IDENT` (reject_ffi_errno_global).
  • Refused: an argument (`cErrno` expects exactly 0 arguments, got 1, reject_ffi_errno_arg), a call without the import (NL2456, reject_ffi_errno_import), and a freestanding wasm32 build, which has no operating system: `cErrno` reaches the operating system, and a wasm32 build has none to reach (reject_ffi_errno_wasm). --profile wasi links wasi-libc and keeps it.
  • It is nish:unsafe and an unsafe call to --deny unsafe, like every export of the module; it reads a defined value and is no panic site.

A u8[] parameter: a buffer and its count

A u8[] or readonly u8[] parameter is a buffer C reads or writes, and a run of one or more of them is followed by exactly one u64: the byte count C may touch in every buffer of the run. Pairing the count by position is the declaration's word, not a proof. fwrite(ptr, size, nmemb, stream) touches size * nmemb bytes, so declared as fwrite(buf: readonly u8[], size: u64, nmemb: u64, f: CPtr) it pairs size with the buffer, and fwrite(buf, 1, 1000, f) on a 10-byte array overflows it. The safe spelling declares the size as the count and passes 1 as nmemb: fwrite(buf: readonly u8[], n: u64, one: u64, f: CPtr), called as fwrite(buf, n, 1, f). Nothing stops a call from passing another nmemb, which is the same class of mistake as declaring free with two parameters.

declare function memcpy(dst: u8[], src: readonly u8[], n: u64): CPtr;
declare function getentropy(buf: u8[], n: u64): i32;
declare function compare(a: readonly u8[], na: u64, b: readonly u8[], nb: u64): i32;
  • memcpy(dst, src, n) is one run of two buffers sharing a count, and (a, na, b, nb) is two runs with a count each (tests/cases/ffi_buffer_declared). The same array passed twice in one run, memcpy(a, a, n), is not refused: C sees two pointers to one block, and overlapping memcpy is undefined in C (memmove is the one that allows it).
  • readonly u8[] is C's const. A u8[] parameter says C may write the bytes; a readonly u8[] says it only reads them. C can cast the const away, and the compiler cannot check that.
  • A run with no count after it is refused, with takes the buffer `md` with no count after it: a run of `u8[]` parameters is followed by one `u64` (reject_ffi_buffer_length_missing, SHA256(d, n, md)'s fixed-size output), and so is a count of any other type, and the parameter after a run of `u8[]` buffers is their byte count, a `u64` (reject_ffi_buffer_length_type).
  • Only u8 crosses. i32[] or f64[] is the only array a declared C function takes is a `u8[]` or a `readonly u8[]` (reject_ffi_buffer_element).
  • A buffer is never null and never a result. u8[] | null is refused (reject_ffi_buffer_nullable, and reject_ffi_buffer_nullable_in_run for one inside a run): it has no length to hold the count to, so the NULL, 0 call C allows is declared with a CPtr | null parameter. A u8[] result is refused by the return rule above (reject_ffi_buffer_return).
  • A call passes the array's data block, and checks the count against every buffer of its run first. Each buffer is lowered to its data pointer, a uint8_t * (i8* in the IR), and the count is compared, unsigned, with the buffer's length before C runs. A count larger than one of them stops the program with an ffi-length panic, `write`: argument 3 (n) counts more bytes than argument 2 (buf) holds, exit 1 (tests/cases/panics_ffi_length, and panics_ffi_length_memcpy for the shorter buffer of a run). The count is a u64, so toU64(0) - toU64(1) is the largest count there is, and fails (panics_ffi_length_wrapped).
  • The pointer and the length are read after the last argument is evaluated, so C sees the array as it is at the call: an argument that pushes or pops before it is seen (tests/cases/ffi_buffer_late_load, panics_ffi_length_shrink, panics_ffi_length_inline_pop).
  • One count is proved: toU64(b.length) of the buffer itself, where b is a const local or a parameter named alone and no later argument may resize an array. Such a call carries no check and is not a panic site, so read(fd, buf, toU64(buf.length)) compiles under --deny-panics. A field path, an alias of the buffer, another array's length and a literal are checked (tests/cases/ffi_buffer_count_proved). --deny-panics refuses any other count and names the one it accepts: pass `toU64(buf.length)` of the buffer itself, which needs no check.
  • C borrows the bytes for the call, and writes through a u8[]. As with a string, the pointer is lent for the length of the call and C may not keep it, so a buffer made in the call's arena scope is released after it (tests/cases/ffi_buffer_read). A u8[] argument is written by the call and a readonly u8[] one is not, and the compiler records it as it records a store. An exported function that hands its own u8[] parameter to such a call writes that parameter: its header drops the const, the wasm glue copies the bytes back, and its define loses readonly (tests/cases/ffi_buffer_export_written). A task a scope spawns may not reach such a call: C's body is unseen, so the call may write memory the parent can see, and the task is refused with writes memory its caller can see (reject_ffi_buffer_call_spawn, and reject_ffi_buffer_scope_race, reject_ffi_buffer_scope_field_race and reject_ffi_buffer_scope_element_race for a call reached through a function).
  • Every kind of call site lowers the same way: a call by name, through a function parameter bound to the declaration, in a constructor reached by new and in a method (tests/cases/ffi_buffer_call_sites), and a call that passes a buffer is refused by --deny ffi like any other foreign call.
  • On wasm32 a u64 count is wider than C's size_t. The data pointer is 32 bits there and the length check is the same 64-bit compare, but a C function whose count is a size_t takes 32 bits: declare its count u64 only on a target where size_t is 64 bits, until a later release gives the count a target-width type. A freestanding wasm build (--target wasm32) refuses a declare function, so a buffer declaration is for --profile wasi there (see "Host modules" below).

A handle class: a class that owns a CPtr

A class with a CPtr or CPtr | null field is a handle class: it owns the C resource the pointer names, and its [Symbol.dispose] names the C function that closes it (tests/cases/ffi_handle_positions, docs/wp53-ffi-buffers-handles.md §4a–§4d).

declare function fopen(path: string, mode: string): CPtr | null;
declare function fclose(p: CPtr): i32;
declare function fgetc(p: CPtr): i32;

class File {
  readonly file: CPtr | null;
  constructor(path: string) {
    this.file = fopen(path, "r");
  }
  read(): i32 {
    const p = this.file;
    if (p === null) {
      return -1;
    }
    return fgetc(p);
  }
  [Symbol.dispose](): void {
    const p = this.file;
    if (p !== null) {
      fclose(p);
    }
  }
}

const first = (f: File): i32 => f.read();

using of a handle class closes it at every exit of its block, once. using f = new File(p) opens f as soon as its initialiser returns, before a later declarator's initialiser runs, and calls the close its [Symbol.dispose] names on every way control leaves the block: falling off its end, return, break, continue, and an orReturn() that propagates, including one in a later declarator's initialiser. A return computes its value first and closes after it, so return twice(n) inside the block is not a tail call. Each declarator opens before the next one's initialiser runs. Falling off the end, break and continue close the declarators in the reverse of the order they opened, beside an arena, a scope or a lock in the same using. A return or a propagating orReturn() joins every scope() before its value is computed (the value may read what the tasks stored), then closes the handles and gives back the locks, innermost first, then releases the arenas (ffi_handle_file, ffi_handle_exits, ffi_handle_arena, ffi_handle_shapes, tests/link/ffi_handle_mutex, tests/link/using_declarator_exit). The close is inlined as the dispose spells it: the field is loaded and the C function called with it, behind a null test only when the dispose writes one (const p = this.f; if (p !== null)); a dispose that is the single call close(this.f) calls it with the field as it is, null included (ffi_handle_shapes). The using is where the program reaches ffi, so --deny ffi refuses it and names the close (caps_ffi_handle, reject_ffi_handle_deny).

A panic and process.exit do not close: the process ends, and the operating system takes back the descriptor. What it does not take back is what a C library still buffers. panic, expect and a failed range entry end through exit(1), which flushes stdio, but a failed check and out of memory end through _exit(1), which does not, so bytes written to a FILE * with fwrite and not yet flushed are lost on a bounds panic. Flush (fflush) before code that may panic when those bytes matter.

  • The class. Exactly one CPtr or CPtr | null field, readonly and with no initialiser (reject_ffi_handle_two_fields, _field_mutable, _field_initialiser). No type parameters, no implements, no export (_generic_class, _implements, _export), and no static readonly json (_json).
  • The write. The constructor writes the field once, as this.file = g(...) at its top level, with g a declare function and no exit before it (_field_copy, _field_copy_twice, _field_copy_branch, _field_copy_non_foreign).
  • The read. The pointer is read only as this.file inside a method of the class, passed straight to a declare function or bound once to a const (_field_read_outside, _field_other_instance, _field_alias_copy, _field_alias_let, _field_position): The pointer `file` of handle class `File` cannot be used here.
  • The close. [Symbol.dispose](): void is close(this.file); or const p = this.file; if (p !== null) { close(p); } for a declare function close of one parameter (_no_dispose, _dispose_body, _dispose_arity, _dispose_arg_type, _dispose_unknown, _dispose_forms, _dispose_shapes). A class declares one, with no modifier (_dispose_twice, _dispose_static). Nothing else may call that close on the field, in a method or in the constructor (_close_call), and [Symbol.dispose] is never called by name (_dispose_call).
  • Where a File may be. It is opened by using (once lowered), called (f.read()), or lent to a parameter of type File, which may lend it on. Everything else is A `File` cannot be used here (_position): new outside a using initialiser, this leaving the constructor, a copy into another binding, an array element, an object literal, a capture, a builtin's argument, a comparison (_new_outside_using, _this_escapes, _alias, _array, _object_literal, _closure_capture, _builtin_arg, _walk_compare).
  • Where File may be written. Only as the type of such a parameter. A local's annotation, a field, a return, a Result arm, a nullable, a type argument and an exported function's parameter are `File` is a handle class, and its name may be written only as the type of a parameter that borrows one (_type_position, _field, _return, _result, _nullable, _type_argument, _exported_param).
  • Not after its close, and not kept past a lend. A using binding's scope is its block, which ends where the close is emitted, so a handle named after its block — after it, in the loop's next iteration, in a do loop's condition, in a for loop's update, in a later case, in the else, later in a constructor — names nothing: Unknown builtin `f.read` or Unknown identifier `f` (_use_after_close, _use_after_close_loop, _use_after_close_do_while, _use_after_close_for_update, _use_after_close_switch, _use_after_close_else, _use_after_close_ctor). A using written straight into a case clause is refused by the clause rule (_use_after_close_switch_clause). A lent handle cannot be kept by its callee. The value rule refuses a local (let saved = f, _use_after_close_keep), an alias (_alias), an array (_array) and a scope() task (_scope_task); the type rule refuses a return (_return) and a type argument (_type_argument), among them a Map or a Channel of the class (_use_after_close_keep_map, _use_after_close_keep_channel); both refuse a field of another class, set in a method, through new or as h.l = this (_use_after_close_keep_field, _walk_ctor_param, _use_after_close_keep_this). Some shapes are refused by a general rule first: a closure by the arrow-capture rule (_closure_capture), and a module binding by the absence of a top-level let and the rule that a module constant is never assigned (_use_after_close_keep_module, _use_after_close_keep_module_expr). In its own constructor this is half built, and is neither lent, called, stored nor aliased (_ctor_this_lend, _ctor_this_receiver, _ctor_this_store, _ctor_this_alias).

Two things stay C contracts, as a buffer's count does. The class's methods pass the field only to functions that use it and return, and that neither close what it owns nor hand it back: freopen, a close through a C wrapper with another name, close(fileno(p)), or a C function returning its argument are not seen by the rules above. And the constructor's foreign call returns a pointer the object owns: two objects built from one shared pointer would both close it.

A declare class: a C struct, lent by pointer

A declare class is a C struct: its fields are laid out in declaration order, each at its natural alignment, and the size is rounded up to the widest alignment, as a C compiler lays out the same fields. new Name() takes no arguments and zero-fills it, and a declare function parameter typed Name borrows it as a struct Name * (tests/cases/ffi_struct_param, ffi_struct_layout, cookbook, docs/wp27-ffi.md §9b):

declare class Timespec {
  sec: i64
  nsec: i64
}
declare function clock_gettime(clock: i32, ts: Timespec): i32

const now = new Timespec()
clock_gettime(1, now)
console.log(now.sec)
%struct.Timespec = type { i64, i64 }
declare i32 @clock_gettime(i32, %struct.Timespec*)
  %Timespec.obj = alloca %struct.Timespec, align 8
  %0 = bitcast %struct.Timespec* %Timespec.obj to i8*
  call void @llvm.memset.p0i8.i64(i8* align 8 %0, i8 0, i64 16, i1 false)
  store %struct.Timespec* %Timespec.obj, %struct.Timespec** %now.addr, align 8
  %1 = load %struct.Timespec*, %struct.Timespec** %now.addr, align 8
  %2 = call i32 @clock_gettime(i32 1, %struct.Timespec* %1)
  • new zero-fills every byte, padding included, with one llvm.memset of the struct's size: C may compare or hash the whole struct, and a zeroinitializer store need not write the padding (ffi_struct_layout).
  • A CPtr field is the target's void *: eight bytes natively, four under --profile wasi (wasm32), where the struct's size and offsets are the ones clang gives the same C struct for that target (tests/ffi-struct.js).
  • A field is i32, i64, u8, u16, u32, u64, f32, f64, CPtr, CPtr | null, or another declare class, embedded by value — C's struct timespec it_value;, not a pointer to one. A number field is refused whatever the mode and however it is spelled — in parentheses, through a type alias or a chain of them, imported or not — since --number-mode picks its width and a C struct has one layout. The resolver decides it where it meets number, so no spelling escapes; an alias of i32 stays a field: has type `number`, whose width `--number-mode` chooses (NL2755, reject_ffi_struct_field_number, _paren, _alias, _alias_paren, _alias_chain; tests/ffi-struct.js: the imported alias, the i32 alias). Any other type is Field `ok` of `declare class Flags` has type `boolean`, which has no C layout (NL2752, reject_ffi_struct_field_type, reject_cls_declare, reject_ffi_struct_field_array, reject_ffi_struct_field_class): a boolean, a string, an array and a Nish class have no C spelling the compiler could promise. A struct cannot embed itself, directly or through another (NL2749, reject_ffi_struct_self, reject_ffi_struct_cycle).
  • A field is plain and has no initializer, and the class has nothing but fields: no readonly, static or optional field (NL2747), no initializer, because new zero-fills (NL2748), no constructor, method, accessor or other member such as an index signature (NL2751, reject_ffi_struct_index_signature), no type parameter (NL2741, reject_generic_declare_class), no extends or implements (NL2750), and no export (NL2746) — each pinned by a reject_ffi_struct_* case.
  • It stays in its module. It cannot be a parameter or the return type of an exported function or an exported class's method, or a field of an exported class or interface: `declare class Point` cannot be a parameter of an exported function (reject_ffi_struct_export_param, _return, _method, _field, _interface). A type that holds one is refused there too, however deep — through Result, T | null, an array, a generic instance or a class of this program: `Wrap` holds a `declare class Point` and cannot be a parameter of an exported function (NL2756, reject_ffi_struct_export_result, _generic, _holder, _holder_nullable). So no .d.ts or N-API binding ever describes one, and --emit-header, which lists every class of the program, declares a class that holds one only as an opaque typedef struct Holder Holder; (tests/ffi-struct.js, docs/wp27-ffi.md §9g).
  • It is a native build's. A wasm module a page loads — freestanding wasm, or any build that writes --emit-dts — links no C to lend a struct to, and its IR is laid out for wasm32 later: `declare class Point` declares a C struct, and a wasm module a page loads links no C to lend it to (NL2757, tests/wordings/nl2757_declare_class_emit_dts, nl2757_declare_class_wasm32). A --profile wasi build is wasm32 from the start and keeps it.
  • It is built with new and nothing else. new Point(1) is refused as for any class without a constructor (reject_ffi_struct_new_args), and an object literal typed as one is an object literal cannot lay out a C struct (NL2744, reject_ffi_struct_literal).
  • C borrows it for the length of the call, as it borrows a string: it may read and write the fields until it returns, and may not keep the address. The escape analysis treats the argument as a read, so an instance that does not otherwise escape stays in the caller's stack frame; a C function that keeps the pointer (an aio_read control block, a struct handed to a thread) is outside the contract. Its accesses carry no TBAA tag, so a field read after the call sees what C wrote, and a function of this program that passes its parameter on to C writes through that parameter, so the parameter is not readonly and its caller's read sees the write too (ffi_struct_param).
  • Name | null passes NULL for null, under the rule a CPtr | null parameter follows: the literal null, or a value narrowed with !== null (NL2539, reject_ffi_struct_nullable_held).
  • An embedded struct is part of its owner. stamp.at.sec reads and writes a field in place, and clock_gettime(1, stamp.at) lends C the address inside stamp; those are the only two uses. Holding it — const at = stamp.at, or passing it to a function this program defines — is cannot be held as a value (NL2743, reject_ffi_struct_nested_bind, reject_ffi_struct_nested_nish_arg), and replacing it is a `declare class` embedded in its object is written field by field (NL2745, reject_ffi_struct_nested_assign).
  • It is not a return type or an array element. C returning a struct by value, or a pointer into memory it owns, leaves the copy with no owner on this side: a C struct is only lent to C by pointer: take a `Point` parameter for C to fill (NL2742, reject_ffi_struct_return); a struct C owns is copied out with readStruct (below) from the CPtr C returns. An array of them would be a C array the program lays out element by element, and is cannot be an array element (NL2740, reject_ffi_struct_array, reject_ffi_struct_array_literal, reject_ffi_struct_array_new), however the array's type is spelled.
  • Its fields are C's to fill. Typed JSON does not decode into one or encode one: a `declare class`: C reads and writes its fields (NL2753, reject_ffi_struct_json). A spread copies a flat one's fields into a class of this program, but not one that embeds another, whose embedded struct is not a value: which a spread cannot copy (NL2754, reject_ffi_struct_spread).
  • --emit-ts refuses a module that declares one, with the same message as for a declare function (NL4010): it is C's layout, and JavaScript has none.

readStruct: a struct out of memory C owns

readStruct(p) from nish:unsafe copies the struct at a CPtr into a new instance of the declare class its context names, and the copy is the program's (tests/cases/ffi_struct_read, cookbook, docs/wp27-ffi.md §9h). It is how a struct C keeps comes back: localeconv()'s struct lconv, getpwnam's struct passwd, each a buffer the next call overwrites.

import { readStruct, readCString } from "nish:unsafe"

declare class Lconv {
  decimalPoint: CPtr
  thousandsSep: CPtr
}
declare function localeconv(): CPtr

const lc: Lconv = readStruct(localeconv())
console.log(readCString(lc.decimalPoint))
%struct.Lconv = type { i8*, i8* }
  %0 = call i8* @localeconv()
  %1 = icmp eq i8* %0, null
  br i1 %1, label %ffi.null, label %ffi.copy

ffi.null:
  call void @nish_panic(i8* bitcast ({ i64, [48 x i8] }* @.str.0 to i8*))
  unreachable

ffi.copy:
  %2 = call i8* @nish_alloc_struct(i64 16)
  call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 8 %2, i8* align 1 %0, i64 16, i1 false)
  %3 = bitcast i8* %2 to %struct.Lconv*
  store %struct.Lconv* %3, %struct.Lconv** %lc.addr, align 8
  • The layout comes from context, as Ok(v)'s does: an annotation, a parameter, a return type or an assignment's target, | null stripped. With none it is `readStruct(p)` needs a contextual `declare class` type to copy into (NL2764), and a context that is not a layout is `readStruct(p)` copies into a `declare class` layout, and Plain is not one (NL2765, reject_unsafe_read_struct_context). A type argument is not written at a call site: readStruct<Lconv>(p) is refused with the annotated spelling in the message (NL2763, reject_unsafe_read_struct_type_args).
  • One copy, the layout's size: the arena allocation new makes, then one llvm.memcpy of the struct's bytes, padding included, from the source at align 1, since C promised the struct and not its address's alignment. The size is the target's: 16 bytes for { CPtr; CPtr } natively, 8 under --profile wasi (tests/ffi-struct.js). After the call C may free or rewrite its struct; the copy is an instance like any new one, and a copy made in a callee and returned is held by its caller.
  • It does not copy deep. A CPtr field of the copy is still C's pointer, read with readCString while C keeps it live.
  • A NULL pointer panics before the copy, with the ffi-null kind: `readStruct` read a NULL pointer into a `Lconv` (tests/cases/panics_ffi_read_struct_null). The argument is a CPtr, so a CPtr | null is narrowed first (`readStruct` expects an argument of type CPtr, got CPtr | null, reject_unsafe_read_struct_null), but a declare function declared to answer CPtr may still answer NULL, so the copy tests its pointer: the icmp eq and the branch to ffi.null above. No proof removes the test, so --deny-panics refuses a call in its scope.
  • The rest of the pointer is not checked: not that it is live, not the size it covers. The layout must be C's struct or a prefix of it. That is why it is in nish:unsafe: refused by --deny unsafe and recorded by --emit-checked.
  • There is no writeStruct. A struct C should fill is lent to it, as above.

What a declaration cannot say

Each of these is refused, and each refusal is pinned by a case:

  • A variadic C function, spelled variadically. A rest parameter is Rest parameters are not supported here before its type is read, ...bytes: u8[] included (reject_ffi_variadic, reject_ffi_buffer_rest). A variadic function declared with a fixed arity, printf(fmt: string, x: i32), is accepted and is the declaration's contract: the compiler cannot tell open is variadic from its name.
  • syscall, by name: `declare function syscall` is refused (reject_ffi_syscall). It is variadic and bypasses libc; declare the libc function that wraps the system call.
  • A callback. C never calls into Nish: an arrow passed to a declared C function is refused (reject_ffi_callback), and a function reference is a tag only this program's dispatchers read, not an address, so it is not a CPtr either.
  • A read through a CPtr. p[0] is Cannot index a value of type CPtr (reject_ffi_pointer_deref); readCString and readStruct above are the two reads.
  • Inline assembly. asm("nop") is an unknown function (reject_ffi_asm).
  • An allocator from C memory. arena() takes no argument, so using a = arena(p) is refused (reject_ffi_allocator).

A foreign declaration has no body (reject_ffi_body) and cannot be exported (reject_ffi_export), because export offers other modules a function this module defines.

A declared function's symbol is its name as written, in whatever package declares it. A function a package defines carries the package's prefix (@probe.probeAbs), but a declare function names the C function, and C has one namespace, so a package or a std/ module that declares abs calls @abs exactly as the program's own modules do (tests/link/ffi_package_declare). Two declarations of one name must agree: the same parameter and return types, which is what a C prototype is, so parameter names and the aliases a type is spelled through do not count (tests/link/ffi_package_declare_same). Agreeing declarations are one C function, whether they are in two packages or in two modules of the same package, which was refused as a duplicate symbol before. Declarations that disagree are refused, naming both: C has one namespace, so every `declare function` of one name must agree on its parameter and return types. A buffer's readonly, a | null and the number of parameters each count (tests/link/ffi_package_declare_clash, tests/link/ffi_package_declare_mismatch). So is a function whose emitted symbol is a declared name: an exported function of the program's own package, or a private one whose name is not reserved to C: a `declare function` binds the C symbol of its name in every package, so rename the defined function (tests/link/ffi_package_declare_root_clash). A private function named like a C library function is emitted as abs$, and a generic one's instantiations as labs$i32, so neither takes the C symbol and both are accepted (tests/link/ffi_package_declare_apart). The two refusals are refused rather than linked because the whole-program analysis keeps one set of facts per symbol, and the two would share it.

The declaration carries no LLVM attributes, and its callers pay for it. Nothing about a body the compiler cannot see is provable, so a function that calls C loses readnone and willreturn, and so does everything above it in the call graph; a function that calls no C keeps both (tests/cases/ffi_scalar). nounwind is the one exception, and it is a decision rather than a proof: unwinding out of a foreign call is undefined in this language, the same position clang takes compiling C, because there is no throw and no landing pad with which to do anything else. See docs/wp27-ffi.md §2 and the cookbook entry.

A program that calls C is no longer one the compiler can reason about end to end. That is the trade the feature is: the whole-program fact fixpoint is the performance thesis, and a foreign call is a hole in it.

And it is outside the differential oracle by construction. The oracle runs a program's frozen JavaScript rewrite against runtime/shim.mjs, on the premise that the same source means the same thing in both worlds; that premise does not hold for a source whose meaning is "whatever this C function does" (wp27-ffi.md §5). The evidence for an FFI program is arranged the other way round: a golden .ll, an llvm-as pass, a native round trip against real libc, and the attribute assertions.

Host modules: the functions a page supplies

A wasm module calls a function of the page that loads it by importing it from a host module: a TypeScript file of the page's, named by --host <file> or by the root package.json's "nish": { "host": [...] } (tests/host-import, docs/wp50-wasm-host.md §3). An import from any other file is an ordinary Nish import.

// host.ts, the page's
import type { f64, u32 } from "nish";
export type Canvas = HostHandle<"canvas">;
export function fill(c: Canvas, rgb: u32): void { canvases[c].fill(rgb); }
export function nowMs(): f64 { return performance.now(); }
declare void @fill(i32, i32) #1
attributes #1 = { "wasm-import-module"="host" "wasm-import-name"="fill" }
  • Only the exported signatures are read. Each export function's parameter and result types, each exported enum, and each export type X = HostHandle<"kind">. A function body is skipped as a balanced run of tokens, so it may use anything TypeScript has. Text the lexer cannot skip, a regex literal or JSX, refuses the file with move that code into a module the host file imports (NL3049); other statements are skipped unread.

  • What crosses. The integer and float widths, boolean, a ranged integer, an enum, a HostHandle, a string parameter, and void as a result. i64 and u64 are a bigint on the page's side (import type { i64 } from "nish"). Refused, each naming the explicit form: an object, a class, a function, unknown or any (NL2661), a generic function (NL2662), a Promise (NL2663) and a string result (NL2667): the page writes text into module storage instead (nish/host/text). An array crosses only as a buffer window, below.

  • A string argument is its UTF-8 bytes, lent for the call. It lowers to two wasm values, the address of the string's bytes and their count, read from the string's own header, so nothing is copied on the module's side; the loader's wrapper decodes them with TextDecoder into a JS string before the page's function runs, and an invalid sequence becomes U+FFFD. The string may be a literal, a concatenation or a template; one the module built lives in the export's arena, which is released after the export returns (tests/cases/host_import_string; tests/run.js, section host-import-text; cookbook):

    // web/host.ts
    export function say(s: string): void { console.log(s); }
    // app.ts
    say(`hello ${n}`);
    
    declare void @say(i8*, i32) #2
    %1 = call i8* @nish_str_concat(i8* bitcast ({ i64, [7 x i8] }* @.str.0 to i8*), i8* %0)
    %2 = bitcast i8* %1 to i64*
    %3 = load i64, i64* %2, align 8
    %4 = trunc i64 %3 to i32
    %5 = getelementptr inbounds i8, i8* %1, i64 8
    call void @say(i8* %5, i32 %4)
    
  • Each host function is a declaration with the wasm import attributes: module host, its own name. It carries no other attribute, and its callers lose readnone and willreturn as for a C declaration. wasm-ld makes it an import with no --allow-undefined.

  • A host module exists only in a freestanding wasm build. A native or --target wasm32-wasi build that imports one is refused (NL3050). A build that makes a page module (--target wasm32, or --emit-dts with no --target) refuses a declare function (NL2666): the module links no C, so the declaration would be an import the page does not supply. --target wasm32-wasi --emit-dts links C, so it keeps declare function legal.

  • The loader and its types. --emit-dts adds Imports = { host: typeof import("./host") } to the .d.ts, and load takes it. The loader passes a function through by reference when every value crosses as it stands, and wraps it once, at load, when a value converts. A u32 or u64 argument arrives signed and is read unsigned (>>> 0, BigInt.asUintN(64, …)); a u8 or u16 argument needs nothing, since the module zero-extends it. A u8 or u16 result is masked, a boolean crosses as 0 or 1, and a ranged integer result is checked by the loader's rangeOut (RangeError naming the function).

  • --emit-ts keeps the import as written, pointing at the host file, so the TypeScript reading calls the same functions.

  • A call to a host function is an ffi call: --deny ffi refuses it and --emit-capabilities names it.

An array crosses as a buffer window: the array, then at: u32, n: u32 (tests/host-window, docs/wp50-wasm-host.md §5.3). The host lends the page the elements [at, at + n) of the array the program passes, in place, with no copy:

// host.ts, the page's
import type { u32 } from "nish";

export function doubleF32(xs: Float32Array, at: u32, n: u32): void {
  for (let i = at; i < at + n; i++) xs[i] *= 2;
}
// the program
const DATA = new Float32Array(64)
export const tick = (): void => { doubleF32(DATA, 16, 8) }
  • Six elements lend: u8[], u16[], u32[], i32[], f32[] and f64[], their readonly forms and their typed-array names. Any other array is NL2681 (a buffer window lends only `u8[]`, `u16[]`, `u32[]`, `i32[]`, `f32[]` or `f64[]` (or the typed array of each): copy the values into one of those). The two parameters after it must be there (NL2682, a buffer window with no `at: u32, n: u32` after it) and be u32, the width of a wasm32 address (NL2683). NL2664, which refused every typed array before windows existed, is retired.
  • The window is checked in the module, before the page runs. at and n are zero-extended and added in i64, so a sum that wraps in 32 bits still fails, and held against the array's length; past the end is an ffi-length panic, `doubleF32`: argument 2 (at) plus argument 3 (n) reaches past the end of argument 1 (xs), numbered and named from the host function's own parameters (cookbook). A call that lends 0, toU32(xs.length) of a local array xs is proven in bounds and emits no check. A window of module storage always keeps the check, even 0, toU32(DATA.length): the range pass proves facts about locals, not about storage.
  • What the page gets is a view of all of memory and an index. The loader's adapter passes a whole-memory view of the element type, the array's first element plus at as the index, and n, so the host function's xs[at] is the program's DATA[at]. A page that indexes outside [at, at + n) reaches the rest of the module: the page is trusted, and it reads only its window (security record).
  • Module storage lends to a window like any array of a lending element, and so does a local array. A record storage does not.

Module storage is the page's view of the module. Under --profile wasm-host the link exports each storage's data, and --emit-dts types what the loader builds from it:

  • layout, a Layout with one entry per storage, keyed by its name (by its symbol when two modules export the same name): { at, length, stride, fields? }, where at is the byte address in memory, stride the element's size, and fields each record field's byte offset within the element. length, stride and each offset are literal types in the .d.ts, so a page that hard-codes one, or reads a field the module does not have, fails tsc once the layout changes. at is read from the instance at load and never moves. A record field is read through the whole-memory view of its width: views().f32[(layout.bodies.at + i * layout.bodies.stride + layout.bodies.fields.x) >> 2].
  • views(), the whole memory as buffer and as each typed array (u8, i8, u16, i16, u32, i32, f32, f64, i64, u64), and storage.<key>, a typed view of each scalar storage (a boolean one as Uint8Array, an enum or range as Int32Array, a HostHandle as Uint32Array). views() answers the same object until a growing memory replaces memory.buffer, so a page calls it again after any call into the module rather than keeping one view across calls.

Under --profile wasm-host a panic is the page's to report, in a program with module storage or a host module (tests/panic, docs/wp50-wasm-host.md §6). The nish.panic import, NishPanic, the dead module, NishReentry (below), and the onPanic and sites options of load and loadSync are for such a program only; loadSync itself is in every loader. Any other program gets the .wasm, loader and .d.ts it got before: a panic is a bare RuntimeError trap, the module stays usable, and its next call runs. The module of such a program imports one function from module nish, panic, and every runtime panic calls it with its kind and the words it would print natively; the loader turns that into an exception:

  • The call that panicked throws NishPanic, an Error whose message is the native text (index out of range: 9 >= 4, slice out of range: [a, b) of length len, an overflow's text, out of memory, the program exited with code 3, or panic(…)'s own words) and which carries kind (panic, index, slice, overflow, memory, exit, divide, or trap, reentry or exception for a module that died some other way) and site, the panic site's number (-1 for none). An array's xs.slice out of range throws with kind panic, not slice: its check is std/array.ts's own panic(...), though the message reads as string slice's does.
  • Before it throws, the loader calls onPanic. load(bytes, { onPanic, sites }) takes the reporter; the default, reportPanic from nish/page, writes nish: <message> to console.error, as a native program writes it to stderr. An exception the reporter throws is swallowed: the NishPanic is thrown either way.
  • sites names where. The wasm-host link writes <out>.sites.json (app.wasm.sites.json for -o app.wasm), [{ kind, at }] with one entry per panic site, indexed by site. Passed to load (parsed), it ends each message with at path:line:col; without it the message is the native text alone. Under --profile wasm the module imports nothing and each panic is a bare trap, with no message and no site; the same loader still marks the module dead on it, so later calls throw as above.
  • A panic ends the module. Its memory survives, as it was, for a page that reads it, but every later call throws NishPanic (the module is dead: …) before the module runs. An exception a host import throws out of a call ends the module the same way, since the call it interrupted never finished.
  • A bad argument does not end it. The loader checks every argument's shape before the module runs: a number of the wrong type or range, a Result that is not an object with a boolean ok and a payload of its arm's type, an array argument whose brand (read off %TypedArray%.prototype, not instanceof) is not the typed array it names. Each throws a TypeError; an error a Result argument's getter throws reaches the caller as it is. Either way the module never ran and stays alive (tests/run.js, section panic: badarg, badshape).

One call at a time. A host import that calls an export while a call into the module is still running gets NishReentry (a call into the module was made while another call into it was running), and the inner call never runs. The reporter is different: the module is already dead when onPanic runs, so an export it calls throws NishPanic (the module is dead: …). The outer call's state is half changed at that point, which is why the loader refuses rather than runs it (docs/security/wasm-host.md, WH-3). Left uncaught, the NishReentry escapes the outer call too, and like any exception out of a call it ends the module. A page that needs the module's answer schedules the call for after the import returns. In a module that links the runtime, a call also gives back, once its result is read, everything it allocated in the arena, so a long-lived page does not grow the module's memory call by call.

HostHandle<K> is a nominal i32 for an object the page keeps (docs/wp50-wasm-host.md §4). K is a string literal type, and HostHandle<"a"> and HostHandle<"b"> are different types, here and under tsc, where runtime/nish.d.ts makes it a branded number or 0, so tsc also refuses any number but 0 where a handle is wanted. A handle supports === and !== against a handle of its kind or 0 ("none"), and is stored wherever a scalar is; it has no arithmetic and no conversion to or from a number, so it is made only by a host function that returns one. A field's or a parameter's default may only be 0 (NL2670). A second type argument, or a kind that is not a string literal of letters, digits and _, is NL2669, and a string literal type anywhere but a HostHandle's kind is NL2668. It lowers to i32, so a program with no host module emits nothing new.