Declarations
Program structure
A module (one .ts file) may contain, at the top level, only function
declarations (a const bound to an arrow, or the legacy function keyword),
class and interface declarations, const declarations
(Module constants), type aliases
(Type aliases), numeric enum declarations
(Enums), declare function declarations
(Calling C), import statements, and export modifiers on
those declarations. The entry module may instead hold
top-level code, which runs as its main. In any other
module, and in an entry that declares main, any other top-level statement
is rejected with
Only top-level function declarations are supported in Phase 1 (found <Kind>),
where <Kind> is TypeScript's name for the statement, ExpressionStatement
or IfStatement (tests/cases/reject_top_level_stmt); a top-level let
names the reason instead (Top-level `let` is not supported; a module has no top-level code, so only `const` is available,
reject_const_top_level_let), and var is refused wherever it is written
(Forbidden constructs). Modules have no top-level code, so there is no
initialisation order to worry about — which is exactly why a top-level const
must be a value the compiler can compute for itself. The one piece of state a
module keeps is module storage, a const array laid out
and zero-filled at compile time, which is a declaration, not code.
Top-level code
An entry module that declares no main may hold statements at the top level,
the way a script does. They run as the program's main: the compiler moves
them, in order, into an exported main(): void before anything else reads
the tree, so the native program is the one a hand-written main gives and
exits 0 unless the code calls process.exit (tests/cases/script_basic,
script_exit).
const LIMIT: i32 = 4
const capped = (n: i32): i32 => (n > LIMIT ? LIMIT : n)
const start = 1
let total = start
for (let i = 0; i < 3; i++) {
total += capped(i + 2)
}
console.log(total) // 10
-
What moves: every top-level statement (an expression,
if, a loop,switch, a block, a labelled statement,throw,try), every top-levellet, and every top-levelconstthat is not exported and that no declaration names.return,breakandcontinuestay refused. -
What stays: functions, classes, interfaces, enums, types, imports, and
a
constthat a function, a class or another staying constant names. Thatconstis a module constant with all its rules, since a function cannot read a local ofmain;LIMITabove is one. A variable of the code that a function names is refused with`count` is a variable of the entry module's top-level code, which runs as `main`, so a function cannot read it(reject_script_local_in_function). -
Declare before the code reaches it. Under Node a top-level statement runs
where it stands, before the declarations below it exist; natively they all
exist first. So a statement that reaches a function, class, enum or
constant, by naming it or through a function that does, must stand below
it:
`twice` is declared below the top-level code on line 5 that reaches it(reject_script_declared_below). -
One scope. Under Node the code's variables share the module scope with
its declarations, so a variable may not take a name the module declares or
imports:
`go` is a variable of the top-level code and also a name the module declares or imports(reject_script_name_collision). -
Only the entry, and only without
main. An imported module has no top-level code, and neither has an entry that declaresmainin either spelling (reject_top_level_stmt,reject_const_top_level_let). An entry of nothing but constants is a module, not a script. -
There is no top-level
await:awaitis written only inside anasyncfunction (NL2641, Async functions), so a script that waits declaresexport async function maininstead. -
--emit-tswrites the code back where it stands, and itsentry.tsimports the module rather than callingmain.
Functions
A function is declared as an arrow bound to a module-level const
(wp22-arrow-functions.md):
const add = (a: number, b: number): number => {
return a + b;
};
const double = (n: i32): i32 => n * 2; // a concise body is that one `return`
The function keyword declares the same thing and is accepted as the legacy
spelling; the two compile to identical IR, instruction for instruction
(tests/cases/fn_arrow).
-
The arrow form is a
const—letisFunction `f` must be declared `const`, not `let`(tests/cases/reject_arrow_let) — binding one name (const f = (): i32 => 1, g = 2isA function declaration binds one name,reject_fn_two_names), with the signature on the arrow itself. Annotating theconstisFunction `f` takes its signature from the arrow; drop the annotation on `f`(reject_arrow_annotated): the signature is written once, on the arrow. A function returningvoiduses a block. -
A concise body (
=> n * 2) means exactly what a block with onereturnmeans, everywhere the difference could show. It takes the return type as its contextual type, so an object literal gets its struct, a class value converts to an interface it implements, a numeric literal takes the declared width and[]takes its element type; and a value it allocates is returned rather than local, so the caller brackets the call with the arena reclaim (Memory model).tests/cases/fn_arrow_concisepins all five, each of which the block form has always had. -
A capture-free top-level function is a value in either spelling: a
name bound to a non-generic, non-async function this program defines, with
no rest parameter, is a value of its function type, so
const alias = doubleis a function reference. It can be held in a local, a field, an array element, a return value or a type alias, passed, called and compared with===and!==, and nothing else (tests/cases/fnref_field,fnref_array,fnref_eq,fnref_stores). An arrow that is not a top-levelconstis no reference: one that captures isthis arrow captures `k`: a function reference names a top-level function and carries no environment(tests/cases/reject_arrow_as_value), and one that does not isan arrow in value position is not yet a function reference: declare it as a top-level function and name it(reject_fnarg_arrow_value). -
Every parameter and the return type must be annotated
(
Parameter `x` needs a type annotation;Function `f` needs an explicit return type annotation,tests/cases/reject_missing_return_type,reject_arrow_return_type). The return type is asked of every function and method as the module is read, a generic one nothing instantiates included (reject_method_return_template,reject_method_return_generic). -
Functions may call each other in any order; signatures are collected
before bodies are checked (
tests/cases/locals,cf_fib). This holds for the arrow form too, which is not what a reader who knows that aconstis not hoisted in JavaScript would expect: a module-levelconstbound to an arrow is a function declaration, so it is in scope for the whole module and not only below its own line, and two of them may call each other (tests/cases/fn_arrow_hoisting). Nothing else would let a file be converted from one spelling to the other without reordering it (wp22-arrow-functions.md §8b). -
Calls take exactly the declared number of arguments (
expects 1 argument,tests/cases/reject_arity), each of exactly the declared type (Argument 1 of `area`: expected Shape, got Rect,tests/cases/reject_cls_not_implements). -
Parameters are immutable:
p = 1,p++,p += 1areCannot assign to `p` because it is a parameter(tests/cases/reject_assign_param,reject_cf_incdec_param). -
Parameters and the body's top-level block share one scope:
let xin the body off(x)isDuplicate declaration of `x`(CLI only). -
A function's name shares one declaration namespace with classes,
interfaces, type aliases, enums and module constants, whichever declaration
came first. A class or interface of the same name is
`Point` is already declared as a class or interface(tests/cases/reject_cls_name_is_function); everything else is`value` is already declared in this module(tests/cases/reject_fn_after_const). -
A function may be generic, and each instantiation is compiled on its
own; see Generic functions. Rejected forms:
generators (
reject_generator), anasyncfunction without itsPromise<T>(Async functions,reject_async_function), destructured parameters (Destructured parameters are not supported,reject_param_destructured), optional parameters (x?: T,Optional parameters are not supported,reject_param_optional, which namesx: T = <default>instead; the compatibility dialect'soptional-paramsfeature accepts one read through??, The compatibility dialect), a default or rest parameter anywhere but the declarations Default and rest parameters lists (reject_param_default,reject_param_rest), overloads (Duplicate function `f`,tests/cases/reject_fn_duplicate; a signature with no body,function f(): void;, isFunctions must have a body; the compatibility dialect'soverloadsfeature accepts signatures before the implementation), an anonymousexport default function ()(Functions must be named,reject_fn_anonymous), function expressions, an arrow anywhere but as a top-levelconstor a function argument (reject_fnarg_arrow_value,reject_arrow_as_value), and nested function declarations (Unsupported statement in Phase 1: FunctionDeclaration,reject_fn_nested, in a template nothing instantiates too,reject_fn_nested_template; animportin a body isUnsupported statement in Phase 1: ImportDeclaration). -
A non-
voidfunction must return on every path (Function `f` must return a value of type i32 on every path,tests/cases/reject_missing_return,reject_cf_missing_return_loop); see Termination.
Default and rest parameters
const scale = (x: number, by: number = 2): number => { return x * by }
const area = (w: number, h: number = w): number => { return w * h }
const total = (...xs: i64[]): i64 => { let s: i64 = 0; for (const x of xs) { s = s + x } return s }
scale(3) // scale(3, 2)
area(4) // area(4, 4)
total() // total([])
total(1, 2, 3) // total([1, 2, 3])
A function, a method with a body, a constructor, and an arrow bound to a
module const may give trailing parameters a default or end in one rest
parameter. Both are filled in at the call site: the callee is compiled as
if every argument were always passed, with no branch on a missing one, so a call
that passes every argument compiles to the same IR it did before the function
had a default (tests/cases/param_default, param_rest, param_method,
param_generic, across modules tests/link/param_modules).
-
A default is a constant or an earlier parameter: a numeric literal (a
negative one too),
true/false, a string literal, an enum member, or the name of a parameter written before it, ornullfor aT | nullparameter (next rule). Anything else is refused (reject_param_default_expression,reject_param_default_later,reject_param_default_array,reject_param_default_fraction). -
A
T | nullparameter defaults tonull, and only tonull:opts: Options | null = nullmakes a call that leavesoptsout pass a typed null pointer, the same IR as passingnullitself (tests/cases/param_default_null, across modulestests/link/param_modules). This is how an options argument is written withoutundefined:fetch(url)andfetch(url, init)are one function.nullis refused as the default of a type with nonull(reject_param_default_null_not_nullable), any other default of aT | nullis refused, since every call that omits it would build the value (reject_param_default_object), andT | nullof a type parameter takes no default (reject_param_default_null_generic). -
A default's parameter is annotated,
n: i32 = 1, nevern = 1(reject_param_default_untyped), and its type is a number,boolean,string, enum or aT | nulldefaulting tonull(reject_param_default_parameter_type), not a type parameter (reject_param_default_generic). Every parameter after one with a default has one too (reject_param_default_order). -
A rest parameter is last, has an array type and no default
(
reject_param_rest_not_last,reject_param_rest_not_array,reject_param_rest_default), and a function has a default or a rest, not both (reject_param_rest_and_default). The call gathers its trailing arguments into one array of exactly that many elements, on the stack unless the callee lets it escape — the same array an explicit literaltotal([1, 2, 3])builds. -
A rest of records takes only fresh object literals. The array copies each
inline record (an interface no class implements) into its slot, so a named
record passed through a rest would be a copy the callee's write never
reaches, where TypeScript would write the caller's own:
bumpAll(p)isA rest parameter copies each `Cell` record it gathers, ...(reject_param_rest_record). Pass[p]to aCell[]parameter, where the copy is written out. A class instance is a reference, so a rest of class instances gathers the caller's objects. -
Not on a function-typed parameter, which a call names by value
(
reject_param_function_default,reject_param_function_type), and a function with a rest parameter is never passed for a function parameter (reject_param_rest_function_value) nor held in a function reference (NL2729,reject_fnref_rest): called through either, Node would gather differently. -
No spread argument:
total(...xs)isSpread arguments are not supported(reject_param_spread_call); passxsto a parameter of array type. -
Too few or too many arguments name the range the function takes
(
reject_param_arity). -
A host sees neither. Defaults and rest are filled in by a Nish call
site, so an exported function's symbol takes every parameter: the
--emit-dtstypings, the--emit-headerprototype and the--emit-napishim require each argument, and take a rest parameter as one plain array (wp8-interop.md). -
What a call evaluates is what TypeScript evaluates: each argument once, left
to right, whether it fills a default's place or gathers into a rest
(
tests/cases/param_eval_order). A rest array the callee returns, stores in a field or lets a caller keep is allocated in the arena, not on the stack (param_rest_escape); a default on an imported class's constructor or method is filled in by the importing module (tests/link/param_default_import).
Module constants
const TOKEN_NAME: i32 = 1;
const TOKEN_END: i32 = TOKEN_NAME + 1;
export const PROMPT: string = "> ";
A top-level const names a value the compiler already knows. It is not a
global variable: no symbol is emitted, no initialiser runs, and every use of
the name lowers to the value itself — AREA below is the literal 64 in the
IR, not a load (tests/cases/const_basic, const_fold, dump_checked_const).
That is what lets a module keep its table of kinds and limits while still
having no top-level code and therefore no initialisation order.
-
The type annotation is required, as on every other top-level declaration
(
Module constant `answer` needs a type annotation,tests/cases/reject_const_no_annotation), and so is an initialiser. -
The type must be
i32,i64,f64,number,boolean, orstring. A class or interface constant would need an allocation, and there is no code to run it (must be a number, boolean, string, or a `readonly` array of numbers,tests/cases/reject_ffi_handle_module_binding). There are two exceptions with rules of their own: areadonlyarray of numbers written out as a literal, which is a constant table, and anew Array<T>(n)or typed-arraynewof a compile-time length, which is module storage. -
The initialiser is an ordinary Nish expression restricted to literals
and other constants: the operators of Operators over numeric,
boolean, and string literals, other module constants (declared anywhere in
the module, or imported), and parentheses. Anything else — a call, a
new(other than module storage's), a local, a parameter — isA module constant's initialiser must be a literal, another constant, or arithmetic over them(tests/cases/reject_const_not_constant,reject_const_unknown_name). There is deliberately no second, larger language insideconst: a constant can compute exactly what a runtime expression can. -
Folding follows the language's own arithmetic, whichever mode the
compiler is in, because a constant must compute what the instruction it
replaces computes. By default that instruction is checked, so a fold that
overflows is
attempt to compute with overflow in a constant— the run-time operation would panic, and a constant has no run time to panic at (tests/cases/reject_const_overflow_arith). A wrap the program means is writtenwrappingAdd,wrappingSuborwrappingMulfromnish:unsafe, the one call a constant's initialiser may make, and that fold wraps:const WRAPPED: i32 = wrappingAdd(2147483647, 1)is-2147483648(tests/cases/unsafe_wrapping). Negation is arithmetic too, so-2147483648is fine (its result fits) while-(-2147483648)is refused. A bare integer literal takes the width of the constant it initialises, soconst BIG: i64 = 1000000000 * 10is computed in 64 bits (const_i64). Mixing widths is an error, as it is anywhere else (reject_const_mixed_widths).f64folds in IEEE-754 (const_f64), and+on two strings concatenates at compile time, interning the result once (const_string). -
The two failures the divisor check catches at run time are refused at
compile time instead:
1 / 0andMIN / -1in a constant areattempt to divide by zero in a constantandattempt to divide with overflow in a constant(tests/cases/reject_const_div_zero; see Checked integer division). -
A value that does not fit its annotation is an error, not a wrap:
const TOO_BIG: i32 = 3000000000isConstant `TOO_BIG` does not fit in i32(tests/cases/reject_const_overflow), and a folded value of the wrong type isdeclared i32 but its value is string(reject_const_type_mismatch). -
A constant cannot be assigned:
Cannot assign to `LIMIT` because it is a module constantcovers=,op=,++and--(tests/cases/reject_const_assign,reject_const_incdec). -
A local shadows a constant of the same name, as it would in TypeScript:
the scope chain is consulted first (
tests/cases/const_shadow). -
A constant may name a constant declared later, in the same module or in
an imported one, because folding is by need rather than in source order. A
cycle is
Module constant `A` is defined in terms of itself(tests/cases/reject_const_cycle). -
A string constant is a value, so it is a receiver like any other string:
NAME.lengthis the folded literal's byte length (tests/cases/const_string_member).
Constant tables
const CLASS: readonly u8[] = [0, 0, 1, 1, 2, 3]
const STEPS: readonly i32[] = [-1, 0, 1, LIMIT]
A top-level const of type readonly T[] whose initialiser is an array
literal is a constant table: its elements are folded at compile time and laid
out as one read-only data global, with a constant array header over it, so no
code runs to build it and nothing can write it (tests/cases/const_array_u8,
const_array_numeric). A use of the name is the header's address — the same
readonly T[] value any other read-only array is — so CLASS[c],
CLASS.length, for (const v of CLASS) and passing it to a readonly T[]
parameter all work, and CLASS[c] with c: u8 into a 256-entry table is one
load the bounds prover has already discharged.
-
The element type is
u8,u16,u32,i32,i64orf64— the numbers whose bytes are their value. Anything else isConstant table `XS` cannot hold string(tests/cases/reject_const_array_element_type). -
The annotation must be
readonly T[](orReadonlyArray<T>): the data is constant, and a plainT[]could be written through.const XS: i32[] = [1, 2, 3]ismust be declared `readonly i32[]`(tests/cases/reject_const_array). -
Each element is a constant expression under the rule above — literals,
other module constants and arithmetic over them
(
tests/cases/reject_const_array_nonconst) — and must fit the element type:256in areadonly u8[]isElement 2 of constant table `XS` does not fit in u8(tests/cases/reject_const_array_element). An empty literal is refused (has no elements,reject_const_array_empty). -
Writing is refused like any write to a
readonlyarray: an element store,push,popor any other mutating method (tests/cases/reject_const_array_write,reject_const_array_push), and handing the table to a mutableT[]— a parameter or a local (reject_const_array_mutable,reject_const_array_alias). -
The symbols are
@nish_const_<module>_<name>.data(private unnamed_addr constant) and@nish_const_<module>_<name>(the header,internal constant),<module>the module's index. An exported table follows module storage's rule: the header in its own module is notinternal, and an importer declares itexternal constant.
Module storage
A top-level const whose initialiser is new Array<T>(n), or new of one
of the typed-array names, is module storage: a fixed array the module keeps
between calls, laid out by the compiler and zero-filled before the first call,
so no code runs to build it and the module still has no top-level code
(tests/cases/storage_module, docs/wp50-wasm-host.md §5). It is the one
place a module keeps state, and the place a page reads and writes a wasm
module's memory in place (Host modules).
It is legal in every build, native included.
interface Body { x: f32; vx: f32; hits: u32; alive: boolean }
export const SLOTS: i32 = 8
export const bodies = new Array<Body>(SLOTS)
export const scale = new Float64Array(2)
export const step = (): void => {
for (let i = 0; i < bodies.length; i++) {
if (bodies[i].alive) bodies[i].x = bodies[i].x + bodies[i].vx
}
}
@nish_storage_0_bodies.data = global [8 x %struct.Body] zeroinitializer, align 16
@nish_storage_0_bodies = global %struct.nish_array { i64 8, i64 8, i8* bitcast ([8 x %struct.Body]* @nish_storage_0_bodies.data to i8*) }, align 8
-
Two globals.
<symbol>.dataholds the elements and<symbol>is an ordinary array header over them, so every read and write is the indexing an array already lowers to. The symbol isnish_storage_<module>_<name>, with<module>the module's place in the program, so two modules may each keep a private storage of one name, and none takes a C symbol such aswrite. The data has external linkage: the--profile wasm-hostlink exports it, and the loader reads its address from that export. The header isinternalunless the storage is exported; another module reaches an exported one as@<symbol> = external global %struct.nish_array(cookbook). -
The length is known at compile time: a literal, a module constant, or
arithmetic over them (
wrappingAddand the othernish:unsafefolds included), from 1 to 16,777,216 elements and at most 256 MiB (268,435,456 bytes). Anything else isModule storage `X` needs a length known at compile time: a literal, a module constant, or arithmetic over them, because the page reads its layout from the type(NL2676,reject_storage_dynamic_length), and a length outside the range isModule storage `X` has length 0; a storage holds from 1 to 16777216 elements(NL2680,reject_storage_length_zero; past 256 MiB the message names the element's size,reject_storage_too_many_bytes). -
The element is pointer-free, and zero is one of its values: a number, a
boolean, an enum or a ranged integer whose zero is a member, aHostHandle, or an interface no class implements whose fields are all of these. A string, an array, a class, aMapor a| null(butC | nullfor a classC, which is root storage) isModule storage `X` cannot hold T: its bytes are zero-filled where a page can read them, …(NL2686,reject_storage_string_element,_array_element,_class_element,_map_element,_nullable_element,_record_string_field). An interface a class implements, generic classes included, is a class's shape and is refused the same way (reject_storage_implemented_interface,_generic_implementer). A field may not be another record, since a record field is a pointer to it:Module storage `X` cannot hold Out.p (In): a record field that is itself a record is a pointer to it, and storage holds no pointer; spell the inner fields on the outer record(NL2685,reject_storage_nested_record). An enum or range without a zero isModule storage `X` is zero-filled, and 0 is not a value of T; give the type a zero member, or store the number and convert it where it is read(NL2687,reject_storage_enum_no_zero,_range_no_zero). -
An annotation must say what the
newmakes:const xs: f64[] = new Int32Array(4)is NL2679 (reject_storage_annotation). The annotation may be left off. -
It never moves and never resizes.
push,pop,sort, any other method andX.length =areModule storage `X` has a fixed length and a fixed place, so `X.m` is not available: index it, and keep a count beside it for the slots in use(NL2688,reject_storage_push,_pop,_sort,_length_assign).X[i]andX.lengthare what remain. -
Its name is a place, not a value. Binding, returning, storing or
passing
Xitself isModule storage `X` is a place, not a value: index it (`X[i]`) or read `X.length`, lend it only to a parameter declared `u8[]`, `Int32Array`, `Float32Array`, `Float64Array` or `BigInt64Array` or to a host function's buffer window, and never alias, return or store the array itself(NL2677,reject_storage_pass_whole). The two exceptions are lendings: a scalar storage passed to a parameter spelled with a typed-array name (which has nopush,poporlength =), as inconst total = (xs: Int32Array): i32 => …; total(SCORES)(tests/cases/storage_lend), or a byte storage passed to au8[]orreadonly u8[]parameter, the spelling a byte buffer has (tests/cases/storage_lend_bytes, whichnish/host/textwrites through), and to a host function's buffer window (Host modules). A record storage is never lent. A lending to a function of the same module holds only while that function keeps the parameter in place: it may index it, read.length(never assign it or++it), loop over it withfor of, call a method that reads it (at,indexOf,slice,map, …), callfill,sort,reverseorcopyWithinas a statement, copy it withString.fromBytes, and lend it on to another such parameter or a buffer window. Anything else (binding it under another name, returning it, passing it to ani32[]parameter, which couldpush) isModule storage `X` cannot be lent to `f`, which lets its parameter `a` go further than indexing: …(NL2684,reject_storage_lend_respell_arg,_const,_return). The rule holds wherever the callee is declared: a function another module declares is judged by its own body, read in its own module, so a lending on from there resolves the next callee by that module's names — its imports and import aliases (import { good as sum }) included — and is judged the same way, for as many hops as it lends on (tests/link/storage_lend_import,reject_storage_lend_import). A callee name the calling function binds itself (a parameter, a local, a pattern) is not a declared function and is refused. -
A record element is a place too.
X[i].xreads and writes a field, andX[i] = X[j]orX[i] = vcopies a whole element, field by field. Binding, returning or storingX[i]is NL2674 (reject_storage_bind_element); passing it isAn element of module storage `X` cannot be passed as an argument: it is a place, not a value; pass the index, or the fields the callee needs(NL2675,reject_storage_pass_element), exceptout.push(X[i])onto an array of the same record, which copies; andfor (const p of X)over a record storage is NL2678 (reject_storage_for_of): loop over the index. Afor-ofover a scalar storage binds numbers and is allowed. -
It is memory every function reaches, so it moves the inferred
attributes: a function that reads storage is at most
readonly, one that writes it is neitherreadonlynorreadnone(tests/cases/storage_attributes), and aparallelMapbody that writes it is refused as any shared write is (reject_storage_parallel_write). -
Under
--emit-tsit is a typed view, and only it. The reading writes one more file,nish-storage.ts, holding oneArrayBufferlaid out as the wasm build'slayoutsays,memory({ buffer, grow }, whosegrowthrows: the buffer never moves),layoutitself,views(),loadSyncandloadas the wasm loader has them (they take the module's exports where the wasm loader takes its bytes), and each storage as a view of the buffer; the module'snewis written as that view (src/emit-ts-storage.ts,docs/wp50-wasm-host.md§5.4). A scalar storage is the typed array its element reads as (Int32Array,Float32Array, …; abooleanone is a byte array behind a proxy that reads!== 0). A record storage is an array of objects whose fields are getters and setters over the buffer, soX[i].x = vwrites the bytes a page reads, and a whole-element copy isObject.assign. Each storage starts on an 8-byte boundary from 0, so the reading's offsets differ from the wasm build's; a page reads them fromlayoutand never spells one. A localf32[]is still a plain JavaScript array there: only module storage is a view.
Root storage: an object graph kept between calls
Module storage whose element is C | null for a class C is root
storage: each slot holds one object, and everything that object reaches —
its fields, their arrays and strings, the objects they hold — outlives the
call that built it (tests/cases/root_storage,
docs/wp55-persistent-root.md). It is how a wasm module keeps a game, a
document or any class instance across init, step and draw.
class World {
balls: Ball[]
tick: i32 = 0
constructor() {
this.balls = []
}
}
const world = new Array<World | null>(1)
export const init = (): void => {
world[0] = new World()
}
export const step = (): void => {
const w = world[0]
if (w !== null) {
w.tick = w.tick + 1
}
}
On a page module each root is a record of its data, its length and the
descriptor of its class, gathered into one table the keep walks
(tests/cases/root_storage_page, whose root is head, a list of Node):
%nish.root = type { i8*, i32, i8* }
@nish_storage_0_head.root = constant %nish.root { i8* bitcast ([1 x %struct.Node*]* @nish_storage_0_head.data to i8*), i32 1, i8* bitcast (%nish.trace* @nish.trace.0 to i8*) }
@nish.roots = internal constant [1 x %nish.root*] [%nish.root* @nish_storage_0_head.root]
define void @nish_root_keep(i64 %mark) {
call void @nish_runtime_root_keep(i64 %mark, i8* bitcast ([1 x %nish.root*]* @nish.roots to i8*), i32 1)
ret void
}
- Natively it is module storage and nothing more. A native program never hands its arena back between calls, so the graph simply stays.
-
On a page module the loader keeps the graph and releases the rest.
Where the loader of a program without root storage releases each export
call's arena, this one calls
nish_root_keep. The runtime traces what the roots reach from descriptors the compiler writes (%nish.trace, one per class, string and array type the roots reach;tests/cases/root_storage_page), slides it down over the garbage, rewrites every pointer to it, and frees what is left. It does so only once the kept region passes twice the live bytes and 64 KiB, so a call that changes a little copies nothing, and memory stays bounded however long the page runs (node tests/run.js persist: 100,000 steps with memory flat). -
What a root reaches is traceable. A number, a
boolean, an enum, a string, an array, a class or a discriminated union of classes. Anything else — aMap, aSet, an interface a class implements, aRegion, aSecret, aCPtr, a handle class or a C struct layout — isRoot storage `X` cannot keep C.f (T): the runtime moves what a root reaches between calls, …(NL2766,tests/wordings/nl2766_root_storage_map). -
It is not exported: the objects move, so a page has no layout to read.
export const world = new Array<World | null>(1)isRoot storage `X` cannot be exported: the runtime moves the objects its elements point to between calls, so a page has no layout it could read; export a function that reads it instead(NL2767,tests/wordings/nl2767_root_storage_export), andlayoutleaves root storage out. -
No threads, no regions. Each allocates from an arena of its own, which
a root could be left pointing into, so root storage in a program that
loads
nish/threadsornish:regionisRoot storage `X` cannot be kept in a program that loads `nish/threads` or `nish:region`: …(NL2768,tests/wordings/nl2768_root_storage_threads). -
No native shim. A build that writes
--emit-napi,--emit-napi-asyncor--emit-headergets a shim that releases the arena after a call that passes a string or an array, which would free what a root reaches, so root storage there isRoot storage `X` cannot be kept in a build that writes a native shim (`--emit-napi`, `--emit-napi-async` or `--emit-header`): …(NL2769,tests/wordings/nl2769_root_storage_napi). A plain native program keeps it. -
No WASI loader. A
--profile wasibuild with--emit-dtswrites a loader that releases the arena after every call, and only a page module emits thenish_root_keepa loader calls instead, so root storage there isRoot storage `X` cannot be kept in a WASI build that writes a loader (`--emit-dts`): …(NL2770,tests/wordings/nl2770_root_storage_wasi_dts). -
A freestanding module without a loader (
--target wasm32and no--emit-dts) still emitsnish_root_keep(mark); the host that calls its exports writes its own loader, and must callnish_root_keep(mark)where it would callnish_arena_release(mark), or what a root reaches is freed. - Every function may store through a root, so a program with root storage does not prove a parameter-only allocation stays in its caller's scope from the signature alone; the compiler's other proofs are unchanged.
Type aliases
type Byte = u8;
type Bytes = Byte[];
type Reading = Sample | null;
A type alias is a second name for a type that already exists. It is not a
type of its own: type Byte = u8 says that Byte and u8 are two spellings
of one type, exactly as Int32Array is a spelling of i32[] (see
Types). So sameType never sees the alias's name, there is no new
LLVM type, and an alias emits nothing at all — the same program written with
and without its aliases produces byte-identical IR, which is what
tests/cases/type_alias_ir and tests/cases/type_alias_expanded are: one
program twice, with one golden between them.
-
The right-hand side may be any type the language accepts: a scalar,
string,void,T[],readonly T[],T | null,Result<T, E>, a declared class or interface, or another alias (tests/cases/type_alias_ir). Everything is refused exactly where it would be in an annotation, sotype Bad = anyis`any` is forbiddenand not a rule of its own, and an object type,type Point = { x: i32; y: i32 }, is the syntax error an inline object type is anywhere else (tests/cases/reject_type_object_literal_alias): declareinterface Pointinstead. -
Order does not matter. An alias may name a class, an interface or an
alias declared further down the file, because resolution is by need rather
than in source order — the rule module constants follow. A cycle is
Type alias `Feet` is defined in terms of itself(tests/cases/reject_type_alias_cycle), the wording the constant cycle uses, because it is the same mistake. - An alias is resolved even when nothing uses it, so a broken right-hand side is reported where it is written rather than never.
-
Generic aliases are forbidden, unlike a generic function: an alias
only renames a type that already exists, so there is nothing for a type
argument to apply to.
type Box<T> = T[]isGeneric type parameters are forbidden on a type alias in Nish; a generic function, class or interface is monomorphised, and an alias only renames a type that already exists(NL1054,tests/cases/reject_type_alias_generic). -
A built-in type name may not be aliased.
type string = i32is`string` is a built-in type name and cannot be used for a type alias(tests/cases/reject_type_alias_builtin). A keyword likestringis resolved from the syntax, andi32,Array,ReadonlyArray,Resultand the typed-array names before any declared name, so such an alias would otherwise sit there meaning nothing. -
The name shares one declaration namespace with functions, classes,
interfaces and module constants: a clash is
`Point` is already declared in this modulewhichever came first (tests/cases/reject_type_alias_duplicate). -
An exported alias is imported, and in the importer it is the type it
names.
export type Pt = Pointinshapes.tsandimport { Pt } from "./shapes"elsewhere makePta second spelling ofPointthere too: a field, parameter or return type, a local, an array element, and aPointfromnewpasses where aPtis declared (tests/link/alias_export_class). The right-hand side is resolved in the module that wrote it, whichever module asks first, soexport type Conn = Socket | nullworks whenSocketis a class the exporter itself imported, and the importer never has to importSocket(alias_export_imported_class). An array, aT | nulland aResultalias are the types they name (alias_export_array,alias_export_nullable,alias_export_result), so is an alias of a generic class's instantiation, which the exporter instantiates even when an importer asks first (alias_export_generic), a chain of aliases across modules resolves by need (alias_export_chain:TableisLine[],Lineis an importedRow,RowisCell[],Cellisi32), and unlike a class an alias may be renamed on import —import { Conn as Handle }— because it has no symbol for its name to be part of (alias_export_rename). Nothing is emitted in either module, so the importer's IR is byte-identical to the same program with every alias written out (alias_export_irandalias_export_ir_local, whose.llgoldens are the same files).-
A mistake on the right-hand side is the exporter's. It is reported
against the module that wrote the alias, even when another module asked
for the alias first (
tests/link/alias_export_broken), and a cycle that runs through two modules' aliases isType alias `B` is defined in terms of itself, at the first alias of the cycle the report reaches (alias_export_cycle). -
The import errors are the ones every import has. An alias declared
without
exportis`Conn` is declared in `./conn` but not exported (add `export`)(tests/link/alias_import_not_exported); a declaration of the imported name is`Conn` is already declared in this module(alias_import_clash); a second import of it is`Conn` is already imported from `./conn`(alias_import_duplicate); and an alias is a type, not a value:new Conn()isUnknown class `Conn`, as it is for a local alias (alias_import_as_value). A module does not re-export an alias it imported. An imported alias is bound before the signatures, for the reason an imported enum is.
-
A mistake on the right-hand side is the exporter's. It is reported
against the module that wrote the alias, even when another module asked
for the alias first (
-
Aliases are top-level only.
type X = i32inside a function body isUnsupported statement in Phase 1: TypeAliasDeclaration(CLI only).
Enums
enum Kind {
If = 1,
While = 2,
Return = 3,
}
enum Level {
Low, // 0
Mid, // 1
High, // 2
}
enum Role {
Admin = "admin",
Guest = "guest",
}
An enum is of one of two kinds. A numeric enum's members are integers;
a string enum's members are each given a string literal, and its rules
past the ones below are in String enums. Either kind
declares a distinct type whose representation is i32: a numeric
member's integer, or a string member's ordinal. Distinct is the point, and it
is stricter than TypeScript, where a numeric enum member is assignable to
number and a string one to string: the rule this language is built on is that
two values are compatible only when their types are identical
(Types), and an enum that were silently i32 would be the one type
compatible with something it is not spelled as. So Kind and i32 are two
types, in both directions, as Role and string are, and the only values of
Kind are the members it declares.
Nothing is emitted for a numeric enum (a string enum adds its one table of
strings, String enums). A member is folded to its integer the way a module
constant is folded to its value, so Kind.While lowers to the literal 2, a
Kind parameter is an i32 parameter, and there is no symbol, no table and no
%struct anywhere. tests/cases/enum_ir and tests/cases/enum_expanded are
one program written twice — with the enum, and with i32 module constants —
and their .ll goldens are byte-identical files.
-
A member's value is a numeric literal, or the one before it plus one. Auto-numbering is TypeScript's: the first member without an initialiser is
0. An enum any of whose members is given a string literal is a string enum instead. A member that is neither — a computed value in a numeric enum, or a numeric member beside a string one — isEnum members must be numeric literals in Nish, or every member a string literal (an enum lowers to a plain integer, and one numbering cannot mix with strings)(NL1043,tests/cases/reject_enum_computed,reject_enum_string_mixed); a numeric literal that is not an integer isEnum member `Kind.Half` must be an integer literal(reject_enum_float), and one outside the width isEnum member `Kind.Big` does not fit in i32(reject_enum_overflow). -
An enum must declare at least one member, because an empty one is a type no value can inhabit:
Enum `Kind` must declare at least one member(tests/cases/reject_enum_empty). Two members of one name areDuplicate member `If` in enum `Kind`(reject_enum_duplicate_member). -
A value is spelled
Kind.Ifand nothing else. A name that is not a member isEnum `Kind` has no member `Nope`(tests/cases/reject_enum_unknown_member), and there is no conversion in either direction:const n: i32 = kisCannot initialize i32 variable `n` with Kind(reject_enum_not_i32) andconst k: Kind = 1isCannot initialize Kind variable `k` with i32(reject_enum_from_i32). -
===and!==are the only operators. Two values of one enum compare; everything else is refused because an enum is not numeric —+isOperator `+` requires two operands of the same numeric type or two strings, got Kind and Kind(tests/cases/reject_enum_arith),<isOperator `<` requires two numeric operands, got Kind and Kind(CLI only), and the bitwise operators areOperator `|` requires two operands of the same integer type, got Flag and Flag(reject_enum_bitwise). Bit flags therefore stayi32module constants:A | Bon an enum would produce a value that is no declared member, which is the one thing the type exists to prevent (wp23-language-surface.md §10 question 1 leaves the wider form open). -
A
switchtakes an enum discriminant and enum-member labels (tests/cases/enum_ir), which is what an enum is for: the statement lowers to LLVM'sswitchexactly as an integer discriminant does. A label of any other type is`case` label is i32 but the discriminant is Kind(reject_enum_case_label), so the integer a member stands for is not a label. Seeswitch. -
A member is also a type:
Kind.Whilenames the type whose one value is that member. A slot of a member type — a field, a parameter, a return, a local, an element, an object-literal field or aResultarm — holds that member and nothing else, which is what lets areadonlyfield record which of an enum's cases its object is, as the tag a later discriminated union reads:class ArrayNode { readonly kind: Kind.While = Kind.While; }The slot is entered by the member's own constant, or by a value already of that member type; the enum's other members, and any
Kindvalue that is not a constant, areType `Kind.Array` holds only the member `Kind.Array`, got `Kind.Scalar`(NL2519,tests/cases/reject_enum_member_type_other) and…, got a value of type `Kind`(reject_enum_member_type_widened), because aKindcould be any member. Every way into a slot is held to it: a call argument, an assignment to aletor a field, apush, an element write, an array literal, an object-literal field,Ok,unwrapOr's default, a ternary arm and areturn(reject_enum_member_type_argument,_assign,_field_assign,_push,_element_write,_array_literal,_object_field,_result_ok,_unwrap_or,_ternary,_return), and so is an argument of a generic call, whether the member type binds a type parameter (fold(xs, Kind.Array)withxs: Kind.Array[]), sits in a concrete parameter beside one, or isgetOrInsert's value (tests/cases/enum_member_type_generic,reject_enum_member_type_generic,_fold,_get_or_insert,_generic_variable,_generic_method,_class_param); a parenthesised constant,(Kind.Array), is the constant. Two generic callstscaccepts are refused, because the language binds a type parameter only from the arguments, and binds it from the constant before it reads a function argument:const y: Kind.Array = id(Kind.Array)bindsTtoKind, since the slot the call returns into binds nothing (reject_enum_member_type_generic_return), andapply(take, Kind.Array)bindsTtoKindbeforetake: (x: Kind.Array) => Kind.Arrayis read (reject_enum_member_type_generic_function_arg). Pass a value already of the member type,id(a)andapply(take, a), and both bindKind.Array. An unannotatedletkeeps the member type of a value already of it, astsc's does, and widens a member constant to its enum:let w = keep(a)holds onlyKind.Array,let w = Kind.ArrayanyKind(reject_enum_member_type_inferred_let). So does every place whose type is inferred from what is written: a ternary whose arms are bothKind.Array, an array literal of them ([a]is aKind.Array[]),??between twoKind.Array[], each of those passed to a generic function, and an arrow's result, which binds the type parameter it returns:map([a], (k) => k)is aKind.Array[]andapply(a, (k) => k)aKind.Array, while an arrow that answers the constant,(k) => Kind.Array, answersKind, astscwidens it (tests/cases/enum_member_type_inferred,enum_member_type_arrow,reject_enum_member_type_ternary_let,_ternary_const,_ternary_generic,_array_inferred,_array_let,_array_generic,_nullish,_arrow_map,_arrow_apply). Two arms or two elements that differ where one is a member type, or an array of one —c ? a : boverKind.ArrayandKind.Scalar,c ? a : kover aKind,c ? a : Kind.Array,[a, Kind.Array], or[[a], [Kind.Array]]— are refused,Ternary branches must have the same type, got Kind.Array and Kind.Scalar: an enum member type is not widened to its enum, so annotate the type it holds, e.g. `Kind`, and the array literal's message reads the same way, namingKind[]for the nested case:tsctypes them as a union of the members (the enum itself when they are all of its members), or as the member, or as the enum, by whether the constant is fresh, so the variable is annotated instead (reject_enum_member_type_ternary_mixed,_ternary_enum,_ternary_constant,_ternary_nested,_array_mixed,_array_constant,_array_nested_mixed). An object literal is always annotated, so its fields always have a type to enter. Two member types of one enum are two types (reject_enum_member_type_between), and a field initializer of a member type is held to the same rule (reject_enum_member_type_field_init). The other direction is free: a member type converts to its enum, and a read of a member-typed slot is the enum wherever a value is computed with, so it compares with===, switches and passes as aKind, and converts toi32no more than aKinddoes (reject_enum_member_type_read). In IR a member type is its enum'si32:tests/cases/enum_member_typelowers to the same bytes as the program with every member type writtenKind(enum_member_type_widened, whose golden is byte for byte the same).-
The qualifier is an enum, local or imported, and the name one of its
members.
import { Kind as K }spells the exporter's member typesK.Array(tests/link/enum_member_type_import). Anything else before the dot is`Point.x` is not a type: only a member of an enum may be named with a dot in a type, and `Point` is not an enum(NL2517,reject_enum_member_type_not_enum), a missing member isEnum `Kind` has no member `Missing` to name as a type(NL2518,reject_enum_member_type_unknown), and type arguments areUnsupported type reference `Kind.Array<i32>`(reject_enum_member_type_args). One dot is read;A.B.Cis a syntax error (reject_enum_member_type_nested). -
Two members of one value are one type, as
tscreads them, spelled by the member declared first (a string enum has no two such members, String enums): withList = 1besideArray, aKind.Listslot is reported as`Kind.Array`(reject_enum_member_type_same_value), andKind.Listenters aKind.Array[](tests/cases/enum_member_type_slots). -
A member type is a type wherever one is written: a
typealias's right-hand side (type ArrayTag = Kind.Array), a type argument (Box<Kind.Array>,Map<Kind.Array, i32>,Set<Kind.Array>), an element type and an arrow argument's return type, and a-gbuild describes it as its enum (enum_member_type_slots). -
A zero-filled array of a member type holds 0, so
new Array<Kind.Array>(n)is`new Array<Kind.Array>(n)` would zero-fill with 0, which is not the member `Kind.Array`; build it with `[]` and `push` instead(NL2520,reject_enum_member_type_zero_fill), and a member whose value is 0,Kind.Scalarhere, is allowed (enum_member_type_slots). TheparallelMap,parallelFromandparallelReducetemplates zero-fill the array their workers write, so a body answering such a member type is refused at the call,… answers the member type `Kind.Array`, and `parallelMap` zero-fills the array its workers write, which a `Kind.Array` cannot hold; declare the body to answer `Kind`(NL2521,reject_enum_member_type_parallel,_parallel_from,_parallel_reduce); a body answering the member at 0 runs in all three (tests/link/enum_member_type_parallel). -
A field initializer may be a member's constant when the field is the
enum or one of its member types:
readonly kind: Kind = Kind.Ifstores the integer before the constructor runs, as a numeric literal would (enum_member_type_slots). Another enum's member, a member the enum does not declare, a member given to a number field and any other dotted initializer are refused (reject_enum_member_type_field_init_enum,_field_init_level,_field_init_unknown,_field_init_i32,_field_init_dotted). -
Typed JSON refuses a member-typed field (
tests/cases/json_reject_enum_member): it reads an enum as any of its members, which a member type cannot hold. The one exception is a union's tag: a member that tags a variant of a union is written as its name, or a string enum's member as its string, the union's typed JSON tag (tests/link/json_tagged_union,tests/link/json_enum_string). A member type crosses to C and JavaScript exactly where its enum does, which is nowhere.
-
The qualifier is an enum, local or imported, and the name one of its
members.
-
A built-in type name may not be taken:
enum i32 { ... }is`i32` is a built-in type name and cannot be used for an enum(tests/cases/reject_enum_builtin_name), for the reason the same rule exists for atypealias — the name would never be looked at. The name shares the one declaration namespace with functions, classes, interfaces, type aliases and module constants: a clash is`Kind` is already declared in this modulewhichever came first (reject_enum_duplicate). -
An exported enum is imported, and in the importer it is the exporter's enum.
export enum Kind { ... }inkinds.tsandimport { Kind } from "./kinds"elsewhere bind one type, not a copy: the importer may useKindas a field, parameter or return type, a local, aMapkey and aswitchdiscriminant,Kind.Iffolds to the integerkinds.tsgave it, and aKindcrosses a call between the two modules as itself (tests/link/enum_export_uses). One enum reached along two import paths is one type (enum_export_diamond), two modules may import each other's enums (enum_export_cycle), and unlike a class an enum may be renamed on import —import { Kind as Level }— because it has no symbol for its name to be part of (enum_export_rename). Nothing is emitted in either module, so the importer's IR is byte-identical to the same program with the enum declared in it (enum_export_irandenum_export_ir_local, whose.llgoldens are the same files).-
Every enum declaration is a type of its own. A
Kinda module declares and theKindit could have imported are two types, so a member of one is not an argument where the other is declared:Argument 1 of `weight`: expected Kind, got Kind(tests/link/enum_import_distinct). An imported enum converts to and fromi32no more than a local one does (enum_import_i32,enum_import_from_i32). -
The import errors are the ones every import has. An enum declared
without
exportis`Kind` is declared in `./kinds` but not exported (add `export`)(tests/link/enum_import_not_exported); a declaration of the imported name is`Kind` is already declared in this module(enum_import_clash,enum_import_clash_function); a second import of it is`Kind` is already imported from `./kinds`(enum_import_duplicate,enum_import_duplicate_function); and an enum is not a function, imported or not:Kind(2)isUnknown function `Kind`(enum_import_as_call). A module does not re-export an enum it imported. -
Why it is bound before the signatures. An enum is a type with no
layout, so a signature that names an imported one has to find it
declared, where an imported class can be resolved provisionally and bound
later. Every module of the program therefore declares its names and binds
its imported enums before any module collects a signature (
loadinsrc/compilation.ts).
-
Every enum declaration is a type of its own. A
-
const enumanddeclare enumare refused.const enumasks for the inlining every member already gets —`const enum` is not supported: an enum member is already folded to its integer, so `const` would ask for nothing(tests/cases/reject_enum_const_enum) — and`declare enum` is not supportedis the ambient form, which has nothing to declare against (tests/cases/reject_enum_declare;declare const enumtoo, sincedeclareis written first). -
An enum does not cross to C or JavaScript.
--emit-header,--emit-dtsand--emit-napihave no spelling for one, so a function whose signature mentions a numeric enum is skipped with the note those generators write for every type they cannot carry (weight(k: Kind): number: not declared; no C spelling for one of its types.), and one whose signature holds a string enum is refused (NL4019, String enums). An enum is a type inside one module, as it is across a module boundary: give a host ani32or astringand keep the enum behind it. -
Enums are top-level only.
enum K { A = 1 }inside a function body isUnsupported statement in Phase 1: EnumDeclaration(CLI only).
String enums
An enum whose members are each given a string literal is a string enum.
It is the spelling Nish takes for a closed set of strings: a union of string
literals, "admin" | "guest", is refused (NL1053, and a lone string literal
type is NL2668), because a value has one fixed layout and a string enum's is
an i32.
enum Role {
Admin = "admin",
Guest = "guest",
Nobody = "",
}
const rank = (r: Role): i32 => {
switch (r) {
case Role.Admin:
return 3;
case Role.Guest:
return 1;
default:
return 0;
}
};
const name = (r: Role): string => String(r); // name(Role.Admin) is "admin"
-
A value is its member's ordinal. A string enum is a distinct type, as a numeric one is, stored as an
i32: the member's position in declaration order,0,1,2, and never the bytes of its string. Every rule of Enums above holds for it as it does for a numeric enum:===and!==between two of its values are the only operators, and they compare ordinals; aswitchover one lowers to LLVM'sswitchon the ordinal; a member is also a type (kind: Role.Admin), and so tags a variant of a discriminated union (tests/cases/enum_string_union_tag); a value is aMaporSetkey, an element, a field and aResultarm; it crosses a thread, a channel and a task as thei32it is; and it is exported and imported as a numeric enum is (tests/cases/enum_string,tests/link/enum_string_export). It is never| null(NL2010), as no enum is. -
Its strings are one table per module. Each module that names the enum's strings holds them as
@.enumstr.<Name>, an array of the module's string constants indexed by ordinal; an importer gets its own copy of the table, so nothing is looked up across a module boundary (tests/link/enum_string_export). Nothing else is emitted:Role.Guestfolds to1asKind.Whilefolds to2.@.enumstr.Role = private unnamed_addr constant [3 x i8*] [i8* bitcast ({ i64, [6 x i8] }* @.str.0 to i8*), i8* bitcast ({ i64, [6 x i8] }* @.str.1 to i8*), i8* bitcast ({ i64, [1 x i8] }* @.str.2 to i8*)], align 8 define internal noundef i32 @rank(i32 noundef %r) #0 { entry: switch i32 %r, label %sw.default [ i32 0, label %sw.case i32 1, label %sw.case.1 ] -
String(r)is the member's string. It loads the table's entry at the ordinal, so it allocates nothing and its answer is the module's constant:%19 = sext i32 %18 to i64 %20 = getelementptr inbounds [3 x i8*], [3 x i8*]* @.enumstr.Role, i64 0, i64 %19 %21 = load i8*, i8** %20, align 8 call void @nish_print(i8* %21)Stringtakes a string enum's value and nothing else; on any other argument it isUnknown function `String`(NL2247,reject_string_call_non_enum). -
A member's value is a string literal, written once. Every member of a string enum has an initializer:
String enum member `Role.Guest` has no initializer: every member of an enum whose members are strings is given its own string literal(NL1064,tests/cases/reject_enum_string_missing). A numeric member beside a string one is NL1043 (reject_enum_string_mixed, and in typed JSONjson_reject_enum_mixed), and a template literal isAn enum member's value cannot be a template literal: write it as a plain string literal, which is fixed when the enum is declared(NL1065,reject_enum_string_template); a computed value,"gu" + "est", isA string enum member's value must be a string literal: a computed value is not supported, because the enum's strings are a table fixed when it is declared(NL1066,reject_enum_string_computed). The value is valid UTF-8, since it is written to and matched against JSON: a lone surrogate,"\uD800", isEnum member `Role.Bad` is not valid UTF-8: a string enum's values are written to and matched against JSON, which is UTF-8(NL2787,reject_enum_string_utf8). The empty string is a value like any other (tests/cases/enum_string). -
Two members do not share a string.
Enum member `Role.Root` repeats the string of `Role.Admin`: each member of a string enum has its own string, because the string is how JSON and `String` name the member(NL2786,tests/cases/reject_enum_string_same_value). A numeric enum allows two members of one value, which are one type; a string enum does not, because its string is how the member is named outside the program: a decoded"admin"must name one member, andStringmust answer one string per member.tscwould also read two members of one string as one value, where Nish's ordinals would make them two. -
A
Roleis not astring, in either direction. A string is never compared with one:Cannot compare a `Role` with a string using `===`: an enum value is not its string, so compare it with a member, such as `Role.Admin`(NL2788,tests/cases/reject_enum_string_compare_literal). ARolewhere a string is wanted,const s: string = ror${r}in a template, isA `Role` is not a string: write `String(r)` for the member's string(NL2789,reject_enum_string_to_string,reject_enum_string_template_use), and a string where aRoleis wanted,const r: Role = "admin", is`"admin"` is a string, and a `Role` is not one: write the member itself, such as `Role.Admin`(NL2790,reject_enum_string_from_string). Arithmetic is refused as on any enum,r + 1beingOperator `+` requires two operands of the same numeric type or two strings, got Role and i32(NL2231,reject_enum_string_arith), andconst enumis NL2285 as above. -
Typed JSON reads and writes its string. A string-enum field, array,
| nullarray orMapvalue is the member's string on the wire, and a union tagged by a string enum's member writes that string as its tag (Typed JSON). -
Under
--emit-tsit is aconstobject.enum Role { Admin = "admin" }is written asexport const Role = { Admin: "admin", Guest: "guest" } as const; export type Role = (typeof Role)[keyof typeof Role];so each member is the string
tscgives it,String(r)is the identity and prints what the native program reads from its table, and a member type istypeof Role.Admin(tests/emit-ts/control/string-enums).tscwould acceptr === "admin",const s: string = randconst r: Role = "admin"on that output; Nish refuses all three, so it is the stricter of the two, and every program it accepts is onetscaccepts too. -
A string enum does not cross to C, N-API or wasm yet. An exported function whose parameter or result is a string enum, or an array,
| nullorResultthat holds one, is refused under--emit-header,--emit-dts,--emit-napiand--emit-napi-async:`rank`'s parameter `r` is `Role`, which holds a string enum: a string enum has no C, N-API or wasm spelling in this version, so take or return a `string` and convert with `String(r)` and a `switch`(NL4019,tests/cases/reject_enum_string_export,reject_enum_string_export_napi). It is an ordinal natively and a string in TypeScript, and a sidecar cannot yet say which a host passes. -
Open questions. A
Role.parse(s)that answers the member a string names is not built: today a program testss === String(Role.Admin)for each member, or decodes the string with typed JSON; the C, N-API and wasm spelling of a string enum is open (NL4019); and a union of string literals stays refused, string enums being the spelling Nish takes instead.
export and import
-
exportis accepted onfunction,class,interface,enum,typeandconstdeclarations (tests/cases/export_fn,tests/link/class_export,tests/link/enum_export_uses,tests/link/alias_export_class,tests/link/const_export). Every other export form is refused by its rule, once:export { a }, with or withoutfrom,export * from,export * as ns fromandexport type { A }are`export { ... }` / `export * from` are not supported; put `export` on the function declaration itself(tests/cases/reject_export_list);export default <value>andexport =are`export default` / `export =` are not supported; use a named `export function`(reject_export_assignment);export default function fis`export default` is not supported; use a named `export function`(reject_export_default), andexport default class Corexport default interface I`export default` is not supported; use a named `export`(reject_cls_export_default,reject_generic_export_default); an anonymous one isFunctions must be namedorClasses must be named.export importisOnly functions can be exported for now, plus classes and interfaces (found `export` on ImportEqualsDeclaration)(orImportDeclaration;reject_export_import_equals), andexport as namespace XisOnly top-level function declarations are supported in Phase 1 (found NamespaceExportDeclaration). Several in one module report one each (reject_module_sweep_together), unless one is a Phase 0 form below, which is then reported alone. -
What
exportpromises depends on the build. In every build it makes a declaration importable by another module of the program. A build whose output something else links or loads —-oalone (IR for a C host to link),--emit-header,--emit-dts,--emit-napi,--emit-napi-async,--profile wasior a wasm--target— also makes an exported function, and every method of an exported class, callable by that host with any arguments, so the compiler assumes nothing about them from the program's own calls (tests/cases/arr_range_call_exported, where a C host'spick(7)panics). A--linkbuild with none of those flags and nodeclare functionis its own final link: the executable holds the program's modules and the runtime and nothing else, so every call to an exported function is one the compiler sees, and it may, for example, prove an exported method's index in range from what every caller passes (tests/link/range_export, AWFY Permute as an exported class, whoseswapcarries no bounds check;tests/link/range_export_unproven, where one caller proves nothing, so the check stays and panics). The.llsuch a build writes beside the executable is that executable's, not an object for another link; build with-oalone, or with a sidecar, to hand a host the program.mainis the one exception: the runtime calls it, and it is never assumed anything about. The rule ishostVisibleinsrc/visibility.ts, anddocs/ARCHITECTURE.md(the call-site ranges row of the attribute soundness rules) is why it is sound. -
An exported function is not named after a C library function. A module of the program's own package has no symbol prefix, so an exported function's symbol is its name, in the entry module and in every other module of the package, and under
--no-strict-exportsevery function's is. Namedwrite,read,free,exit,sinor another function the C library defines for the runtime or for the program's own arithmetic, it would be the definition those references reach in the linked program (console.logprinting nothing, a crash at the first file read,Math.sinanswering what the program'ssinreturns), so it is`write` is the C library's `write`: an exported function of the program's own package keeps its name as its symbol, so the references the C library and the runtime make to that name would reach this function instead; rename it(tests/cases/reject_export_c_symbol_const,reject_export_c_symbol_function,reject_export_c_symbol_libm,reject_export_c_symbol_libm_f32,tests/link/export_c_symbol_sibling,export_c_symbol_sibling_function; with--no-strict-exports,reject_export_c_symbol_no_strictandreject_export_c_symbol_no_strict_exported). A name that begins with_, which C reserves for its library and start-up code (C11 7.1.3), or withnapi_ornode_api_, which Node-API reserves, is refused the same way withbegins with `_`, which C reserves for its library and start-up codeorbegins with `napi_`, which Node-API reserves for its functions(ornode_api_) (reject_export_c_symbol_reserved_prefix, which pins__libc_start_mainand__register_atfork, each of which crashed;reject_export_c_symbol_napi_prefix). The rule is the same in a wasm or an N-API build, where the exported name is the one JavaScript calls: rename the function there too. The names are reserved on every target, whichever target each one binds on: a name only wasi-libc defines (strdup,unlink) is refused in a native build as well, and no flag lifts the rule. The names and where each comes from are insrc/c-symbols.ts: the symbols the runtime leaves undefined on every platform it builds for, the libm calls behind everyMathbuiltin and%on a float indoubleand infloat, with the ones LLVM rewrites them into, thememcpyfamily, the rest of the C allocator, and, because--profile wasilinks wasi-libc statically, every name a wasi-libc member that link pulls in defines or references (an exportedsbrktrapped at start), and every name LLVM's TargetLibraryInfo recognises as a library function on the targets the compiler emits for (abs,atan,rename,qsort: LLVM folds or rewrites a call to one of them whatever its definition, soabs(-3)answered 3;reject_export_c_symbol_tli), and every function C23's<math.h>and C99's<complex.h>declare, indouble,floatandlong double(tgamma,erff,ilogbl,cexp), whether or not the LLVM in use knows it, because a later LLVM than CI's recognises more of them (#651).tests/run.jsreads them back withllvm-nmfrom the runtime's objects,runtime-wasm.c, the N-API shim, and two programs — one that calls everyMathbuiltin, one that multipliesi64s, copies arrays and prints strings — linked at three optimisation levels and compiled for wasm32, the wasi-libc members a traced--profile wasilink of each pulls in, and fails on a name the file does not reserve ("c-symbols: ..."); it reads the TargetLibraryInfo names withllvm-tli-checkerfor five target triples. A function whose symbol is not its ABI is spelled apart instead of refused: aninternalone whose name is reserved takes a$at the end of its symbol (@floor$), because LLVM folds a call to a function it knows by name as the C library's, and a privatefloorcalled asfloor(1.5)answered 1 (tests/link/c_symbol_private_libm), and ahiddenone, which another module's instantiation calls, takes it whatever its name (#610, under function parameters;tests/link/thread_no_strict_hidden_task, under--no-strict-exports). A method, whose symbol carries its class, and a function of another package, whose symbol carries the package, may have any of these names (tests/link/export_c_symbol_allowed).mainis the entry's alone (tests/link/main_in_import), and emitted as@nish_main. -
A wasm module's function is not named
memory.wasm-ldexports the module's linear memory asmemory, andscripts/build.sh --profile wasmlinks with--export-all(--profile wasm-hostwith an export list of the module's exported functions andmemory), so in that module an exported function of that name, or any function under--no-strict-exports, is a second export of it, and Node refuses the module ("Duplicate export name 'memory'"). Thewasicommand module exports onlymemoryand_start, so the function does not collide there; the name is reserved there too, so that a program that builds for wasi also builds as a freestanding wasm module. On a wasm--target, under--profile wasi, and in an--emit-dtscompile, whose loader is for the modulescripts/build.sh --profile wasmorwasm-hostlinks its IR into, it is`memory` is the name a wasm module exports its linear memory under: an exported function of the program's own package keeps its name as its symbol, so the module would export two things named `memory` and no host could load it; rename it(NL3038;tests/cases/reject_export_wasm_memory,reject_export_wasm_memory_wasi,reject_export_wasm_memory_dts,tests/link/export_wasm_memory_sibling; with--no-strict-exports,reject_export_wasm_memory_no_strict; without the flag amemorynobody exported stays internal and compiles, "wasm memory: a --target wasm32 compile keeps amemorynobody exported" intests/run.js). IR compiled for the host with-oalone reaches build.sh without the compiler knowing the build is wasm, sobuild.sh --profile wasm,wasm-hostandwasirefuse IR that defines an exported@memorywith exit 2 ("wasm memory: ..." intests/run.js). A native build has no linear memory to collide with and keeps the name (export_memory_native,tests/link/export_memory_native_sibling). The linker's other names,_start,__heap_base,__data_endand__indirect_function_table, are already refused by the rule above. A wasm program with module storage reserves two more: its loader answers the storage aslayoutandviewsbeside the exports (§5.2 ofdocs/wp50-wasm-host.md), so in such a program an exported function of either name is`views` is the name the wasm loader answers the program's module storage under: an exported function of the program's own package keeps its name as its symbol, so the loader would answer two things named `views` and the page could not reach this function; rename it(NL3051,tests/cases/reject_storage_loader_name). A program without module storage keeps both names (export_loader_names_no_storage). -
The only import form is a named import:
import { square, cube as pow3 } from "./math"(tests/link/two_file). The specifier must start with./or../, name one of the three builtin modules below, name a standard-library module asnish/<module>, or be a package name —hashor@scope/hash, each of which may be followed by a subpath (@scope/hash/blake3), resolved throughnode_modulesand thenishexport condition (tests/link/package_bare; a package that is not installed isCannot find package `hash`,tests/cases/reject_bare_package). Anything else isImport specifier ... must be relative(tests/cases/reject_bare_import)..tsis optional and./x.jsmaps to./x.ts; a relative path resolves relative to the importing file. Every other import form is refused by its rule, once, before anything is loaded: a default import,import d from "./m",import d, { a } from "./m"orimport { default as d } from "./m", isDefault imports are not supported; use `import { d } from "./m"`(reject_default_import); a namespace import isNamespace imports (`import * as ns`) are not supported; import functions by name(reject_namespace_import); a side-effect import isSide-effect imports (`import "./m"`) are not supported; modules have no top-level code(reject_side_effect_import);import type, andtypein front of one name, isType-only imports are not supported(reject_import_type_only); a specifier that is not a string literal isImport specifier must be a string literal(reject_import_template_specifier); andimport x = require("./m")orimport x = A.BisOnly top-level function declarations are supported in Phase 1 (found ImportEqualsDeclaration). Three are refused by Phase 0, because no later release could give them a meaning: import attributes (with { type: "json" },assert { ... },reject_import_attributes), a module export name written as a string (import { "a-b" as b },reject_import_string_name) andimport defer(reject_import_defer) — see Forbidden constructs.type,as,from,defer,requireandassertare not reserved, and are the modifier only where TypeScript reads one:import type from "./m"is a default import of a binding calledtype, and{ type },{ type as t }and{ type as as }each import a function calledtype(tests/parser/names-modules.ts). A missing file isCannot find module `./does_not_exist`(reject_missing_module). -
A file is one module however it is named. The command line and the imports may spell one file differently —
./types.ts, its absolute path, a path through a linked directory, and thetypes.tsanimport from "./types"resolves to — and it is loaded once, under the first spelling that reached it, so its declarations never clash with themselves and its.llis written once (tests/link/root_named_twice,tests/link/identity_symlink_cwd). A module's identity is its real path: the file the operating system opens under a spelling, with every symbolic link resolved, as Node andtsc(preserveSymlinks: false) identify one. A command-line path is taken as the operating system reads it, sofar/../types.tswithfara link intoother/isother/types.ts, a second module beside./types.ts(tests/link/identity_symlink_dotdot); an import specifier is joined to its importer's name, as a relative URL is. What a module is called — its IR header, its diagnostics, its output file — is still the first spelling, so no emitted byte depends on where the links point. -
Two modules never share an output file.
-o <dir>/names each.llafter its module's basename, after its path relative to the entry when two modules share a basename, and — when those still meet, astypes.tsandfar/../types.tsdo — a later module takes<stem>_2.ll,<stem>_3.lland on, in load order (tests/nish/cli.ts). -
The standard library is imported as
nish/<module>(tests/link/std_bare_specifier). It resolves tostd/<module>.tsbeside the running compiler rather than relative to the importing file, so the same specifier works at any depth. A module the library does not have isModule `nish/toml` is not part of the standard library (it has: collections, compress/checksum, compress/deflate, compress/gzip, compress/zlib, crypto/aes, crypto/base64url, crypto/chacha20poly1305, crypto/ct, crypto/ed25519, crypto/hkdf, crypto/hmac, crypto/p256, crypto/pki, crypto/pki-roots, crypto/rsa, crypto/sha256, crypto/sha512, crypto/x25519, crypto/x509, json, map, pair, testing, text, threads)(reject_std_unknown_module), and one that would leave the library — an empty segment, or a segment beginning with a.— is refused rather than resolved:Module `nish/../../escape/lib` climbs out of the standard library; a `nish/` specifier names a module inside it, so no segment may be empty or begin with a `.`(reject_std_escaping_module). The module is named by that specifier, not by where the library is installed:nish/textcarriesstd/text.tsin its; ModuleID, itssource_filename, itsDIFileand — with no-o— the path its.llis written to, whichever directory the compiler sits in and however the compiler was invoked (tests/link/std_bare_specifier). Two installs of one compiler therefore emit the same bytes for the same program, which is the reason the specifier is refused when it could not be a name. Unlike anish:builtin this is ordinary Nish source: it is compiled into the program that imports it and is subject to every rule in this document. What it costs is what you call —speedandsizelink with--gc-sections, and a module imported but never called is byte-for-byte free (std/README.md). Its symbols are scoped to packagenish, so a program may declare a function astd/module also exports (tests/link/std_package_scope); the package is decided by the specifier rather than by where the file sits, because reading it off the path would answer differently for an installed compiler and a checkout of the same version. -
nish/pairisPair<A, B>, two values answered by one call. It isinterface Pair<A, B> { first: A; second: B; }and nothing else: a generic interface, so an object literal takes its type from the annotation and must set both fields, and there is no tuple syntax and no constructor behind it. It crosses a module boundary like any exported interface, over scalars (tests/link/std_pair_scalar), a string and anf64under--number-mode f64(std_pair_f64), an array (std_pair_array), anotherPair(std_pair_nested), and held in a class field and an array (std_pair_held). The instantiations are ordered — aPair<i32, string>returned where aPair<string, i32>is declared isReturn type mismatch: function returns Pair<string, i32> but expression is Pair<i32, string>(reject_std_pair_swapped) — and a literal that leaves a field out isObject literal for `Pair<i32, boolean>` is missing field `second`(reject_std_pair_missing). It lowers as any generic interface does (gen_interface): one struct per instantiation,%struct.Pair$i32$bool, returned by pointer. Use it to return two values, not to store them side by side;docs/wp23-language-surface.md§5 is the reasoning. -
nish/crypto/<primitive>is a module in a subdirectory of the library, one per primitive:sha256,sha512(SHA-512 and SHA-384),hmac,hkdf,ct(constant-time compare),base64url,x25519,ed25519(Ed25519 signatures),chacha20poly1305(ChaCha20-Poly1305),aes(AES-128 and AES-256 with GCM),p256(P-256 ECDSA),p384(P-384 ECDSA verification),rsa(RSA signature verification),x509(DER, PEM and self-signed P-256 certificates),pki-roots(the trust anchors fromSSL_CERT_FILE,SSL_CERT_DIRor the system bundle; native only) andpki(RFC 5280 path validation and RFC 6125 hostname matching of a TLS server's chain) (tests/link/crypto_base64urlis the case that resolves a nested specifier). Each is ordinary Nish source under the rules above, and a byte string is au8[]. They are written branch-free on secret data, and WP34 N6's disassembly check (tests/ct-asm.js) verifies that of the functions itsct_asm_*fixtures hold, on x86-64 and aarch64; the rest is discipline.std/README.mdlists what each module exports, the specification it reproduces, which of its functions are verified, and the rules they share. -
A package is imported by name, and what is resolved is its source.
import { scale } from "pkg_bare"looks fornode_modules/pkg_barein the importing file's directory and in every directory above it — above the directory the compiler was run in included, which is the ordinary npm layout (tests/link/package_above) — stepping over an ancestor already callednode_moduleswherever the module's own name spells one, as Node does (tests/link/package_doubled) — reads that package'spackage.json, and compiles the file itsexportsmap offers for thenishcondition (tests/link/package_bare,docs/wp21-packages.md§2):{ "exports": { ".": { "nish": "./src/index.ts" }, "./util": { "nish": "./src/util.ts" } } }Source, not a built artifact, and that is the decision rather than a stage: the whole-program fact fixpoint, monomorphisation, and
--number-modebeing a compile-time ABI each make a prebuilt library strictly worse than the source that is already there (docs/wp21-packages.md§3). The consequence a reader should carry away is that a dependency is compiled, checked and optimised as if it were your own code, and its non-exported functions areinternaland dead-stripped like your own.-
The number mode rides in the condition.
nish-i32andnish-f64are the mode-qualified spellings, and the compiler asks for its own mode first and then plainnish— so a package correct under either declaresnish, a package that needs different source per mode declares both, and a mismatch is a resolution failure at the package boundary rather than a wrong answer at run time. "First" is the compiler's order and not the manifest's: the mode-qualified condition wins wherever the package declares it, unlike Node, which would take whichever of the two the manifest names first (tests/link/package_mode_order,tests/link/package_mode_order_f64). Node's order is right for rival conditions and wrong for these two, becausenish-f64is not a rival ofnishbutnishrefined by the mode: honouring the order would let a package writenishabovenish-f64and never have its f64 source compiled, with nothing said about it — the silent mismatch this spelling exists to make loud. -
Presence of the condition is the claim, and every failure to find it names its cause (
docs/wp21-packages.md§6, §11). Each is refused at the import, before a byte of the package is checked, with its own--jsoncode:-
A package whose entry offers no Nish condition in any spelling — an
ordinary npm package, with
importandrequirerows or a bare".": "./index.js"— isPackage `plainjs` has no Nish entry point: its `exports` gave this compiler no file to compile for `.`, because that entry declares none of the conditions this compiler compiles source from (`nish`, `nish-i32`, `nish-f64`)(NL3020,tests/link/package_not_nish,tests/link/package_no_condition). -
A package that offers only the other mode's condition is
Package `pkg_f64` supports Nish in f64 mode only: its `exports` offers `nish-f64` for `.` and neither `nish-i32` nor `nish`, and this program is compiling in i32 (`--number-mode i32`)(NL3017,tests/link/package_other_mode, and the reverse intests/link/package_other_mode_f64). -
A
package.jsonthat is not well-formed JSON, when no entry point could be read out of it, names the manifest and the line and column where it breaks:Package `pkg_broken` has a malformed manifest: …/package.json:4:3 is not well-formed JSON(NL3021,tests/link/package_malformed). One the reader got a file out of before the break compiles, as it always has. -
Anything else — no
exportsat all, no key for the subpath asked for, or a value the reader does not follow — isPackage `pkg_one` has no Nish entry point: its `exports` gave this compiler no file to compile for `./extra`(NL3014,tests/link/package_no_subpath). The sentence reports what the compiler came away with rather than what the manifest declares, because anish-f64that names something other than a file outranks anishthat is there and correct, and is what the message is about.
A package that is not installed at all is
Cannot find package `@x/y`(tests/cases/reject_bare_package). -
A package whose entry offers no Nish condition in any spelling — an
ordinary npm package, with
-
A compiler floor goes in
engines.nish, npm's slot for it, and it is checked whether or not the entry point resolves. The one range read is a floor,>=X.Y.Zor>=X.Y, blanks allowed around the version; a package whose floor is above this compiler isPackage `pkg_future` needs a newer compiler: its `engines.nish` asks for `>=999.0.0` and this is nish 0.8.0(NL3018,tests/link/package_engines_floor), and one met compiles (tests/link/package_engines_met). Any other shape — a caret, a tilde, an upper bound, a bare version, a value that is not a string — is refused rather than taken as met, because a floor the compiler cannot read is one it cannot claim to meet:declares `engines.nish` as `^0.1.0`, which is not a range this compiler reads(NL3019,tests/link/package_engines_range). -
A package is its real directory, and a program compiles one copy of it. One package reached both directly and through a symlink — pnpm's layout, and npm's whenever it cannot hoist — is one package, as it is to Node's resolver, and its own dependencies are looked for beside where it really is (
tests/link/package_symlink). One package name at two real directories — npm's duplicate copies, or two versions of it under two dependencies — is refused at the import that reached the second, naming both directories and both versions:Package `hash` is at two places, …/node_modules/hash (1.0.0) and …/node_modules/user/node_modules/hash (2.0.0): Nish compiles one copy of a package per program, so every import of it has to reach the same directory(NL3029,tests/link/package_two_dirs,docs/wp21-packages.md§7; a copy whose manifest declares noversionis said to haveno version,tests/link/package_two_dirs_unversioned). A directory no link leads through keeps the spelling the search found it by. One a link leads through is named by its real directory, spelled relative to the working directory when the program was named relatively, and every module under it is in that package — so a workspace link to a directory outside anynode_modules(node_modules/foo -> ../packages/foo) keepsfoo's own relative imports infoo(tests/link/package_workspace). -
Only the
nishconditions are honoured, and a subpath is an exact key:default,importandnodeare skipped rather than matched, and the"./*"pattern form is not read. A target is a string beginning with./and naming a file inside the package. -
A package's symbols carry its prefix (
@pkg_bare.scale), so two packages may each keep a privatehelper(); the root package — the program being compiled — has no prefix and its IR is unchanged (docs/wp21-packages.md§9). -
A diagnostic in a dependency says it is the dependency's. An error or warning spanned in a package's own source keeps its code and wording, adds
= note: this is in the dependency `pkg_broken` 0.3.1, not in this program's own sourceunder the excerpt (the version as its manifest gives it, left out when it gives none), and under--jsoncarries"package":"pkg_broken"(tests/link/package_dependency_error). What decides is where the module was resolved, so a dependency that calls itselfnishis still one, and the standard library never is. -
A builtin with no runtime on the target is refused by what the program reaches, once every module is checked. A freestanding
--target wasm32build has no file, clock, environment, process, signal, terminal or entropy call (runtime/runtime-wasm.c), so a program whosemainor any exported function of its own modules reaches one —readFileSync,getenv,spawnSync,monotonicNanosamong the ones no call-site rule refuses — is refused with the N3 rule's words (NL2404), one error per capability, spanned at the first call of the witness chain, which names every hop; when the chain passes through a dependency the message names the one nearest the builtin:`getenv` reaches the operating system, and a wasm32 build has none to reach; the dependency `pkg_conf` calls it in `load`(tests/link/package_wasm_hostless,tests/link/wasm_hostless_own,reject_read_file_wasm). A module that calls one in a function the program never reaches is not refused: a wasm32 program may importnish/cliforStyle(npm test"wasm hostless").--target wasm32-wasiand--profile wasihave the calls, and refuse none of them.
-
-
An imported constant contributes no
declareand no relocation: it is folded into every use site in the importing module exactly as it is in the exporting one (tests/link/const_export,tests/differential/corpus/const_modules). -
Functions may be renamed on import (
as); classes and interfaces may not, because the type name is part of the ABI (%struct.Point,@Point.constructor):Classes cannot be renamed on import(tests/link/class_import_rename). -
Importing a name the module declares but does not export, or does not declare at all, is an error with distinct messages (
tests/link/not_exported,tests/link/unknown_export); importing the same local name twice, or a name also declared locally, is an error (tests/link/duplicate_import). -
An imported class or interface brings the layouts of the classes and interfaces its own members mention — field types, and the parameter and return types of its methods and constructor, through arrays and nullables — so a module that imports
Registrymay hold, call and read theEntryvaluesRegistry.all(): Entry[]hands it without importingEntry(tests/link/reachable_struct). Only the layout travels, not the name: a type annotation still needs the import, andconst e: Entryin that module isUnsupported type reference `Entry`(tests/link/reachable_struct_annotation). An imported function brings its own signature's layouts the same way —import { lex }wherelex(): Tokengives this moduleTokenvalues it never names (tests/link/reachable_struct_return) — and the layouts travel however many modules apart the declaration is (tests/link/reachable_struct_chain). -
Import cycles are allowed (
tests/link/cycle); a shared dependency is compiled once (tests/link/diamond). -
Linkage. An
exported function is an external C-ABI symbol; every other function getsinternallinkage by default, so LLVM may inline, specialise or drop it (tests/cases/export_fn,tests/link/strict).--no-strict-exportsmakes every function external again, which is what a C driver calling a non-exported function needs (tests/cases/export_no_strict).-
A method another module calls is in the link. The layouts an import
brings (above) include a class its module does not export —
Conn.space(): Spacehands an importer ofConnaSpacewhether or notSpaceis exported — and the importer may call that value's methods, generic ones included. Such a method (or instantiation of one) ishiddenrather thaninternal: one the final link resolves and the output exports to nothing, with theResultABI an exported one has. A method only its own module calls staysinternal(tests/link/package_hidden_method, #442). The class's name stays its module's: importing it is`Space` is declared in `pkg_q` but not exported (add `export`)(tests/link/package_hidden_class_import), and naming it in an annotation isUnsupported type reference `Space`(tests/link/package_hidden_class_annotation).
-
A method another module calls is in the link. The layouts an import
brings (above) include a class its module does not export —
-
A function name is unique across the whole program, exported or not, in either mode (
tests/link/duplicate_export,duplicate_internal).internallinkage keeps a name out of the linker's way but it does not buy a second namespace: the whole-program attribute analysis is keyed by symbol name, so two functions sharing one would be emitted with each other's attributes — a miscompile rather than a link error, which is why the rule does not depend on the flag. A proposal to qualify a private function's symbol by its module when it would clash is WP57; the rule above holds until it ships.
main
export const main = (): number => { // or `: void`; `: i32` in f64 mode
console.log("hello");
return 0; // the process exit code
};
-
Only the entry module (the first file on the command line) may declare
export const main(Only the entry module may declare,tests/link/main_in_import);--linkrequires it (tests/link/no_main), unless the entry holds top-level code, which becomes it. -
It takes no parameters (
`main` cannot take parameters,tests/cases/reject_main_params; the command line isprocess.argv) and returnsvoid(exit code 0,tests/cases/entry_main_void) or ani32-lowerednumber(tests/cases/entry_main,tests/link/two_file). In f64 mode declaremain(): i32;main(): numberis rejected with`main` must return void or an i32 number ... under --number-mode f64 declare `main(): i32`(CLI only). -
The user's function is emitted as
@nish_main; the compiler adds a C@main(i32 %argc, i8** %argv)wrapper that buildsprocess.argvwhen the program reads it (call void @nish_argv_init(i32 %argc, i8** %argv),tests/cases/argv_echo,tests/link/argv_import), calls it, frees the arena, and returns the code. A non-exportedfunction mainis an ordinary function named@main(for C drivers such astests/driver.c).
Classes
class Point {
x: number;
y: number;
label: string = "p"; // literal initializer
constructor(x: number, y: number) {
this.x = x;
this.y = y;
}
manhattan(): number {
return this.x + this.y;
}
}
-
Fields need a type annotation (
Field `x` of class `Point` needs a type annotation,tests/cases/reject_cls_field_no_type). An initializer must be a literal (a number, a negated number, a string, or a boolean) of the field's type (initializers must be literals(CLI only);tests/cases/cls_initializers). Layout is declaration order with natural alignment and padding, exactly as clang lays out the same C struct (tests/layout/structs.ts). -
Constructor: at most one, with a body
(
Constructor of class `Point` must have a body (no overload signatures),tests/cases/reject_cls_ctor_no_body), no return type annotation (Constructors cannot declare a return type,reject_cls_ctor_return_type), and no parameter properties (Parameter properties (`constructor(public x: number)`) are not supported; declare the field and assign it,reject_cls_param_property;public,private,protected,readonlyandoverrideare a parameter's modifier only before its name on their line, soconstructor(public: i32)is a parameter calledpublic,tests/parser/names-classes.ts).new C(args)takes exactly the constructor's arity (`new Point` expects 2 argument(s), got 1,tests/cases/reject_cls_ctor_arity). A class without a constructor may benewed with zero arguments and gets its initializers stored inline (tests/cases/cls_initializers). -
Definite assignment (
tests/cases/reject_cls_uninitialised,reject_cls_uninitialised_no_ctor,reject_cls_read_before_assign): after the constructor returns every field holds a value. The check is syntactic, top to bottom:this.f = e;statements (and chains) markf; anifmarks what both arms mark; assignments inside loops, nested expressions, ternaries or calls do not count; everyreturnmust see every field assigned; readingthis.f, callingthis.m(), or usingthisas a value before every field is assigned is an error. A class with no constructor must initialise every field (has no initializer and no constructor assigns it). -
Methods need a body (
Method `scale` of class `Point` must have a body,tests/cases/reject_cls_method_no_body: a signature without one is an overload, and there are none) and an explicit return type; they are ordinary functions@Class.methodwiththisfirst.thisis valid only inside a method or constructor (`this` is only valid inside a method or constructor,tests/cases/reject_cls_this_outside) and may be aliased or passed on (const self = this(CLI only);tests/cases/cls_this_method_call). Methods may call each other and free functions, recursively. -
Field access:
p.xreads,p.x = vandp.x op= vwrite, for the arithmetic and the bitwise compound operators alike (tests/cases/cls_field_write,cls_compound_field,cls_field_bitwise_assign);++/--on fields is not supported (Only simple variables can be assigned). Unknown fields:Unknown field `z` on class `Point`(tests/cases/reject_cls_unknown_field,reject_cls_assign_unknown_field). -
readonlyfields may be assigned only asthis.f = vin their own class's constructor (tests/cases/cls_readonly_ok;Cannot assign to readonly field `value` of `Id` outside its constructor,reject_cls_readonly_write). A compound assignment is a write like any other, sothis.f |= bitis refused too, in the declaring class's own constructor as much as anywhere else (reject_cls_field_bitwise_readonly).public,private,protectedare accepted and ignored. -
Equality:
===/!==on two values of the same class compare identity (pointer equality) (tests/cases/cls_this_method_call,Account.same);<and friends are rejected (tests/cases/reject_cls_ordering). Both operands must have the same declared type. -
No inheritance (WP25).
extendson a class is`extends` is not supported: Nish has no inheritance(tests/cases/reject_cls_extends), andsuperin every spelling —super(...),super.m(), a baresuper— is`super` is not supported: Nish has no inheritance(reject_cls_super). A class is never a subtype of another class, so every method call in a program names exactly one symbol and there is nothing for a vtable to decide. The wideningextendsused to provide isimplements, below: repeat the fields you were going to inherit as the class's first fields and name an interface that declares them. Shared behaviour is a free function over that interface. -
Rejected:
staticon a field, a method or the constructor (tests/cases/reject_cls_static,reject_cls_method_static,reject_cls_ctor_static) — the word is still a legal member name, andstatic?: i32is an optional field calledstatic(reject_cls_field_named_static), and on an interface field too, where the message says which kind it is (reject_cls_static_interface), with one exception: a class's typed JSON options,static readonly json = { ... }, which is not a field and is written without a type (Typed JSON) —readonlyon a method or the constructor, which has no storage to be read-only (reject_cls_method_readonly,reject_cls_ctor_readonly), getters/setters (Getters and setters are not supported in class `Box` (use a method),reject_cls_accessor;getandsetare still member names,tests/parser/names-bindings.ts), optional fields (cannot be optional), index signatures (Index signatures are not supported in class `Point` (object layout is fixed),reject_cls_index_signature),!assertions,abstracton the class or a member (Abstract classes are not supported,reject_generic_abstract), adeclare classthat is not a C struct layout (a method, accessor, constructor or initializer in one is refused as thedeclare classsection lists,reject_cls_declare,reject_ffi_struct_*), a class with no name —export default class { }or a class expression — (Classes must be named,reject_cls_anonymous; a named class expression isUnsupported expression in Phase 1: ClassExpression), a member named by a string or a number (Members of `Point` must have plain identifier names,reject_cls_field_string_name), astatic { }block (Unsupported class member in `Point`: ClassStaticBlockDeclaration,reject_cls_static_block), type parameters on the constructor (reject_generic_method_constructor; a method may declare its own — see Generic methods), decorators (reject_decorator), computed member names (Computed property names are forbidden in Nish (member names are fixed at compile time),reject_cls_computed_field,reject_cls_computed_method;[Symbol.dispose]is the one bracketed name read, see Scoped tasks), overloaded constructors.
Generic functions
A function may declare type parameters, and every instantiation is compiled
to its own specialised function: one define per (template, type-argument
tuple), named by appending $ and each argument's mangled type to the
template's name. There is no boxing, no type dictionary and no runtime type
information, and the IR of an instantiation is the IR of the monomorphic
function somebody would have written by hand, with the symbol renamed
(tests/cases/gen_identity).
const identity = <T>(x: T): T => x;
export const main = (): i32 => {
console.log(identity(7)); // @identity$i32
console.log(identity("hi")); // @identity$str
return 0;
};
-
Both spellings take type parameters.
function identity<T>(x: T): Tandconst identity = <T>(x: T): T => xdeclare the same thing, as they do without them (tests/cases/gen_identity). -
A type argument is inferred from the arguments, and is never written at a
call site.
identity<i32>(7)isType arguments are not written at a call site in Nish: `T` is inferred from the arguments, so write `identity(...)`(tests/cases/reject_generic_call_type_args). One token of lookahead cannot tellf<i32>(x)from(f < i32) > (x), and inference already covers the spelling. Inference matches the shape of each declared parameter against its argument's type —Tagainsti32,T[]againsti32[],Result<T, string>againstResult<i32, string>— left to right, first binding wins (tests/cases/gen_infer_two,gen_result_payload). -
A type parameter that appears in no parameter is an error at the
declaration, because it could never be inferred and the function could
never be called:
Cannot infer `T` for `empty`: a type parameter is inferred from the arguments, and `T` appears in none of them; give `empty` a parameter that mentions `T`(tests/cases/reject_generic_function). -
Every other type rule applies to a type argument unchanged. A type
parameter is not a hole through which something the language refuses can be
smuggled in:
Result<T, void>is still refused at an instantiation whoseEisvoid,T | nullis still only over a pointer type, and each instantiation's body is checked withTbound to a concrete type. -
Each instantiation gets its own facts. The attribute fixpoint and the
escape analysis are keyed by symbol, so
eq<T>isreadnoneatT = i32andreadonlyatT = string— one source line, two sets of attributes (tests/cases/gen_eq_purity). -
Recursion terminates or is refused by name. A recursive call whose type
argument is the parameter itself, or a type that does not mention it, is
accepted:
countDown<string>asking forcountDown<i32>is two instantiations and no more (tests/cases/gen_recursive_ground). A call that puts the parameter under a type constructor has no end and is refused:Monomorphising `grow` would not terminate: `grow<i32>` asks for `grow<i32[]>`, which puts `i32` under a type constructor instead of passing it on, so the chain has no end; pass `T` itself, or a type that does not mention it(tests/cases/reject_generic_polymorphic_recursion). Behind the rule are two compiler limits — 256 instantiations of one template, 4096 in one module — whose message says it is a limit rather than a rule of the language. -
$may not appear in a declared name. It separates a generic's name from its type arguments in every symbol the compiler emits, so a function calledidentity$i32would be the symbolidentity<i32>already has:`identity$i32` cannot be the name of a function in Nish: `$` separates a generic's name from its type arguments in the symbols the compiler emits(tests/cases/reject_generic_dollar_name). A class or interface, generic or not, is held to it too: aclass Box$i32would be the structBox<i32>already is, and one type with it wherever either is declared (#193;tests/link/class_dollar_name,…_imported). Locals, parameters and fields are unaffected — only names that become symbols are restricted. -
An instantiation is spelled as written wherever a person reads it, and
mangled wherever a machine does. Every diagnostic, the
--emit-checkeddump and the-gDWARFnamesayidentity<i32>,identity<Box<i32>>,Box<i32>andBox<i32>.get; the symbol, the%structname, the DWARFlinkageNameand a header'sNISH_SYMBOLbinding keepidentity$i32,identity$$Box$i32andBox$i32. Every instantiation'sDISubprogramcarries the template'sline, so a breakpoint in a generic body has one address per instantiation, as a C++ template's does (tests/cases/dbg_generic). A polymorphic-recursion chain through a class reads`grow<i32>` asks for `grow<Box<i32>>`(tests/cases/reject_generic_recursion_through_class). -
An exported instantiation reaches a host as
nish_gen_<name>. Its symbol holds a$, and a.when an argument is an array, a nullable or aResult(identity$arr.i32), so neither C nor JavaScript can spell it.--emit-headerdeclaresnish_gen_identity_i32andnish_gen_identity_arr_i32, bound to the symbols withNISH_SYMBOL, and--emit-dts, the wasm loader and--emit-napiexport the same names; the comment above each says which instantiation it is. An instantiated class isstruct nish_gen_Box_i32with methods such asnish_gen_Box_i32_get. A user name cannot start withnish_, so no declaration of the program can take one of these names — a useridentity_i32besideidentity<i32>is two C functions (tests/cases/interop_generic_fn,interop_generic_collision). The sidecars declare instantiations only:identity<T>itself has no symbol, so nothing declares it. -
An exported generic that nothing instantiates is refused when a sidecar is
asked for. It has no symbol, so a header, a
.d.tsor an N-API shim would leave it out without a word. With any of--emit-header,--emit-dts,--emit-napior--emit-napi-asyncit is`identity` is generic, so it has no single C signature: a generic function becomes a symbol only where it is instantiated, and this program instantiates none(NL4007), and for a class`Box` is generic, so it has no single C layout: a generic class becomes a struct only where it is instantiated, and this program instantiates none(tests/cases/reject_interop_generic_unused); without a sidecar flag the same program compiles. The same flags refuse two functions whose C names collapse to one, which only a method's.-joined name can still cause —Point.shiftedbeside a functionPoint_shifted:`Point.shifted` and `Point_shifted` are both `Point_shifted` in C, so a header that declared both would not compile: rename one of them(NL4008,tests/cases/reject_interop_generic_clash). They also refuse a signature or an exported struct field that holds a discriminated union, which has no C spelling yet (NL4009,tests/cases/reject_union_export_param), and a signature that holds a string enum, alone or in an array, a| nullor aResult:`rank`'s parameter `r` is `Role`, which holds a string enum: a string enum has no C, N-API or wasm spelling in this version, so take or return a `string` and convert with `String(r)` and a `switch`(NL4019,tests/cases/reject_enum_string_export,reject_enum_string_export_napi). -
A template may be exported and instantiated from another module, and each
instantiation is defined exactly once, in the module that declares the
template. Everywhere else it is a
declarecarrying that definition's attributes, which is what an imported plain function gets — the difference is that one imported name becomes one symbol per distinct type-argument tuple rather than one symbol (tests/link/generic_import). Two modules asking for the same tuple share the one definition (tests/link/generic_two_importers), and the symbol carries the declaring package's prefix, so a package that exportspick<T>owns@pkg_lib.pick$i32however many packages call it (tests/link/package_generic_import). A type argument may be a class the declaring module has never heard of: the layout travels with the request, and the instantiation is emitted where the template is (identity<Point>intests/link/generic_import). A template that is not exported cannot be imported, in the words every other private declaration is refused in (tests/link/generic_import_private), and an imported generic may be renamed —import { Box as B }— without moving the symbol, because the instantiation is named after the template rather than after whatever the caller called it. -
A type-argument list belongs to a template and to nothing else. An
imported name that is not one is refused when the import binds, rather than
where the annotation is written, because a module's signatures are resolved
as soon as it is parsed and before any import is bound:
`Point` in `./lib` takes no type arguments: only a generic class or interface is written with them(tests/link/generic_import_plain). -
A type parameter has no members until a constraint names them. A
template may pass a
Ton, return it, store it, compare it and hand it to an operator that works at the concrete type, but it may not read, write or call a member through one, even when every instantiation it has would have that member:Cannot read `x` of `T`: an unconstrained type parameter has no members; say which ones it has with `<T extends Point>`(tests/cases/reject_generic_member_unconstrained), and the call formCannot call `bump` of `T`: ...(reject_generic_member_unconstrained_method). A value isT-typed however it reached the receiver: a local inferred from aT, an element of aT[], aT | nullnarrowed toT, a field of typeTread throughthis, and the result of a generic call that returns its argument (reject_generic_member_through_local); a field ofnew Box<T>(t), an element of[t](reject_generic_member_through_expression); and aResult<T, E>'svalue,pop()on aT[], an assignment's value andthiscopied into a local (reject_generic_member_through_payload). What decides it is where the value came from, never its type — atT = PointaTand aPointare one type, and only theTis refused (reject_generic_member_same_type_local). -
<T extends Shape>constrains a type parameter, and then aThas exactlyShape's members: its fields, and its methods whenShapeis a class. Each instantiation still reads its own struct directly —TatCirclereads field 0 of%struct.Circleand calls@Circle.bump, with nobitcastand no vtable (tests/cases/gen_constraint,gen_constraint_method). A field may be written through aTon the same terms (gen_constraint_write), and each parameter answers from its own constraint, so<T extends Shape, U extends Counter>readsareathrough aTand callsbumpthrough aU(gen_constraint_two). A member the concrete type has and the constraint does not is refused, once for the template however many instantiations it has, or the constraint would be decoration:Unknown field `radius` on interface `Shape`, the constraint of `T`: a constrained type parameter has only the members its constraint declares(tests/cases/reject_generic_member_not_in_constraint). -
A constraint is a declared class or interface, or an instantiation of a
generic one written with concrete type arguments
(
T extends Container<i32>,tests/cases/gen_constraint_generic_bound). It is resolved once, in the template's own module, so a template imported from another module carries its constraint with it (tests/link/generic_constraint_import). Anything else is refused at the constraint — a scalar,string, an array or aResult:`T extends i32` is not supported: a constraint must be a declared class or interface, because the members a type parameter has are its constraint's— and so is one that mentions a type parameter, which would mean something different at every instantiation:`U extends T` is not supported: a constraint cannot mention a type parameter, ...(tests/cases/reject_generic_constraint). -
A type argument must satisfy its constraint: it is the constraint
itself, or a class that
implementsit. Having fields of the same names is not enough, because satisfying a constraint is a declaration, and a class constraint is satisfied by that class alone because a class can onlyimplementsan interface. It is checked at each request, where the argument was inferred or written, and reported in that module:`T` of `areaOf` requires `T extends Shape`, and `i32` does not implement it; pass a class or interface that declares `implements Shape`(tests/cases/reject_generic_unsatisfied_constraint,tests/link/generic_constraint_import_unsatisfied). The constraint is a declaration, not a name: a module that declares aShapeof its own has not satisfied another module's<T extends Shape>, whatever fields the two share — neither with thatShapeitself nor with a class that implements it — and a same-named class does not satisfy a class constraint either. The request is refused in those words at the call in the asking module (tests/link/generic_constraint_same_name,…_same_name_fields,…_same_name_itself,…_same_name_class). The constraint may still be the asking module's own declaration when the template's module imports that very declaration (tests/link/generic_constraint_import_back). Two modules can no longer declare aShapeeach at all (A class or interface name is program-wide, under Generic classes and interfaces), so the four programs above are refused at the second declaration now, before any call is checked; this rule is still what decides a constraint. Every position a type argument can be written in is a request of its own — a field, an array element, aResultarm, inside another instantiation, animplementsclause (reject_generic_unsatisfied_constraint_positions). -
A type parameter list names each parameter once, on a function, an
arrow, a method, a class and an interface alike:
Duplicate type parameter `T`(NL2302). The secondTcould never be bound to anything the first was not, so the list is refused on the first name that repeats, and only there:<T, T, T>is one diagnostic, on the secondT, because a declaration reports one. The rule needs no type, so a template nothing instantiates is refused too (tests/cases/reject_generic_duplicate_type_parameter,…_arrow,…_method,…_class,…_interface,…_template,…_triple). A type alias's list is refused before it is read, by NL1054. -
Not supported yet: a default type argument (
<T = string>,a default type argument (`T = ...`) is not supported: a type argument is inferred from the arguments, NL2292, refused on a template nothing instantiates too,tests/cases/reject_decl_sweep_together) and type parameters on a constructor, each with its own message. Type parameters on a type alias are forbidden rather than unsupported (NL1054, Type aliases). A genericmainis refused too, in words of its own: the entry point is called by the C runtime, which has no type arguments to give it. Multiple bounds (T extends A & B), a bound that mentions the parameter it constrains, and a bound that is "any type with a methodcompare" are not planned on this path. The design and the milestones are in wp18-generics.md.
Generic classes and interfaces
A class or an interface may declare type parameters too, and an
instantiation is an ordinary struct: Box<i32> is %struct.Box$i32 with
Box's fields at T = i32, its methods are @Box$i32.get and
@Box$i32.constructor, and everything the language already does to a struct —
the layout, implements, new, field access, the C header, the debug info —
works on it unchanged (tests/cases/gen_box_i32, gen_box_string).
class Box<T> {
value: T;
constructor(v: T) { this.value = v; }
get(): T { return this.value; }
}
interface Pair<A, B> { first: A; second: B; }
export const main = (): i32 => {
const b = new Box<i32>(7); // %struct.Box$i32
const p: Pair<i32, string> = { first: 1, second: "" }; // %struct.Pair$i32$str
return b.get();
};
-
The type arguments are written out, never inferred.
Boxon its own is not a type, because there is no layout untilTis bound:`Holder` is generic: it must be written with its type arguments, e.g. `Holder<number>`(tests/cases/reject_generic_class). The two positions that take them are an annotation (const b: Box<i32>) andnew(new Box<i32>(7)) — the two a one-token parser can read a type-argument list in, which is the same reason a call infers instead (§2a of wp18-generics.md). A class that is not generic takes none:`Point` is not generic, so `new Point` takes no type arguments(tests/cases/reject_cls_new_generic). -
A type argument may be anything a type may be, including another
instantiation:
Box<Point>is%struct.Box$$PointandBox<Box<i32>>is%struct.Box$$Box$i32, the>>split back into two by the parser (tests/cases/gen_box_struct,gen_nested). A type parameter takes none, though:Tis whatever the instantiation bound it to, soconst y: T<i32>isUnsupported type reference `T<i32>`(tests/cases/reject_generic_type_param_args). -
A template obeys every rule a declared class obeys. Type parameters
change what a declaration means, not what it may be written with, so
export default class Box<T>,abstract class Box<T>andinterface Box<T> extends Baseare refused in the same words as their non-generic spellings, anddeclare class Box<T>isA generic `declare class` is not supported(a C struct has one layout;tests/cases/reject_generic_declare_class,reject_generic_export_default,reject_generic_abstract,reject_generic_interface_extends). -
A template claims its name in the one declaration namespace.
class Box<T>is a declaration ofBox, so a function, a constant, an alias, an enum or a second class of that name is refused whichever was written first — the structs pass runs before the functions pass, so the template is always the one already there (tests/cases/reject_generic_class_fn_clash,reject_generic_class_after_fn,reject_generic_class_redeclared). -
A class or interface name is program-wide, because a struct type is
identified by its name alone:
%struct.Base, and a type id interned by that name. So two modules may not both declare a class or interface of one name, whether or not either is exported, whether it has fields only, and however different the two field lists are. The later declaration in load order is refused, once per name per module, naming the module that declared it first:Class `Base` is also declared in main.ts; a class or interface name must be unique across the program whether or not it is exported, because a struct type is identified by its name alone(tests/link/class_clash_fields,tests/link/iface_clash_private; NL3028). Left alone, a value of one was accepted where the other was expected and its fields were read at the other's offsets, with no diagnostic (#193). Two classes that share a name and a constructor or method are refused in the same words, once (tests/link/class_clash_constructor,…_method,…_package), and across two packages the wording is the onetests/link/two_packages_structpins. A declaration is compared with the first of its name in its own package before any other, so when a third package declared the name first, the first of the two is refused across packages and the second in these words (tests/link/class_clash_after_package,…_package_after_root). Importing the one declaration is how two modules share a struct (tests/link/iface_same_name_imported). -
An instantiation's name is program-wide, because
%struct.Box$i32and@Box$i32.constructorare. So two modules may not both declare a generic class or interface of one name once both instantiate it:Generic class `Holder` is also declared in helper.ts; a class or interface name must be unique across the program, and an instantiation is named after its template(tests/link/generic_class_clash), once per template even when signatures have made both instantiations, constructor and methods included, before any body is checked (tests/link/generic_class_clash_signature), and across two packages the wording is the onetests/link/package_generic_class_clashpins. As for a declared name, a template is compared with the first of its instantiation's name in its own package before any other, so when a third package instantiated the name first, the second module of the package is refused in these words (tests/link/generic_class_clash_after_package). Nothing about the two declarations is visible at a call, so left unreported the second layout would simply replace the first and a field read would land on the wrong offset. -
implementsmay name an instantiation.class Box<T> implements Container<T>is checked once per instantiation, againstContainer$i32, by the ordinary field-prefix rule — the interface's fields are the class's first fields, so the conversion is onebitcast(tests/cases/gen_implements). -
Each instantiation gets its own facts, exactly as a generic function's
do: one
new Box<T>(v)in a body that does not escape is an entry-blockallocaatT = i32and an arena bump atT = stringwhen the box is returned — one source line, two memory behaviours (tests/cases/gen_stack). -
It is named as written, and linked as mangled. A diagnostic says
Unknown field `nope` on class `Box<i32>`andArgument 1 of `Box<i32>.set`: expected i32, got string(tests/cases/reject_generic_member_display,reject_generic_implements_display,reject_generic_ctor_display), and-gnames the composite typeBox<i32>and its methodsBox<i32>.get, withBox$i32.getas thelinkageName(tests/cases/dbg_generic). In a header it isstruct nish_gen_Box_i32; the rule for both is the generic function's (Generic functions). -
A class or interface may constrain its parameters with the rules a
generic function's follow. Its own methods read the constraint's members
through a field of that type —
this.item.areainclass Holder<T extends Shape>is two loads,itemand thenareaof%struct.Circle(tests/cases/gen_constraint_class) — and every instantiation is checked against the constraint where its type arguments are written, afternewand in an annotation (tests/cases/reject_generic_unsatisfied_constraint_class,reject_generic_unsatisfied_constraint_annotation). -
A field may not grow its own type argument.
class Nest<T> { inner: Nest<T[]> | null }asks for a strictly larger instantiation every time it is laid out:Monomorphising `Nest` would not terminate: `Nest<i32>` names `Nest<i32[]>`, which puts `i32` under a type constructor instead of passing it on, so the chain has no end; name `T` itself, or a type that does not mention it(tests/cases/reject_generic_expanding_field).Nest<T>and a type that mentions no parameter are both fine — it is the same rule a generic function's recursive call is held to, stated over fields. -
A generic class or interface may be imported, and the whole-program rule
is the one a generic function's is:
%struct.Box$i32and every@Box$i32.*symbol are defined once, by the module that declaresBox<T>, anddeclared by every module that holds one (tests/link/generic_import,tests/link/package_generic_import). Unlike a declared class it may be renamed on import, because the name that crosses the ABI is the template's and not the caller's. Named without its type arguments it is refused in the same words a locally declared template is (tests/link/generic_import_no_args).
Generic methods
A method of a class — generic or not — may declare type parameters of its own,
and it is compiled the way a generic function is: one define per (receiver
struct, method type-argument tuple), named by appending $ and each of the
method's mangled type arguments to the receiver's method symbol. The IR of an
instantiation is the IR of the method somebody would have written for that type,
with this first and the symbol renamed (tests/cases/gen_method).
class Chooser {
flip: boolean = false;
pick<T>(a: T, b: T): T { return this.flip ? b : a; }
}
class Box<T> {
value: T;
constructor(value: T) { this.value = value; }
keep<U>(other: U): T { return this.value; }
}
const c = new Chooser();
c.pick(1, 2); // @Chooser.pick$i32
c.pick("a", "b"); // @Chooser.pick$str
new Box<i32>(7).keep("seven"); // @Box$i32.keep$str
-
The type arguments are inferred from the arguments, exactly as a generic
function's are, and never written at the call:
h.get<i32>(7)isType arguments are not written at a call site in Nish: `T` is inferred from the arguments, so write `h.get(...)`(tests/cases/reject_generic_method, and through a field path,reject_generic_method_path_type_args). A method type parameter no parameter mentions is refused at the declaration:Cannot infer `U` for `Holder.make`: a type parameter is inferred from the arguments, and `U` appears in none of them; give `Holder.make` a parameter that mentions `U`(reject_generic_method_uninferable). -
The receiver is part of the key. On a generic class,
Box<i32>.pair<string>andBox<string>.pair<string>are two instantiations, and a tuple asked for twice is one: two receivers times two method tuples are fourdefines and no more (tests/cases/gen_method_generic_class). Inside the body the class's parameters and the method's are both bound. -
A method's type parameter may not reuse a name of its class's, because
both are in scope in its body and every
Tthere would have two meanings:Type parameter `T` of `Box.map` shadows `Box`'s own `T`: a class's type parameters are in scope in its methods, and a diagnostic that names `T` has to mean one of them; give the method's another name(reject_generic_method_shadow, once per class however many times it is instantiated). -
A method type parameter may be constrained, with a generic function's
rules: a
U extends Shapehas exactlyShape's members, read straight out of the concrete struct (tests/cases/gen_method_constraint); an unconstrained one has none, through a local, a field or another generic call alike (reject_generic_method_member,reject_generic_method_member_through_call); a type argument is held to the constraint at each call (reject_generic_method_unsatisfied); and the constraint may mention no type parameter, the class's included, because it is resolved once for every receiver (reject_generic_method_constraint_class_param). -
Recursion terminates or is refused by name, with a generic function's
rule and sentence:
Monomorphising `Walker.walk` would not terminate: `Walker.walk<i32>` asks for `Walker.walk<i32[]>`, ...(reject_generic_method_recursion). -
-gnames an instantiation as written —Box<i32>.pair<string>— and links it as mangled,Box$i32.pair$str(tests/cases/dbg_generic_method). -
The whole-program rule holds: a generic method of an exported class is
instantiated, and defined once, in the class's module, and every other module
that calls it
declares it (tests/link/generic_method_import). -
In the C header an instantiation is declared under the
nish_gen_spelling a generic function's takes, bound to its symbol, with the object first:int32_t nish_gen_Holder_pick_i32(struct Holder *this_, int32_t a, int32_t b) NISH_SYMBOL("Holder.pick$i32");(tests/cases/gen_method_export). The wasm and N-API sidecars bridge no method. An exported class's generic method that nothing calls has no symbol, so with a sidecar flag it is refused:`Holder.pick` is generic, so it has no single C signature: a generic method becomes a symbol only where it is instantiated, and this program instantiates none(reject_generic_method_unused). -
A constructor has no type parameters of its own: it takes its class's,
written after
new.constructor<T>()is a syntax error (reject_generic_method_constructor), as it is in TypeScript. -
A generic method is a member like any other, so a field or another method
of the same name is
Duplicate member `pick` in class `Chooser`(reject_generic_method_field_clash). -
$is refused in a method name where it could spell an instantiation: in any generic method's own name, which every instantiation's symbol is built on (pick$i32<V>ati32would bepick<U1, U2>ati32, i32;reject_generic_method_dollar_generic,…_generic_first,…_generic_class), and in a plain method's name when the part before the$is a generic sibling's —pick$i32beside a genericpickwould bepick<i32>'s symbol (reject_generic_method_dollar).
Function parameters
A parameter of a top-level function may have a function type written
inline in its annotation, and its argument is then a function the checker can
name at the call: a top-level
function written by name, an arrow written right there, or a local holding a
function reference, which the call wraps in an arrow that calls it
(Function references). Such a function is
a template, with or without type parameters, and each callee it is given is
an instantiation of its own, exactly as each type-argument tuple is: one
define per (template, type arguments, function arguments), whose body calls
its callee directly (tests/cases/fnarg_named). There is no function pointer
and no indirect call; the parameter has no LLVM parameter at all, and the whole-program fixpoint sees an ordinary direct call
it can prove things through (wp23-language-surface.md
§6, and wp28-compatibility-mode.md §7.4 for what
the other kind of callee would cost). A parameter whose function type comes
through an alias (type F = (x: i32) => i32, then f: F) is not a function
parameter but an ordinary parameter holding a
function reference.
const apply = (f: (x: i32) => i32, x: i32): i32 => f(x) + 1;
const square = (x: i32): i32 => x * x;
export const main = (): i32 => {
const a = apply(square, 3); // @apply$fn.6.square, calling @square
const b = apply((x) => x - 1, 3); // @apply$fn.16.nish_main$arrow0
return a + b;
};
-
The callee is part of the symbol. An instantiation's name is the template's, then its type arguments, then
$fn.<length>.<symbol>for each function argument, which is self-delimiting however many$the callee's own symbol holds; a person reads it asapply<square>ormap<i32, str, (x) => ...>in every diagnostic, the--emit-checkeddump and the-gDWARFname. Two callees are twodefines; the same callee twice is one (tests/cases/fnarg_named). -
An arrow argument is lifted into a function of its own, named after the function it is written in (
nish_main$arrow0) — a symbol no declaration can spell, because a declared name may not hold a$. It may be writtenx => ...,(x) => ...or with annotations, with a concise or a block body (tests/cases/fnarg_arrow), and under-git has aDISubprogramof its own, placed where the arrow starts (tests/cases/dbg_fnarg_arrow). -
Inference runs in two phases. The value arguments bind what they can first; then each function argument is resolved against its parameter's function type. An arrow's unannotated parameters take the types the value arguments bound, a named function's signature binds any type parameter it mentions, and an arrow with a concise body binds the result type from its body, so
map(xs, (n) => n > 1)over ani32[]ismap<i32, boolean, (n) => ...>. A numeric literal argument is checked after the others, against what they bound, so a fold's identity0.0over anf64[]is anf64in either number mode (tests/cases/fnarg_generic). -
A function parameter may be called and passed on, and nothing else. Passing it to another function-typed parameter forwards the callee as part of the key, so the innermost call is still direct (
tests/cases/fnarg_forward). Storing, returning, comparing or putting it in an array is`f` is a function parameter and can only be called or passed on as a function argument: it names the callee each call is compiled for, not a value, so it cannot be stored, returned, compared or put in an array(tests/cases/reject_fnarg_store,reject_fnarg_return). -
An arrow may read the
constlocals and the parameters of the function it is written in (WP29 capture). Each name it reads is handed to it as an extra argument when the call is made: the lifted arrow gains a parameter per capture, the instantiation that runs it gains one too and forwards it, and every call stays direct — no environment object, no indirect call. The captures are not part of the instantiation's key, so one arrow is onedefinehowever many values it reads. A narrowing in force at the call holds in the body (if (name !== null) { each(xs, (x) => use(name)) }readsnameas astring). The arrow may write through what it captured —infos.push(x),box.n = v— because the call runs before the function goes on (tests/cases/fnarg_capture,fnarg_capture_narrow,fnarg_capture_local,fnarg_capture_escape). What it may not do:-
read
this(NL2590,reject_fnarg_capture_this): bind it, or what the arrow needs of it, to aconstbefore the call; -
read a
let(NL2585,reject_fnarg_capture_let), whose value could change after the arrow is handed it;nish --fixcopies it into aconstjust before the call (tests/fix/fnarg-capture-let); -
assign to a name it captured (NL2586,
reject_fnarg_capture_write): the assignment would change only its own copy; -
name a function parameter of the function it is written in (NL2587,
reject_fnarg_capture_function_param), which is bound at instantiation and is no value; -
capture a
Secret, a scope, a lock, a guard, a channel, a foreign handle or aCPtr(NL2588,reject_fnarg_capture_secret,reject_mutex_guard_capture,reject_ffi_handle_closure_capture,reject_ffi_handle_field_capture), which their own rules keep in the block that made them; -
read anything from outside it when it is written inside another arrow
(NL2501,
reject_fnarg_capture_nested).
An arrow given to
exposeorexposeWithstill captures nothing (NL2335, NL2589;reject_secret_expose_capture,_forwarded), an arrow handed on to a parallel form orspawnmeets their rules where it runs (reject_fnarg_capture_parallel_write,_spawn_write,tests/link/fnarg_capture_forward), and a recursive function that passes itself a new arrow on every call meets the instantiation limit (reject_fnarg_capture_recursion). Escape analysis and the parameter attributes read a capture as an argument to the arrow, at the arrow's own parameter for it: what it names escapes only when the arrow keeps it, or hands it to something that does, so an object an arrow only reads may be an alloca and a parameter it only readsreadonly nocapture(mem_capture_read_stack,fnarg_capture_readonly; the negatives aremem_capture_forward_keep,mem_capture_callee_keep). An element of a captured array is kept unless the arrow proved it keeps none, and a captured array is never one whose strings go to the caller with it (mem_capture_element_keep). The capture stays an escape, as it was before, when anything but an ordinary function stands between the call and the arrow (a data-parallel form,spawn, a task, an async function:tests/link/mem_capture_parallel_forward,mem_capture_async), and a capturedpartsarray is not released byjoin, whose release depends on order rather than escape (mem_join_parts_captured). An arrow anywhere but as such an argument or as a top-levelconstisan arrow in value position is not yet a function reference: declare it as a top-level function and name it(tests/cases/reject_fnarg_arrow_value), or NL2731 when it captures (Function references). -
read
-
A function argument has exactly its parameter's type, once the type parameters are bound, because the body calls it with those types and reads that result:
Argument 1 of `apply`: `widen` is `(x: i32) => i64`, and `f` is `(x: i32) => i32`; a function argument must have exactly its parameter's type(tests/cases/reject_fnarg_mismatch). The argument must be the name of a top-level function, an arrow, or a local that holds a function reference (tests/cases/fnref_arg): the call passes an arrow that captures the local by value, so the callee calls the function that was passed even if it reassigns where that came from (fnref_arg_reassign). An arrow passed there may call a reference it captures, narrowed first if it is nullable (fnref_arg_capture,fnref_arg_capture_param). A field or element would be read again at each call the callee makes, where JavaScript reads it once, so it is refused,ops[next(b)]andb.opalike:`ops[next(b)]` is read again at each call `twice` makes, where JavaScript reads it once at the call: read it into a `const` first and pass the `const`(NL2759,reject_fnref_arg_effect,reject_fnref_arg_field,reject_fnref_arg_element_nullable). A nullable local is passed once a test has narrowed it:`h` may be `null`, and `apply` calls it: pass it once a test such as `h !== null` has narrowed it(NL2760,tests/wordings/nl2760_fnarg_nullable_reference). A local that a later argument of the same call assigns is refused, since the call reads it after its arguments where JavaScript reads it before them:`h` is assigned by a later argument of `apply`, and the call reads it after its arguments where JavaScript reads it before them: read it into a `const` first and pass the `const`(NL2761,reject_fnarg_ref_assigned_later). A reference of another signature is refused as any other function argument is (NL2340,tests/wordings/nl2340_fnarg_reference_arity); any other value is refused and named by its type,`n` is `i32`, which is not a function(reject_fnarg_not_name). The builtins that check the body they are handed —parallelMap,parallelMapInto,parallelReduce, a scope'sspawn,expose,exposeWith,regionandenterWith— take only a function or an arrow named at the call, so a reference there is`h` is a function reference: `parallelMap` takes a top-level function or an arrow named at the call(NL2734,tests/wordings/nl2734_fnref_to_function_parameter). A generic function names no one function and is refused with the arrow to write instead (reject_fnarg_generic_name), and an arrow takes exactly its type's parameters (reject_fnarg_arity). An unannotated arrow parameter whose type no value argument binds (reject_fnarg_infer) and a block-bodied arrow whose result only it could bind (reject_fnarg_block_return) must be annotated. -
Only a top-level function's parameter makes a function type a function parameter. Written inline on a parameter of a method or a constructor, or of a function type that is itself a function parameter's, it is
`f` cannot have a function type here: a function type may only annotate a parameter of a top-level function, which is monomorphised for each function it is given, and this is a parameter of a method or a constructor, witha parameter of a function typefor the second (tests/cases/reject_fnarg_method,reject_fnarg_nested); a method or a constructor takes a reference through an alias instead. Anywhere else — a field, a local, an array element, a return type, an alias — a function type is the type of a function reference, and what is stored there is checked against it:Field `run` of class `Handler` is (i32) => i32 but its initializer is i32(reject_fnarg_field),Return type mismatch: function returns (i32) => i32 but expression is i32(reject_fnarg_return_type), and a reference local is initialised where it is declared,Variable `g` must be initialized(reject_fnarg_local). -
Across modules, an instantiation is made in the template's module (WP18 G7), so it may call a private function, or an arrow, of the module that asked. That callee is
hiddenrather thaninternal— in the final link, exported from nothing — and the template's moduledeclares it with the attributes of itsdefine(tests/link/fnarg_private). A hidden symbol is still global in the executable, so one the program's own package declared, which carries no prefix, ends with a$(@square$): C cannot spell it, so a task calledread,writeorfreeis never the definition the runtime's calls into the C library reach (tests/link/thread_scope_libc_named_tasks, #610). The interop sidecars describe no instantiation that was given a function, and no lifted arrow: neither has a name a host could call.
Function references
A name bound to a top-level function, in either spelling, written anywhere but
a call's callee or a function argument is a function reference
(a local holding one may itself be passed as a function argument, below): a value
of the function's type, (x: i32) => i32. A function type names a reference
everywhere but inline on a parameter of a top-level function
(Function parameters): on a local, a field, an array
element, a return type, a type alias, and a parameter typed through an alias.
type Op = (x: i32) => i32;
const double = (x: i32): i32 => x * 2;
const negate = (x: i32): i32 => 0 - x;
class Stage {
op: Op;
constructor(op: Op) {
this.op = op;
}
run(x: i32): i32 {
return this.op(x); // a direct call of the dispatcher for `Op`
}
}
export const main = (): void => {
const s = new Stage(double);
console.log(`${s.run(5)}`); // 10
s.op = negate;
console.log(`${s.run(5)}`); // -5
};
-
What a reference can hold. A function this program defines, at the top
level of a module, that is not generic, not
async, has no rest parameter and takes no function parameter, and whose arrow, if it is one, captures nothing. Each other kind is refused where it is named:-
`id` is generic: a function reference names one non-generic top-level function(NL2725,tests/cases/reject_fnref_generic); -
`later` is async: a function reference cannot hold it, because its call is a promise(NL2727,reject_fnref_async); -
`abs` is declared in C: a function reference names a function this program defines(NL2728,reject_fnref_ffi); -
`sum` has a rest parameter: a function reference names a function whose every parameter the caller passes(NL2729,reject_fnref_rest; a defaulted parameter the same,nl2729_fnref_default), and a written function type with a rest, optional or defaulted parameter,`(x: i32, y?: i32) => i32` has an optional parameter: a function reference names a function whose every parameter the caller passes(NL2729,reject_fnref_type_optional,reject_fnref_type_rest); -
`bump` takes a `MutexGuard`: a guard is handed only to a function named at the call, so a function reference cannot name it; call the helper by name(NL2739,tests/wordings/nl2739_fnref_guard); -
`apply` takes a function parameter, so it has no single body to refer to(NL2726,tests/wordings/nl2726_fnref_template), and a function type whose own parameter is written as a function type,`(g: (x: i32) => i32) => i32` takes a function parameter, so no function it could name has a single body to refer to(NL2735,tests/wordings/nl2735_fnref_type_function_param); -
a method,
`c.step` is a method: a function reference names a top-level function, and a method needs its receiver(NL2732,tests/cases/reject_fnarg_alias); -
an arrow that captures,
this arrow captures `k`: a function reference names a top-level function and carries no environment(NL2731,reject_arrow_as_value,reject_fnref_capture), and one that does not but is not a top-levelconst,an arrow in value position is not yet a function reference: declare it as a top-level function and name it(NL2336,reject_fnarg_arrow_value).
-
-
Where a reference goes. It is held in a local, a field (of a class or an
interface), an array element, a
Mapvalue or a return value, and passed as an argument to a parameter typed through an alias, of a function, a method or a constructor (tests/cases/fnref_field,fnref_array,fnref_stores,fnref_generic,fnref_map), or to an inline function-typed parameter (Function parameters,fnref_arg): that call passes an arrow that calls the reference, so the callee is instantiated for the call and calls through the dispatcher of the argument's place. AMapmay hold an array of references or a nullable of either as its value (fnref_map_array,fnref_map_nullable);??on a nullable reference tests it againstnull, tag 0 (fnref_nullable_coalesce); aMapkey may not be a reference,`(i32) => i32` cannot be the key type of `Map<(i32) => i32, i32>`(reject_fnref_map_key). A reference has no zero value: a local is initialised where it is declared (reject_fnarg_local), andnew Array<(i32) => i32>(n)is`new Array<(i32) => i32>(n)` would zero-fill with 0, which names no function; build it with `[]` and `push` instead(NL2730,reject_fnref_zero_fill);new Array<F | null>(n)fills withnulland is allowed, except as module storage, which holds no| null(NL2686,reject_fnref_storage_nullable). For the same reason an array of references has nofill,unshiftorsplice:fillis refused as a byte copy (reject_fnref_row_fill), the other two as an unknown method (reject_fnref_row_unshift,reject_fnref_row_splice). A module constant may not hold one (Module constant `alias` must be a number, boolean, string, or a `readonly` array of numbers written out as a literal, not (i32) => i32; there is no top-level code to build anything else). An arrow argument may capture aconstreference local like any other value (fnref_capture). -
What a reference does. It is called —
h(3),this.op(4), or an element called in place — and compared with===and!==, which are true when the two name the same function (fnref_eq). Nothing else: printing it, putting it in a template string or any other use is`f` is a function reference: it can be called, compared, stored and passed, nothing else(NL2737,reject_fnref_print, and asJSON.stringify's argument,reject_fnref_stringify; a field, an element or a nullable reference alike,reject_fnref_stringify_place);jsonEncoderefuses a type that holds one (`jsonEncode` cannot handle `Stage.op`, which is `(i32) => i32`,reject_fnref_json);<and the other orderings take numbers only (reject_fnref_order,reject_fnref_order_gt), as+and the other arithmetic does (reject_fnref_arith); and a function type inside another type prints in parentheses,((i32) => i32)[](reject_fnref_type_name). -
A reference may be
null.F | nullholds a function or nothing: a field of that type startsnull,new Array<F | null>(n)holdsnnulls, and in memorynullis tag 0, a tag no function has. A call goes through it only once a test proves it holds a function,h !== nullor an earlyreturnonh === null; a call before one is`h` may be `null`: a nullable function reference is called only after a test such as `h !== null` proves it holds a function(NL2758,reject_fnref_nullable_call). A field is not narrowed: read it into aconstand test that (reject_fnref_nullable_field_call). A number is no reference, so a nullable field initialised with one is refused as a non-null one is (NL2013,reject_fnref_nullable_field_literal,reject_fnref_nullable_field_negative). The rules otherwise follow Nullable types (tests/cases/fnref_nullable). -
In memory a reference is an
i32tag, the function's position, from 1, among the program's functions of that signature: never an address. A call through it is a direct call of an internal dispatcher,@$ref.<module>.<signature>.<n>, that takes the tag and the arguments, switches on the tag and tail-calls each function directly, the last one as thedefaultarm; there is no function pointer and no indirect call (tests/cases/fnref_field). Each place — a local or a parameter, a function's return, a field, and the elements of every array of one reference type (aMap's values are such elements) — has a set, the functions that can flow into it, and a call dispatches over its place's set only: a set of one's dispatcher is one tail call with noswitch, and an empty set's dispatcher traps (fnref_empty_set,fnref_place_set). The dispatcher carries the attributes its members share, so a call through a set whose members all leave an argument alone leaves it on the stack. A set that mixes members that keep a pointer argument with members that do not is refused,this call through `h` can reach `u3`, which keeps `p`, and `u0`, which does not, so every call here would allocate `p` in the arena; ...(NL2736,reject_fnref_escape_join,fnref_escape_join).Map.getreads the place aMap's values share with the arrays of that type, so a call through what it returns dispatches over the functions stored into any of them (fnref_map,fnref_place_set_map); where one of those keeps a pointer argument the others leave alone, that call is the mixed set above and refused (reject_fnref_place_set_map). AResultarm is not a place, so a reference directly in one —Result<F, E>,Result<F | null, E>,Result<T, F>, including through a generic's type parameter — is refused,`Result<(i32) => i32, string>` cannot hold a function reference in an arm: ...(NL2762,reject_fnref_result,reject_fnref_result_generic); an array of references in an arm holds the element place and compiles (fnref_result_array). An assignment used as a value,k((r = inc)), stores into its target and hands the same function on (fnref_assign_value). A set past eight members is warning NL9017, aswitchwhere a direct call would not branch (tests/wordings/nl9017_fnref_wide_set). Under--profile debugthe dispatcher first checks that the tag is in its set and panics withfunction reference holds a function outside its place's setif it is not; the speed build checks nothing (fnref_debug_check). The IR is in IR_COOKBOOK.md. -
A reference does not cross to a host. A tag means something only to this
program's dispatchers, so in an open-world build (
-o) an exported function whose parameter or return type holds a reference, or a class an exported signature reaches with such a field, isParameter `f` of `apply` holds a function reference that code outside this program can reach: a reference is a tag this program's dispatcher reads, not an address, and a host could hand back one no dispatcher knows(NL2733,reject_fnref_host,reject_fnref_host_field,reject_fnref_no_strict,tests/wordings/nl2733_fnref_host_return). A closed-world--linkbuild exports such a function between its own modules (tests/link/fnref_cross_module).
Still refused: a closure (an arrow that captures), a reference to a method, a
generic or an async function, and a reference as a Map key
(reject_fnref_map_key).
Async functions: async and await
An async function is a task the compiler can size: it may wait, and every
wait is written await on a direct call
(wp49-async-server.md §2). No expression ever has a
promise type, so there is no promise to store, pass or chain, and an awaited
call's frame nests in its caller's. An async function compiles to a frame
and a resume switch (async_lowered, async_await_single,
async_await_chain, async_await_all, async_spawn, async_main,
async_result): one that awaits becomes i1 @f.resume(i8*), which picks up
at the state its frame records and answers whether it finished; one that
awaits nothing is a plain function, and awaiting it is a plain call. The
frames of a program are sized when it is linked, and export async function main is resumed from the process entry until it finishes and its spawned
tasks are done (async_main). A task waits on the outside world with
await sleep(ms), await readable(fd) or await writable(fd): it is parked
on the executor's kernel wait (epoll, kqueue on Darwin) and the others run
until it is ready (below;
wp49-async-server.md, "Stage 5, part 1"). A
program with no async function compiles byte for byte as before, and links
no executor; one
whose async functions are never called keeps them internal, as it would a
plain function (async_unused). A function that awaits loses willreturn
and readonly; one that does not keeps its attributes (async_attributes).
A failure is the answer's Result, Promise<Result<T, E>>, read after the
await as any other Result (async_result).
import { awaitAll, spawn } from "nish:async"
const double = async (x: i32): Promise<i32> => {
return x * 2
}
const log = async (x: i32): Promise<void> => {
console.log(x)
}
const run = async (): Promise<void> => {
const a = await double(3)
let b: i32 = 0
b = await double(a)
await log(b)
const [c, d] = await awaitAll(double(1), double(2))
spawn(log(c + d))
}
-
Who may be
async. A top-levelfunctionor an arrow, top-level or passed to a function-typed parameter. A method, a field or a constructor isOnly a top-level function or an arrow may be `async`(NL1063,reject_async_method), because a task never keeps athis. -
Promise<T>is only anasyncfunction's return type. Anasyncfunction without it isAn `async` function's return type is `Promise<T>`(NL2655,reject_async_function,reject_async_arrow_declaration,reject_async_return_missing);Tis written inside it,Promise<void>for none. Anywhere else — a parameter, a field, a local, a type alias, an interface member —Promise<T>is`Promise<T>` is only an `async` function's return type(NL2654,reject_async_promise_parameter,_field,_local,_alias,_interface_member). -
awaitis written inside anasyncfunction (`await` is only written inside an `async` function, NL2641,reject_await,reject_await_constant), on a direct call to one:await xon a value is NL2643 (reject_async_await_value),awaiton a plain function's call NL2644 (reject_async_await_plain_call). Inside anasyncfunctionawait (f(x))is the operator, as TypeScript reads it; outside oneawait(f(x))is a call to a function namedawait, NL2651 (reject_async_await_parenthesised). -
awaitstands at the top of a statement: an expression statement, a declaration's initializer, areturnor the right side of a plain=. Inside another expression — an argument, a condition, an operand, a compound assignment — it would hold a value across the wait, and is NL2642 (reject_async_await_nested,_condition,_compound); aforinitializer is not a declaration statement, forawaitAlltoo (reject_async_await_all_for). Bind the answer to aconstfirst. -
A call to an
asyncfunction is awaited where it is made, or handed tospawnorawaitAll. Anything else would make a promise a value:`load` is `async`: a call to it is awaited where it is made(NL2645,reject_async_unawaited_call,reject_async_unawaited_argument)..thenis NL2650 (reject_async_then),Promise.allNL2649 (reject_async_promise_all), andfor awaitis`for await` is not supported here(NL2133,reject_for_await_template,tests/wordings/nl2133_for_await). -
nish:asyncexportsspawnandawaitAll, reached only through the import.spawn(f(x))startsfas a task of its own and does not wait for it; it takes a direct call to anasyncfunction (NL2644), and since nothing reads its answer,fanswersvoidor aResultit reports itself (NL2646,reject_async_spawn_answer).awaitAll(f(x), g(y))waits for two to eight calls at once and is awaited into one name per call,const [a, b] = await awaitAll(f(x), g(y)); one call, or anawaitAllnot so destructured, is NL2647 (reject_async_await_all_one,reject_async_await_all_bare). So is a pattern over it that is not one plain name per call, each with the reason: an object pattern (the answers are read by position), a hole (it would drop an answer; name it_b), or a default, rest or nested slot (reject_async_await_all_object,reject_async_await_all_hole,reject_async_await_all_default). -
An
asyncarrow is passed only where it is awaited. The parameter's function type answersPromise<T>, and the callee awaits each call to it. Anasyncarrow where the type answers a plainTis NL2639 (reject_async_arrow); a plain function where it answersPromise<T>is NL2640 (reject_async_plain_to_promise_parameter). -
No recursion through
await. An awaited call's frame is nested in its caller's, so a cycle in the call graph that runs through anawaithas no size:recursion through `await`is refused at theawaitthat closes the cycle, direct or mutual (NL2648,reject_async_await_self,reject_async_await_mutual). Write it as a loop. -
A spawned task's arguments outlive the spawner.
spawn(f(x))may return, and its caller with it, whilefstill waits. A number, abooleanor an enum is copied by value, and astringis copied into the task's own region before the task first runs (async_sleep_spawn_string,tests/link/async_spawn_args). An object, an array, aMapor aSetis shared: the task holds the spawner's own, and sees and updates its numbers (async_spawn_shared). A function that feeds a task a shared argument, itself or through a callee, releases no arena scope over what it allocated, so the object outlives every task that holds it (tests/link/async_spawn_shared_outlives). An argument of any other type —T | null, aRegion, aSecret— is`spawn` hands `f` an argument of type `T`, which lives in the spawner's region and is freed when the spawner returns(NL2673,reject_async_spawn_array_arg). A shared object or array that holds, in any field or element, aRegion, aSecret, a handle class ausingblock closes, or a scope, lock, guard or channel ofnish/threadsis refused too, with what it holds named: its spawner may free, wipe or close that part while the task still waits (NL2691,reject_async_spawn_region_field,reject_async_spawn_secret_field,reject_async_spawn_handle,reject_async_spawn_threads_field). -
A spawned task keeps its own memory out of what it shares. The task's
region is freed when the task ends, so the region rules hold it to a
shared argument, and everything reachable from it, as memory outside its
region (Regions): storing an object or a
stringit made into a field or an element is NL2609, directly, through an alias, a method or a callee (reject_async_spawn_store_region,reject_async_spawn_store_string,reject_async_spawn_store_alias,reject_async_spawn_store_method,reject_async_spawn_store_callee,reject_async_spawn_store_reachable,reject_async_spawn_store_element,reject_async_spawn_store_bound,reject_async_spawn_await_all). A call that hands a function or method both a shared object it writes into and astringor object of the task's is NL2609 as well, since nothing says the one is not kept in the other (reject_async_spawn_store_string_callee,reject_async_spawn_store_string_method,reject_async_spawn_store_copied), and growing a shared array,MaporSet—push,set,add,spawnPipedorspawnPipedWithinappending to its output arrays, on the argument or on what a method answers out of it — is NL2615 (reject_async_spawn_grow,reject_async_spawn_grow_method,reject_async_spawn_grow_map,reject_async_spawn_grow_piped), and so is a builtin with no body of its own that does either,reserveon a sharedMap(reject_async_spawn_grow_reserve) orjsonDecodeinto a shared object (reject_async_spawn_store_json). Copy numbers into it instead. The task keeps none of what it shares in memory of its own: a shared object, or anything reachable from it, put into a constructor argument, an array or object literal, a field or element store,push, aMap'ssetor aSet'sadd, a method that may keep its argument in its receiver, or a callee that stores one argument into another is NL2610, as in aregion(...)body (reject_async_spawn_keep_holder,reject_async_spawn_keep_literal), and so is an element read out of a shared array (for (const t of shared),reject_async_spawn_keep_element). An object a helper makes may hold what the helper was handed, so in a task a fresh helper's answer other than a string, awaited or not, carries its arguments' origins too, and a store through a holder a helper built from the shared object is NL2609 (reject_async_spawn_keep_returned). An argument is left out of that join only when the answer's type cannot reach it: no class, array or string type the answer may hold (through fields, elements, a nullable or a union) is one the argument may hold, and neither side holds a class that implements an interface. So aPgAsyncResultawaited from a query on a sharedPgAsyncPoolis the task's own, while aHolderof a sharedTallyor aBadgeof an interfaceTallyimplements is still the shared object's (reject_async_spawn_answer_reach). Read and update the shared object in place instead. Three refusals are conservative, and known to refuse sound programs: a call that hands a callee both a shared object it writes into and any string or object of the task's, even when the callee writes only numbers; a store into the shared object of what anawaitanswers (const x = await helper(tally); tally.child = x), since an awaited answer is unknown to the walk; and a store of the task's memory into a holder a helper answered from the shared object (wrap(tally).note = s), which may be the task's alone. A task may not itself spawn a task with an object or an array, directly or through a callee, since what it hands on may be its own region's (NL2689,reject_async_spawn_nested); a task with only copied arguments may. -
A spawned task's function argument captures only what the task may
share. A top-level function, or an arrow written at the call, is a
compile-time argument: the call is an instance with the function in its
body (
async_spawn_fn_arg). The arrow may read locals of the function it is written in when each is a number, abooleanor an enum, copied into the task's frame, or an object or array that holds, through any field or element, noRegion, noSecret, no handle class and nonish/threadsscope, lock, guard or channel. Its strings and nested objects are fine (aPgAsyncPoolholds both). Any other capture is NL2690: astring, a nullable value or a union itself, or an object holding aRegion(reject_async_spawn_block_arg,reject_async_spawn_handler_holds_region). A capture of a handle, or of an object holding aSecret, is refused sooner, by the rule for any arrow argument (NL2588,reject_async_spawn_handler_holds_handle,reject_async_spawn_handler_holds_secret); the capture rule refuses them as well, a check behind that one. A captured object is shared with the task as if handed to it: the spawner releases no arena scope over it, and the arrow's body is walked with the task's rules, the capture as memory the task does not own, so the arrow may update its numbers and call its methods (a serve handler awaiting a query on a capturedPgAsyncPool) but may not store the task's memory into it or grow it (NL2609, NL2615,async_spawn_handler_reach,reject_async_spawn_handler_*). The arrow's parameters carry what the task passes at each direct call; one it reaches through a further function parameter is unknown, so storing it into a capture is refused. Each arrow is an instance of its own, so a refused arrow at one call site is not hidden by an admitted one at another. -
Each spawned task has a region of its own, entered whenever the task
runs and freed when it finishes, so what a task allocates is given back
when it ends, not when
maindoes (tests/link/async_region_per_task). -
Each
awaitAllcall has a region of its own, made before it first runs and entered whenever it runs. When the call finishes, what it allocated joins the caller's memory, so an answer that points into it (astring, an array, a class) stays readable, and is given back when the caller's would be. So two calls that each hold memory across anawaitnever free or overwrite each other's, whatever its size or whichever finishes first (async_await_all_lanes_large,tests/link/async_await_all_lanes). What a call allocates and does not answer is given back when it finishes, so its temporaries never reach the caller's memory (tests/link/async_await_all_retain). What the caller does keep is the answer and about 1.3 KB more per call, the unused end of the region's first block, until the caller's own memory is given back (ASYNC-17 indocs/security/async.md). -
A
mainthat never waits runs the loop too. Anasyncmainthat awaits nothing compiles to a plain call, as any such function does; one that spawns tasks returns first, then the executor runs every task it spawned, and the program exits withmain's answer once they are done (tests/link/async_sync_main_loop). -
No
asyncin a wasm build. A task is resumed by the native event loop, which a wasm module does not have: under awasm32target or--profile wasianasyncfunction is NL2652 (reject_async_wasm,reject_async_wasi_profile).
Waiting on a timer or a descriptor: sleep, readable and writable
nish:async exports three waits on the executor. Each takes one i32 and
answers a boolean, and each is written only where it is awaited:
| Call | Waits until | Answers false at once when |
|---|---|---|
await sleep(ms) |
ms milliseconds have passed |
ms is negative |
await readable(fd) |
fd has bytes to read, or its peer closed it |
fd is not an open descriptor |
await writable(fd) |
fd can take bytes |
fd is not an open descriptor |
A wait that is answered at once does not suspend the task. A regular file
never blocks, so readable and writable answer true for one at once.
While a task waits, the others run; when none can run, the executor blocks
in the kernel until a descriptor is ready or the first timer is due, and
resumes the tasks it woke: the descriptors in the order the kernel reported
them, then the expired timers, earliest deadline first, ties in the order
they were made (async_sleep, tests/link/async_timer_order,
tests/link/async_readable). A wait holds a duplicate of its descriptor, so
closing the descriptor while a task waits on it does not end the wait, and
the number's reuse cannot answer it (docs/security/async.md,
ASYNC-4).
import { readable, sleep, spawn } from "nish:async"
import { netRead } from "nish:net"
const echo = async (fd: i32): Promise<void> => {
const ready = await readable(fd)
if (!ready) {
return
}
const buf: u8[] = new Array<u8>(4)
console.log(netRead(fd, buf, 0, 4))
}
const tick = async (): Promise<void> => {
await sleep(10)
console.log("tick")
}
export const main = async (): Promise<i32> => {
spawn(echo(0))
spawn(tick())
return 0
}
-
A wait is awaited where it is written.
sleep(10)as a statement, asspawn's argument or anywhere else a value would be kept is`sleep` is a wait on the executor, written only where it is awaited(NL2671,reject_async_sleep_bare,reject_async_sleep_spawned), and so arereadableandwritable. -
It takes one
i32:`readable` takes one `i32`(NL2672,reject_async_wait_argument). -
readableandwritablehave no deadline of their own. A descriptor that never becomes ready parks its task until it does, or until the task is cancelled;readableWithinandwritableWithin, below, take one (docs/security/async.md, ASYNC-6).
Bounding a wait and cancelling a task
nish:async bounds a descriptor wait, and cancels a task, with six more
calls:
| Call | Answers |
|---|---|
await readableWithin(fd, ms), await writableWithin(fd, ms) |
true once fd is ready; false when ms milliseconds passed first, when the wait could not be made, or when the task is cancelled. A negative ms waits without a limit |
taskId() |
the running task's handle, an i64; main's is 0 |
cancel(task) |
true when the task was still running, so it is now cancelled; false for a finished task or main |
cancelled() |
whether the running task has been cancelled |
cancelAfter(task, ms) |
as cancel, ms milliseconds from now unless the task finishes first; a later call replaces the deadline, a negative ms removes it |
A cancel is cooperative: it never stops a task in the middle of its code.
Every sleep, readable, writable or bounded wait the task has parked
answers false at once, and every one it makes later answers false
without parking, so the task runs on to its end, its using bindings
dispose and its region is freed. A task that does not await is not reached
until it does. A handle names its task and no later one: a handle kept past
its task's end, once the executor reuses the task's slot, cancels nothing
(docs/security/async.md, ASYNC-18).
import { cancelAfter, cancelled, readableWithin, taskId } from "nish:async"
const serveOne = async (fd: i32): Promise<void> => {
// The whole task gets two seconds; this one read gets half of one.
cancelAfter(taskId(), 2000)
const ready = await readableWithin(fd, 500)
if (!ready) {
console.log(cancelled() ? "cancelled" : "timed out")
return
}
}
-
The bounded waits are waits. Each is written only where it is
awaited (NL2671,
reject_async_within_bare) and takes twoi32s:`readableWithin` takes two `i32`s: the descriptor it waits on, and the milliseconds it waits at most before it answers false(NL2692,reject_async_within_argument). -
The cancellation calls are plain calls, written anywhere, the task's
handle an
i64:`cancel` from `nish:async` takes one `i64`, a task's handle from `taskId()`(NL2693,reject_async_cancel_argument). -
spawnruns its task to its first wait before it returns, so a cancel written afterspawn(f(x))lands afterfhas run that far. -
std/net/servecancels a handler's task when its client hangs up, when itsdeadlinepasses and when the server stops; the asyncfetchand the async Postgres pool answer a cancelled wait as a failure of their own (tests/link/async_cancel,tests/link/net_serve_hangup,tests/link/std_postgres_async_cancel).
Where it is not Node. The source is TypeScript and runs under Node, but the resume order is not Node's, and these are declared differences:
| Node | Nish |
|---|---|
Every await yields to the microtask queue, even on a value that is already there |
An await on a call that finishes without waiting returns at once; nothing else runs in between |
| Microtasks run before I/O callbacks, in a defined order | Tasks resume in the order the kernel reports readiness, then by timer deadline |
A rejected promise throws at the await |
There is no throw: a failure is the call's Result or null, as every Nish call reports it |
spawn(f(x)) runs f to its first await, and the rest of f after the current task yields |
spawn(f(x)) runs f at once until it first waits on the outside world, then returns; a task that never waits has finished, its output printed, before spawn returns (async_spawn) |
setTimeout and socket callbacks run on libuv's loop, which keeps the process alive while anything is registered |
sleep, readable and writable park a task on the executor; the program exits once main has returned and no task waits, a synchronous main included |
A string passed to a spawned call is a shared reference |
It is copied into the task's region; an object, array, Map or Set is shared, and the task may store only numbers into it (NL2609, NL2615) and keep none of it in memory of its own (NL2610); one holding a Region, Secret, handle or nish/threads value is refused (NL2691), as is any other argument (NL2673) |
| A task that never awaits holds the thread, as here | The same: the executor is cooperative and does not preempt a task |
A program whose output does not depend on how its tasks interleave prints the same under both.
Under --emit-ts. An async function, Promise<T>, await, an async
main, spawn(f(x)) and const [a, b] = await awaitAll(f(), g()) are
written as the TypeScript they already are, so the output runs on Node's own
async/await and resumes in Node's order, the left column above
(tests/emit-ts/async/). nish:async is nish/runtime/async.mjs: awaitAll
is Promise.all over its two to eight calls, and spawn starts its task
without awaiting it and keeps it. The launcher of a program that declares an
async function awaits main, then settleTasks(), which waits for every
task spawn started, those tasks' own included, before it exits, as the
native driver drains its queue. await sleep(ms) is a setTimeout that
answers true, or false at once for a negative ms
(tests/emit-ts/async/sleep); readable and writable have no Node
counterpart a task can await, so a program that calls either is
--emit-ts cannot write readable or writable from nish:async (NL4017,
tests/emit-ts/async/refused-readable), and so is one that calls
readableWithin, writableWithin, taskId, cancel, cancelled or
cancelAfter, since a Node promise is not a task the executor can cancel
(NL4018, tests/emit-ts/async/refused-cancel). A program whose spawned tasks print in an
interleaving prints the same lines in another order (async_spawn, the
order row of tests/differential/emit-ts-known.txt).
Data parallelism: nish/threads
parallelMapInto, parallelMap, parallelFrom and parallelReduce run a
function over every element of an array, or every index of a new one, on as
many threads as the machine has and the length is worth
(wp29-thread-surface.md §4.1). They are imported from
the standard library, and the function is a function parameter:
a top-level function by name, or an arrow written at the call — which may read
the const locals of the function it is written in
(below).
import { parallelFrom, parallelMap, parallelMapInto, parallelReduce } from "nish/threads";
const square = (x: f64): f64 => x * x;
export const main = (): i32 => {
const src: f64[] = [1.0, 2.0, 3.0];
const dst: f64[] = [0.0, 0.0, 0.0];
parallelMapInto(src, dst, square); // dst[i] = square(src[i])
parallelMapInto(dst, dst, (x) => x + 1.0); // in place
const k: f64 = 2.0;
const scaled = parallelMap(src, (x) => x * k); // a new array: 2, 4, 6
const halves = parallelFrom(3, (i) => toF64(i) / k); // a new array: 0, 0.5, 1
const total = parallelReduce(dst, (a, b) => a + b, 0.0); // 17
return toI32(total + scaled[2] + halves[2]);
};
-
The module is ordinary Nish, and its bodies are the meaning.
std/threads.tswrites each function as the sequential loop, which is what runs under Node and whatnpm run checktype-checks. The compiler recognises the two templates by module and name and lowers one call in each instance — the loop over the whole range — ontonish_parallel_range(runtime/runtime-parallel.c), which cuts the range into contiguous pieces, up to eight per thread, that the threads claim one at a time as they finish the last, and joins them all before it returns. Everything else in the instance is emitted as written (tests/link/par_map, whosethreads.llis the whole lowering). -
parallelMapInto(src, dst, f)writesf(src[i])intodst[i]for every index ofsrc.dstmust be at least as long, which is checked once, before any element is written or any thread started: a shorter one panics withparallelMapInto: dst has 2 elements and src has 3and exits 1 (tests/link/par_dst_short).srcanddstmay be the same array (tests/link/par_map). -
parallelMap(src, f)answers a new array off(src[i])for every index ofsrc, andparallelFrom(n, f)a new array ofnelements,f(i)at indexi—Array.from({ length: n }, (_, i) => f(i)). Each allocates its result on the calling thread before any element is computed, so nothing but the region can hold it while the threads write it, and then runs the map:parallelMapisparallelMapIntointo that array, andparallelFromthe same region over the indices (tests/link/par_map_new,tests/link/par_from).parallelFrom(0, f)answers an empty array. The rules below hold of theirfas they hold ofparallelMapInto's, except the one aboutdst, which nothing else can reach. -
A short array is the loop it would have been. A map is divided only
once every thread's share carries enough work to pay for handing it to
another thread (wp20-threads.md §8e). How many elements
that is — the grain — is sized per call from a static estimate of what
fcosts: 2^21 over the estimate, which counts one per operation, 32 per allocation, 4 per call, and a loop's body as many times as its literal bound says or 64 times when the bound is not a literal. A map longer than the grain is divided intoceil(length / grain)chunks, at most eight per thread with one thread per core, so the first division is two chunks of half the grain: 2^20 units each. That is where a measured body,(x) => x + 1at an estimate of 3, beat its loop by 1.31x on the minimum and 1.19x on the median, more than any difference between the loop and a byte-identical copy of it (at most 3.3% and 9.1%); at two chunks of 2^19 units the minimum won by 1.16x and the median by only 1.04x, within that 9.1% (wp29-thread-surface.md §8a). So a body estimated at 3, such as(x) => x + 1or(x) => x * x, is divided past 699,050 elements (tests/link/par_map),(x * 31 + 7) % 1000past 262,144 (tests/link/par_map_large), a body with a loop far sooner, and a cheaper body later:(x) => x, estimated at 1, past 2,097,152. Up to the grain the map calls its loop directly on the calling thread, where it inlines; there is no hand-off and no indirect call. The estimate counts a call as a call and not as the callee's body, so a body that hides its work behind a call is divided too little rather than too much. There is no knob for the grain (wp29-thread-surface.md §9). -
The threads are a pool, as many as the cores the process may use. The
first region or scope join that divides starts one worker per core but the
caller's, which is the last of them, and at most 63 workers; they park
between calls (wp20-threads.md §8e). The core count is
the smallest of the CPUs online, the process's affinity mask (
taskset, a container's cpuset) and the cgroup v2cpu.maxquota rounded up, so a program in a container limited to two cores divides two ways, not as many as the host has.NISH_NUM_THREADS, when it is set to a positive number, lowers that count and never raises it:NISH_NUM_THREADS=1runs every region and every scope's tasks on the calling thread, in spawn order, and anything that is not a positive number is ignored. It is read once, by the first region or join that asks, and it decides only how the work is divided: a map's answer and a reduce's bits are the same at any count (tests/run.jsruns everytests/link/thread_*andpar_*program underNISH_NUM_THREADS=1and=2and requires the output it gave without). A Node addon (--emit-napi) keeps a thread per chunk or task for each call instead of a pool, because Node can unload it and a parked worker would outlive its code, and on WASI every region and task runs on the one thread there is. -
parallelReduce(src, f, identity)is deterministic.srcis split intomin(64, ceil(n / 2^20))blocks, each block is folded fromidentity, and the block results are combined left to right.std/threads.tsdoes the same blocking on one thread, so anf64sum is the same bits on one core or sixty-four, compiled or under Node — which is not the bits of a plain left fold over the array (tests/link/par_reduce). The blocks are part of the answer, so where blockkstarts,floor(n * k / blocks), is computed exactly even wheren * kpasses 2^31 (tests/link/par_reduce_blocks, 64 blocks over 70,000,001 elements). An emptysrcanswersidentity. For that to be the fold it looks like,fhas to be associative andidentityits identity. An arrow whose whole body is one operator on its two parameters is held to both: a non-associative one is`(a, b) => ...` combines with `-`, which is not associative: `parallelReduce` folds each block from the identity and then combines the blocks, which is a left fold only for an associative operator(-,/,%,**and the shifts;tests/cases/reject_par_reduce_operator), and a recognised one given anything but its identity —0for+,|and^,1for*,truefor&&,falsefor||, written as a literal — is`parallelReduce` folds every block from its identity, and the identity of `+` is `0`, not `1`: any other value would be counted once per block(tests/cases/reject_par_reduce_identity). A named function is opaque: its associativity, and the identity it is given, are its author's promise, and a wrong one gives a deterministic answer that is not the fold. -
Importing
nish/threadscompiles the program with--threads, because every worker bumps an arena of its own:@nish_arenais declaredthread_localin every module and the runtime is built with-DNISH_THREADS(tests/link/par_map). A program that does not import it is compiled exactly as before.
What makes a region safe to run on several threads is checked at the call, from the whole-program facts, and each rule is reported there:
-
The function writes nothing its caller can see. A store through a
parameter, a call that makes one, or a runtime call that writes —
console.logincluded — is`bump` writes memory its caller can see at main.ts:10:3, and `parallelMapInto` runs it on several threads at once: a parallel body may read what its caller owns and write nothing but its result, naming the write, or the callee that makes it (tests/cases/reject_par_shared_write,reject_par_shared_write_via, andreject_par_console_log, which names the runtime'snish_print). A method of an atomic —add,maxand the rest — is the one shared write a body may make (tests/link/par_atomic). A bounds check, an integer division, the length check ofnew Array(n)andpanicare not writes: each prints and exits, and nothing runs again to see memory. When several threads panic at once, the first claims the program: it prints its whole message and exits 1, and every later one waits for that exit without printing, so a failing run prints one message, never two or a mix of them (tests/link/par_panic_once,thread_panic_once). Output is a write —console.errorandprocess.exitas much asconsole.log(reject_par_console_error,reject_par_process_exit). Reading a sharedMaporSetthat an element or a capture reaches, withget,hasorsize, is a read (tests/link/par_map_table_reads);set,delete,clearand iterating one, whose walk counts itself on the table, are writes (tests/cases/reject_par_map_set,_delete,_clear,_values). Handing a helper a number, aboolean, an enum or astring— astringelement of the caller's array included,verify(hashes[k])— gives it no slot to store into, so it is a read (tests/link/thread_bcrypt_batch, whose task verifies against the parent's hashes). The instance performs every store itself, into slots the partitioner has made disjoint, so a function that writes nothing cannot race. -
A store into an array only this call reaches is not a shared write. A
constthe function binds to its own array literal ornew Array(n)— one whose elements are not stored inline — is private while every use of it is an element, a store into a slot,.length, an array method, afor...of, or an argument of a function that keeps nothing of it. Such a function may fill it,pushonto it, or hand it to a helper that writes through its parameter, and still write nothing its caller can see: the helper's writes through that parameter land in the private array (tests/link/thread_task_private, whose body takes its element apart into a scratch array). Returned, stored, pushed, bound to a second name (in any declarator), named in an arrow, or handed to a function or a constructor that keeps it, the array is shared, and so is every store into it (tests/cases/reject_par_private_kept; the scoped-task twins below). What a private array's slots hold is not private: a store intorows[0][0], or a helper that makes one, writes whatrows[0]reaches (reject_par_private_helper_deep). -
Nor is a write into an object's own slots, when the object is one this
call made. A
constbound to the function's ownnewor object literal is memory no caller, thread or task can see while the call runs, so a method or a function that writes only that object's own fields — scalar state such as anRng's — writes nothing shared:const r = new Rng(seed); r.nextI32()is a body's own business (tests/link/std_random). That is the whole of what is admitted. A write below the object's own slots, into a field or an element of it, counts as a write into whatever it may hold, and once a constructor, a store, apush, aMapor aSethas put the caller's memory there, or memory of unknown origin, that write is refused as any shared write is:hs.push(box); hs[0].bump(),h.b = box; h.b.bump(), a constructor that keepsbox, a method that forwards to a field's method,o.a.b = box, and the same with a capturedRng(tests/cases/reject_thread_task_private_object_element,_field,_ctor_alias,_capture,_map,_set,_nested,_array_own,_forward,_forward_store,_array_rng, andreject_par_private_object_field_rng,_array_rng,_for_of,_held,_kept). Freshness is not followed through a call: an object a helper made and returned is not one this call made. -
What the function allocates dies with its element. A function may build
strings, arrays and objects on the way to its result. They are given back
after every element: the function gets an arena scope of its own, so each
call marks the arena of the thread it runs on and releases it before it
returns, and a map over a million elements holds one element's temporaries
at a time (
tests/link/par_alloc). Under glibc, a program that has divided a region or joined a scope on other threads keeps the memory a release gives back rather than returning it to the kernel, so an element or a task that builds a 1 MB temporary does not fault it in again on every call: 200 tasks that each build one took 48 ms on 1 thread, against 314 ms when every release trimmed the heap. A worker that then finds no work for a second gives the memory back to the kernel before it parks, its own heap included: after ten rounds of 8 tasks that each build a 16 MB temporary, an idle program holds 2.3 MB rather than 68 MB, and after four maps of 1.2 million strings on four threads, 2.3 MB rather than 82 MB (tests/link/par_map_idle). With no worker (one CPU, orNISH_NUM_THREADS=1) nothing parks, so nothing gives memory back early: what a join freed is kept up to the 64 MB at which glibc returns the top of the heap by itself. A region below its grain runs as the loop it would have been, on the calling thread, and keeps that loop's costs: 200 elements that each build a 1 MB temporary take some 320 ms, as the loop does. It compiles with a performance warning, NL9012, because every element pays for the allocation and the release:the body of this `parallelMapInto` allocates per element: `label` answers `i32` but allocates on every call, so each thread marks and releases its arena around every element. Compute the answer without building a string, an array or an object to save both. Two things make that release unsound, and each is refused. An allocation stored into memory, which the escape analysis stops following, is`width` allocates at main.ts:21:21 and stores the allocation into memory, and `parallelMapInto` releases what a body allocates after every element: a parallel body may allocate only temporaries it drops before it returns, and this analysis stops following a value once it is stored(tests/cases/reject_par_alloc_escapes). And a function that reads or moves the arena would see the arena of whichever thread ran the element:`used` reads or moves the arena, and `parallelMapInto` runs it on several threads that each have an arena of their own: a parallel body may not call `Arena.mark` or `Arena.used`, nor open a `using a = arena()` block, nor call `region`, `regionWith`, `child`, `childWith`, `enter`, `enterWith` or `free` (`nish:region`)(tests/cases/reject_par_arena). -
A map's result
Uis a number, aboolean, an enum, astring, or an array of numbers, booleans or enums, because the element's memory is released before the next element runs and a worker's arena is freed when its thread exits. A string or a scalar array crosses by copy-at-join, and the caller then adopts it: as the chunk loop rewinds its arena after each element, it moves the answer to the end of the chunk's transfer block, amalloc'd block of 4 KB doubling to 64 KB, or of the answer's size when that is larger (nish_cross_keep). Once every chunk has finished, blocks holding 64 KB or more in all are linked into the caller's arena's chunk list (nish_cross_adopt), so the answers stay where they are and the join costs one link per block rather than a copy per element; less than that is copied into the caller's arena and the blocks freed, so a small map run in a loop does not leave a chunk behind on every run (tests/link/par_map_join_small). Adopted blocks are then the caller's arena memory like any other, given back when the caller's scope releases what the result was made in, exactly when a copied value would have been (tests/link/thread_adopt_lifetime); a result the body did not allocate — a literal, a string it was handed — is not moved at all (tests/link/par_map_string:parallelMap,parallelFromandparallelMapIntointo astring[], underchurn()). Anything else — an array of strings, of arrays or of objects, aMap, aSet, a class — is`name` answers `string[]`, and `parallelMapInto` hands back only a number, a `boolean`, an enum, a `string` or an array of numbers, booleans or enums, which the join copies out of the worker's arena: anything else would point into memory the worker frees or reuses(NL2348,tests/cases/reject_par_result_type;reject_par_map_result_stringsforparallelMap), because its copy would have to follow every element.parallelMapandparallelFromallocate their result before the region, so theirs is refused there, beforestd/threads.tsis instantiated (reject_par_map_new_result_class,reject_par_from_result_type). A map of 1M short strings,`s${x % 100000}`, took a median of 23 ms on 4 threads and 24 ms on 8, against 38 ms for the same loop written sequentially, and peaked at 32 MB against the loop's 55 MB. On 1 thread it is the loop's time: 54.9 ms against the loop's 57.6 ms and its copy's 56.4 ms, minima of 9 interleaved rounds on one machine. It used to take 72.8 ms there, because the mark sat at the end of the full chunk the result had filled, so every element's body allocated a fresh 64 KB chunk and the rewind freed it; the rewind now keeps that chunk, emptied (tests/runtime-test.c'stest_cross_keep_rewind). Before adoption, when the caller copied every string into its arena one at a time, the map took 56 ms on 4 threads and peaked at 67 MB; two byte-identical copies of that binary differed by 5 ms in the same 7 interleaved rounds. What the body allocates for an element is released with that element, also when the body stores what it builds into an array literal on the way, which the arena scope above does not cover: the chunk loop marks the arena before each call and keeps only the answer (nish_cross_keepmoves it to the transfer block and rewinds to the mark; on the sequential path, below a map's grain, it copies it out and back). A body that builds a 2 KB string, stores it in an array literal and answers a 4-byte slice of it, over 1M elements, peaks at 32 MB on 1 thread and on 8 (63 and 67 MB before adoption), where it used to hold every element's temporaries until its chunk ended and peaked at 2.1 GB (tests/link/par_map_release). -
A frame that runs a map gives its result back when it returns. A
function that calls
parallelMap,parallelFromorparallelMapIntoand lets nothing the map made outlive it frees the result array, and the blocks a string map adopted from its workers, when it returns, as the same loop written sequentially does. A result that is returned, stored into a field or an outer array, captured by a later task, or nested in an array of arrays is kept with whatever holds it (tests/link/par_map_reclaim_escapes). A function that maps 2^22 elements and answers one of them, called 40 times on 4 threads, takes 140.3 ms and peaks at 33.8 MB, where it took 493.3 ms and peaked at 658 MB: 1.87x its sequential loop (262.3 ms) instead of 0.56x (main against a byte-identical copy of itself: about 2%). One call over 2^25 elements is unchanged, 251.3 against 252.5 ms, inside that band (tests/link/par_map_reclaim). A string map's result handed to a function whosestring[]parameter keeps none of it, reading it or handing it on only to another such parameter, is still freed with the frame: 72 string maps over 2^17 whose frames each pass the array to such a callee peak at 20 MB on 1 thread and 22 MB on 8, where they peaked at 237 MB. The parameter keeps it, and the frame keeps the strings, when the callee stores the array or one of its strings into a field, an outer array or aMap, returns either, passes it tonewor to a constructor that handsthisout, stores it through any declarator of a statement or through a local alias, or hands it to a callee that keeps it, and whenever the callee holds an arrow, such as a map's body; the proof covers a parameter typedstring[]exactly, so one typedstring[][]keeps the strings it reaches (tests/link/par_map_reclaim_strings). -
A reduce's
Tstays a number, abooleanor an enum. It is the element, the identity and every partial as well as the answer, folded on several threads, so a string would be copied at every combine; a reduce over strings or objects is refused at the call:`(a, b) => ...` answers `string`, and `parallelReduce` hands back only a number, a `boolean` or an enum: a worker's arena is freed when its thread exits, so anything else would point into freed memory; answer a number and fold those, or run it on the thread that called it, `let joined: string = ""; for (const b of words) { const a = joined; joined = a + b }`(NL2348,tests/cases/reject_par_reduce_result_type, andreject_par_reduce_result_classfor a class). Map the strings to numbers first —parallelReduce(parallelMap(names, (s) => s.length), ...). -
dstis not reachable from an element ofsrc. A function that only reads can still readdstthrough its argument while another thread writes it, so an array ofdst's element type reachable fromT, through fields, elements and| null, is`(r) => ...` can reach a `f64[]` through `r.cells`, which could be the array `parallelMapInto` is writing: another thread would be writing it while this one reads, so `dst` may not be reachable from an element of `src`(tests/cases/reject_par_reaches_dst). It is judged by type, so such an array is refused whether or not it isdstat run time. A globalMaporSetreaches only what it stores: its own arrays are private to it, so aMap<string, i32>beside adst: i32[]is not refused (tests/link/par_map_read_into,par_map_read_into_keys,par_map_read_capture), while aMap<string, i32[]>is, throughx.index.get(k)(tests/cases/reject_par_map_value_dst). A program's own class namedMapis walked like any class (tests/cases/reject_par_map_user_class). A reduce writes no array of its caller's and has no such rule.
A data-parallel arrow reads its function's const locals
An arrow passed to one of the four forms may read the const locals and the
parameters of the function it is written in, and so may an arrow given to a
scope's spawn, by the rules of scoped tasks.
An arrow given to any other function-typed parameter of a top-level function
captures too, by the sequential rules of
function parameters (tests/cases/fnarg_capture_local);
the rules below are the stricter ones a body that runs on other threads keeps.
import { parallelMapInto } from "nish/threads";
export const main = (): i32 => {
const k: i32 = 3;
const table: i32[] = [10, 20, 30, 40];
const xs: i32[] = [1, 2, 3, 4];
const dst: i32[] = [0, 0, 0, 0];
parallelMapInto(xs, dst, (x) => table[x % 4] * k); // 60 90 120 30
return dst[0];
};
-
Each name the body reads is a parameter of the body. The checker
appends it to the arrow's parameters after the ones it wrote, and the call
passes the local's value through every instance on the way to the body and
through the region's context with the other arguments, so a scalar is
copied and a reference is passed as the pointer it is
(
tests/link/par_capture, andtests/link/par_capture_large, whose regions divide). A narrowing in force at the call holds in the body, because aconstcannot change:if (w !== null)around the call lets the body readw.scale. No closure is made: the arrow is still a function of its own, with a direct call to it. -
A captured reference is read-only. A store through it — an element, a
field,
push,pop,fillorset, through a second name the body binds to it, or through an object or an array of the body's own that holds it — is a store into memory the caller owns, and is refused as any shared write is (reject_par_capture_element_write,_field_write,_push,_pop,_fill,_set,_map_set,_alias,_alias_object,_alias_literal,_nested_array,_param_write, and_reduce_writeand_into_writefor the other forms). So is handing it to a function or a method that writes it (_helper,_method). Assigning to the name itself isThe arrow assigns to `k`, which belongs to the function it is written in: a data-parallel body reads a captured value and never writes it, because every thread reads the same one; answer the value from the body instead, and read the answers from the array the call gives back, `const kValues = parallelMap([1, 2], (x): i32 => x)`(reject_par_capture_assign, andreject_par_capture_updatefor ak++after a read). A reducer that assigns to one is shown the sameparallelReducewithout the write,answer the value from the reducer instead, and let `parallelReduce` fold it, `const callsTotal = parallelReduce([1, 2, 3], (a, b): i32 => a + b, 0)`. -
A
letis not captured, since it could change between the call and a thread's read of it:The arrow reads `m`, a `let` of the function it is written in: a data-parallel body may read only a `const` or a parameter of that function, whose value no thread can change while the body runs; bind the value to a `const` before the call, `const mNow = m`, and read `mNow` in the arrow(reject_par_capture_let). -
Only an arrow written directly in a function's body captures. An arrow
inside another arrow reads nothing from outside itself, neither the
function's locals nor the outer arrow's parameters:
The arrow reads `k` from outside the arrow it is written in: only an arrow written directly in a function's body may read that function's `const` locals and parameters, and an arrow inside another arrow reads only its own parameters and the module's top-level names; move the inner call into a top-level function that takes the value as a parameter(reject_par_capture_nested_arrow,reject_par_capture_nested_param). -
thisis not captured by that name, nor a field through it; bind it, or what the body needs of it, to aconstfirst:The arrow reads `this`: a data-parallel body may read the `const` locals and the parameters of the function it is written in, and `this` is neither; bind it, or what the body needs of it, to a `const` before the call (`const self = this`, `const k = this.k`)(reject_par_capture_this). This is a matter of spelling, not of safety: a capture becomes a parameter of the arrow, and a parameter namedthisis what makes a function a method to every analysis that reads one. Aconstbound tothisis captured as the object reference it is, read-only like any other (const self = this, intests/link/par_capture). -
A captured name is read at the call. The body runs where the call is,
so the rules that follow a local of the function through its reads count a
capture as a read of that local there. Three refuse: a destination of a
scope's task read through a capture before the scope joins, by any of the
four forms and on a loop's later pass (
reject_par_capture_scope_dst_index,_indexof,_for_of,_large,_map,_map_into,_reduce,_loop), an element reference read through a capture after apushor apopmay have moved its storage (reject_par_capture_stale_record,_into,_loop,_pop,_for_of,_helper), and an array read through a capture aftersecret(raw)took it (reject_par_capture_secret_moved), each with the rule's own message. Two warn under--warn-portability, at the read inside the arrow as at a direct one: a record copied into an array and written through one copy (NL8003,tests/link/port_rec_capture_copy, andport_rec_capture_quietfor a read before the write), and a byte offset compared with a count or answered by the arrow, which stores it in the form's array (NL8001,tests/link/port_str_capture_offset,port_str_capture_result). A captured name is not a parameter of the arrow to NL8009, which reports it where its function declares it (tests/link/port_num_capture_wide). -
Some values are never captured: a
Mutex, its guard, aChannelor a scope, which are shared only by their own block's rules, aResult, a value that may be absent, aSecret, or anything that reaches one of them:The arrow reads `m`, which is `Mutex<i32>`, and a data-parallel body cannot capture it: a scope, a lock, a guard or a channel is shared only by the rules of its own block(reject_par_capture_mutex,_guard,_channel,_scope,_result,_maybe,_secret;_mutex_fieldfor a lock in a field of a captured object,_mutex_mapfor one in aMap's values). A body that runs a data-parallel form of its own is refused as a shared write, captures or not (reject_par_capture_nested_parallel). -
dstis not reachable from a capture either. A captured array ofdst's element type, or a captured value that reaches one, could bedst, which other threads are writing while the body reads it:`(x) => ...` reads `dst`, and can reach a `i32[]` through `dst`, which could be the array `parallelMapInto` is writing: another thread would be writing it while this one reads; a captured array is known not to be `dst` only when one of the two is a `const` bound to an array of its own, such as a literal or a `new Array`, and the other is another such `const` or a parameter(reject_par_capture_dst,_dst_alias,_dst_field,_dst_param;_self_field_dstforself.outwhilethis.outisdst, and_generic_same_dst, refused in each of two instantiations). A capture is a binding the call can name, so one case is admitted by identity: a captured array and adstthat are two different bindings, one of them aconstbound to an array of its own and the other another suchconstor a parameter of the function, astableanddstare above. A fresh array is memory no binding held before it was made, and a parameter's value was fixed before the function ran (tests/link/par_capture, where a parameter table, and two freshconsts in a generic function, are admitted). Two parameters may be one array and are refused.parallelMapandparallelFromallocate their owndst, which no capture can reach. -
A fresh
dstnothing was handed is apart from every capture (#612). Whendstis aconstbound to an array of its own, and nothing before the call — or in a loop around the call that does not also declare it, or in a function written inside — names it except to index it or read a member, then no capture can reach it: the only way into a value is through a binding, and none was given this one. So the body may read aparallelMapresult, a config object or aconst self = thisbeside it (tests/link/par_into_fresh_pipeline,par_into_fresh_object); indexing it, readingd.length, callingd.fill(0)or walking it withfor...offirst hands it to no one (par_into_fresh_reads). Anything that handsdston first is refused with the message above: a storep.arr = dorthis.tab = d(reject_par_into_store_field,_this_field), an argumentkeep(h, d)orts.push(d)(_helper,_push), an aliasconst e = d(_alias), a branchc ? d : w(_ternary), a holdernew Holder(d)(_holder), a store after the call inside a loop around it (_loop_store, and_loop_store_while,_do,_for_offor the other loops) — the outermost such loop, so a store after an inner loop inside an outer one counts (_nested_loop) — a hand-on on one branch of anif(_if_branch), an earlier call writing the samedst(_twice),dstpassed as the call's ownsrc(_as_src), and the arrow readingdstitself (_captured). Adstbound to a captured array is no freshconst(_from_capture). The rule asks for the one binding of that name in its function, so adstwhose name is declared twice there keeps the stricter rule above (_shadow).
Scoped tasks: using s = scope()
A scope runs a handful of different functions at once, each on a thread of its
own, and joins them all when the block that declares it ends
(wp29-thread-surface.md §4.2). It is the shape
parallelMapInto cannot take: work that is not one function over one array.
import { scope } from "nish/threads";
const sumOf = (xs: f64[]): f64 => { /* ... */ return 7.0; };
const primesBelow = (n: i32): i32 => { /* ... */ return 168; };
export const main = (): i32 => {
const xs: f64[] = [1.5, 2.5, 3.0];
const sums: f64[] = [0.0];
const counts: i32[] = [0];
{
using s = scope();
s.spawn(sumOf, xs, sums, 0); // sums[0] = sumOf(xs)
s.spawn(primesBelow, 1000, counts, 0); // counts[0] = primesBelow(1000)
} // both have run, and both answers are stored
console.log(`${sums[0]} ${counts[0]}`); // 7 168
return 0;
};
-
s.spawn(entry, arg, dst, at)runsentry(arg)on a thread of its own and stores the answer indst[at](tests/link/thread_scope_basic).entryis a function parameter: a top-level function named at the call, or an arrow written there, which may read its function'sconstlocals (below).atis checked at the spawn and again when the answer is stored; out of range isspawn: destination index 3 is out of range for an array of 1 elementsand exit 1. -
s.spawn(entry, arg)ands.spawn(() => ...)have no destination. The task runs as any other does and its scope stores nothing when it joins, soentryanswersvoid: it is seen through the guard of aMutexits argument or a capture reaches, or through aChannelit sends on (tests/link/thread_task_void,thread_task_arrow). A block body isvoidwithout an annotation. An entry that answers a value is`sumOf` answers `i32`, and a `spawn` without a destination stores nothing: a task's answer is kept only by `s.spawn(entry, arg, dst, at)`, which stores it in `dst[at]` when the scope joins, so make the task answer `void` or give it a destination, `const answer2 = new Array<i32>(1)` above the scope's block and `s.spawn(sumOf, xs, answer2, 0)`(NL2511,tests/cases/reject_thread_task_void_result, and_arrowfor() => sumOf(xs)). TypeScript would spell the three forms as overloads; the checker picksThreadScope's method by the number of arguments,spawnWithfor two andspawnThunkfor one, and under Node every form reachesspawn, which runs a task without a slot at once (RUN_UNDER_NODE.md). -
An arrow task reads its function's
constlocals and parameters, which is the whole spelling of a task that needs more than one value:{ using s = scope(); s.spawn(() => binInto(xs0, hist)); // xs0: a `const` array; hist: a `Mutex` s.spawn(() => binInto(xs1, hist)); }Each name the arrow reads is appended to its parameters, as a data-parallel arrow's is, and copied into the task's payload at the spawn beside its argument; the task reads it on its own thread when the scope joins. A capture is the task's argument to every rule of this section: the parent may not write what it reaches before the join, however it does so — a store, a second name for it, a helper, a
newwhose constructor writes it or keepsthisin it, in a scope bound by any declarator of itsusing(tests/cases/reject_thread_task_capture_parent_write,_alias,_helper,_new,_this_escape,_declarator); a destination is never captured (_dst); the task writes nothing its caller can see except through a guard, a constructor it runs included (_task_write,_task_new,_task_this_escape); aMutexit captures is one the parent may not lock before the join (R7,_mutex_lock) and that it may reach only as a capture or a field of one (R8,The task's arrow reads `o`, which is `Outer` and reaches a `Mutex` through `o.inner.m`: a captured value is the task's argument, …, NL2510,_mutex_deep); the parent'sorReturnbefore the join is refused (NL2394,_or_return); a captured element reference is read at the join, so apushthat may move its storage before the end of the block is NL2290 (_stale_record,_stale_join, andreject_thread_task_arg_stale_joinfor a task's argument, read at the join the same way); and an array moved into aSecretis not read again (NL2445,_secret_moved). Aletis not captured, since it could change before the task runs (NL2506,_let), northis(NL2507,_this; aconst self = thisis captured as the reference it is,tests/link/thread_task_arrow), and a captured name is never assigned (NL2508,_assign). A guard, a channel, a scope, aResult, a value that may be absent and aSecretare never captured:The arrow reads `ch`, which is `Channel<i32>`, and a task cannot capture it: a guard, a channel or a scope is shared only by the rules of its own block; a task reaches a channel through its argument, `s.spawn(entry, arg)`, where the scope counts its senders(NL2509,_guard,_channel,_secret). An arrow inside a task's arrow reads only its own parameters and the module's top-level names (NL2501,_nested). The arrow is lifted into a function named after the one it is written in and a counter (main$arrow0), which is the name a debugger or a profiler shows for the task's thread. -
A task runs when its scope joins, not when it is spawned. Joining a scope runs every task it was given at once — the first on the thread that opened the scope and each other on a thread of its own, so
Ntasks startN - 1threads — waits for all of them, and then stores each answer in the order the tasks were spawned, on the thread that opened the scope. That is what makes a scope free of races: while the tasks run, the thread that opened the scope is waiting for them, the tasks write nothing another can see (below), and every store into the program's memory is made after the last task has finished. So a task may be handed any argument — an array, an object. -
scopeWithStack(bytes)is a scope whose tasks have a stack of at leastbytesbytes, for a task that recurses deeper than a thread's default stack allows (8 MiB on Linux):using s = scopeWithStack(64 * 1024 * 1024)(tests/link/thread_stack_deep, a million calls deep on 256 MiB; under an 8 MiBulimit -sthe same program faults withscope(), and withscopeWithStack(131072)on its guard page,tests/link/thread_stack_fixtures). It covers a scope's tasks only:parallelMap,parallelFromand the other data-parallel calls run on the pool's workers, whose stack is the platform's default. It is a scope in every other rule. Its tasks all run on new threads, a thread each, none on the thread that opened it; each thread's stack isbytesrounded up to the page, with a guard page below it, so a task that still overflows faults rather than writing past it. If one cannot be started the program ends with exit 1 andnish: cannot start a thread for scopeWithStack: out of threads or of memory for its stackwhen the system is out of either, ornish: cannot start a thread with the stack scopeWithStack asked forwhen it refuses the size. The smallest size is 131072 (128 KiB, the largestPTHREAD_STACK_MINamong the targets): a number written at the call below it is refused,`scopeWithStack(4096)` asks for a stack below the minimum of 131072 bytes: no target can start a thread on less, and a task that ran on it would overflow before it did any work; ask for at least 131072, `scopeWithStack(131072)`(NL2584,tests/cases/reject_thread_stack_small,_negative), and a computed one panics when the scope is made,scopeWithStack: a stack of 65536 bytes is below the minimum of 131072and exit 1 (tests/link/thread_stack_computed_small). Under Node, on WASI and wasm32 (no threads: every task runs on the one thread, in spawn order) and in a build without--threads, the size is checked the same way and has no other effect, so a program that needs the stack needs it from Node too (node --stack-size). -
A destination is a fresh
constarray that is never handed on, and until the block ends the scope's thread neither reads a destination nor writes what a task may read. Natively a task runs, and its answer is stored, when the scope joins; under Node each runs and stores where it is spawned. Two rules keep the program's meaning the same either way:-
A
spawn's destination is aconstlocal bound to a literal array ornew Array(...)in the same function, used only by index,.length, its methods, afor...ofand as a destination — never an argument, a task's argument, a stored value or a returned one — so nothing else can reach it. Anything else isThe destination of this `spawn` must be a `const` local bound to a fresh array (a literal or `new Array`), used only by index, `.length`, its methods and as a destination: a scope's tasks run, and store their answers, when its block ends, and under Node each runs and stores where it is spawned, so a destination is a fresh `const` array that is never handed on, and between a scope's first `spawn` and the end of its block its thread neither reads a destination nor writes memory a task may read(tests/cases/reject_thread_region_dst_not_fresh, andreject_thread_region_dst_as_argfor one handed to a later task). -
From the first statement of the block that holds a
spawnon the scope to the block's end — areturn's value aside, computed after the join — the program may not read a destination a task of the scope has been given by then (This reads `sums`, a destination of a task of `s`, before the scope joins: …,tests/cases/reject_thread_region_read), and may not write memory a task could read. A task reads only what its argument reaches, so a store is fine into the own slots of aconstbound to a fresh array or object literal that is never handed on, and a call is fine unless it writes through a pointer it is handed; a function that only fills an array it allocates, such as one that builds and returns a row, writes nothing a task can see. Every other write is refused, whatever alias it goes through — a store,pushorpop,Arena, a call that writes through its argument, a task of another scope, whose join stores (reject_thread_region_write,reject_thread_region_alias_write,reject_thread_region_call_write). A constructor is a call: anewwhose constructor writes through an argument, directly or through a constructor it calls, is`new Bump` writes memory a task of `s` may read, before the scope joins: …(tests/cases/reject_thread_region_new_write, and_declaratorfor the second declarator of a statement,_nestedfor a constructor that calls one,_thisfor one that handsthisto its argument, which is a store through that argument). A write counts however deep it lands in what the argument reaches: a field of a field, a field of an element, through aconstthat names one, afor...ofvariable over one, a helper, or a method of what a field holds (reject_thread_region_new_deep,_new_element,_new_helper,_new_method,_call_deep,_call_element,_call_alias,_call_loop). Its stores into the fresh object itself are not counted, sonew Point(3, 4)is fine there (tests/link/thread_task_private). An object a function or a constructor made itself is its own memory, and a write into it counts against nothing, until memory an argument reaches may have got into it: stored into a field or an element at any depth, pushed, put there by a method or a function it was handed to, kept by its constructor from an argument, or any non-number value rooted at it — the object, a field or an element of it at any depth, a constructor'sthisorthis.f— used as a value anywhere but a read, a comparison or an argument the checker follows into its callee: the right side of an assignment, aconst's initialiser, an element of an array literal, a property of an object literal,{ k: v }or{ k }, an argument ofpush, of aMaporSetmethod or of a call, an arm of a?:. Each position is refused for each form of the value, a local, a field, an element andthis.f(reject_thread_region_escape_<position>_<form>, with positionsassign,alias,array,object,shorthand,push,call,ternaryandelement, formslocal,member,indexandthis_field, and_thisfor the barethisin every position butshorthand, which cannot name it). AMapor aSetkeeps exactly what its methods are handed —set's key and value,add's element,getOrInsert's fromnish/map— so each is a store into the table, as apushis (reject_thread_region_map_value_param,_map_fused_value_param,_map_own_subpath,_map_route_subpath,_map_route_value_param,_set_own_subpath), and its methods write only the table's own storage, never what it holds. What a container gives back is its element: whatgetorgetOrInsertanswers, and theconstvariable of afor...ofover an array, aSet, or aMap'skeys()orvalues(), is memory below that container, so a write through it counts against what was stored into the container, and a store through it is a store into the container (_readback_get,_readback_get_or_insert,_readback_nested,_readback_store_alias,_table_param_store,_walk_array_param,_walk_array_twice,_walk_values,_walk_keys,_walk_set_param,_walk_array_store,_walk_set_store). So a call that answers a table or what one holds —getOrInsert(m, …),m.set(…),s.add(…)— is a value rooted at that table, and escapes it wherever a name would (reject_thread_region_call_escape_<position>_<form>, with positionspush,field,index,arrayandobject, formsgoi,set_chainandadd_chain, and_ternary_goi,_nullish_get,_let_init_goi,_let_assign_goi,_table_arg_set_chain). Only a call that keeps what it is handed stores it —setits key and value,addits element,getOrInsertits key and default — so a lookup,has,getordelete, keeps nothing (reject_thread_region_table_key_set,_table_key_get_or_insert); and a number, a string or abooleanread back through| undefined, asm.get(k) ?? 0, names no memory. A lookup in an argument's table,hasorget, reads it and writes nothing (tests/link/thread_scope_region_param_lookup). A?:or a??stored somewhere may be either of its sides. Two values may be anything where they are written through: anx ?? y, and the variable of a walk of anything else — aChannel's messages are another task's. An object literal whose property is an argument's memory carries it however it is written (reject_thread_region_literal_object_param,_shorthand_param,_array_object_param,_field_object_param,_push_object_param,_new_object_param,_object_const_param,_object_method_param). A write below such a slot counts against the arguments whose memory may be there, or against all of them when that cannot be told (reject_thread_region_new_alias_field,_alias_push,_alias_element,_alias_const,_alias_helper,_alias_method,_alias_item,_call_wrap,_fresh_field,_fresh_method_bind,_fresh_attach,_fresh_push,_fresh_element,_fresh_array_field,_fresh_nested,_fresh_bump_method,_fresh_const,_fresh_ctor_field,_fresh_literal,_fresh_escape,_fresh_held,_fresh_held_temp,_fresh_captured,_this_local,_this_ternary,_this_literal,_this_object). A constructor that keeps what it is handed keeps it per argument, so one that keeps the caller's own array, or pushes an argument's object onto an array it made, writes nothing of the task's (tests/link/thread_scope_region_fresh_object,thread_scope_region_kept_own), and a call that only writes its argument, or returns an object of its own, keeps nothing of the caller's in it:w = build(h)on a function that returns a fresh object, andc.reset(h.nums[0])on a fresh counter followed by writes intoc(thread_scope_region_own_handoff). So aMapor aSeta function makes and fills itself writes nothing of the task's (thread_scope_region_map_set,thread_scope_region_map_get_or_insert,thread_scope_region_set_add), and so does one that walks, memoises or indexes a table of its own, or writes through what it reads back out of a container of fresh objects it made (thread_scope_region_own_tables,thread_scope_region_read_back). A map handed tosetas a namedconstescapes, as any own object handed to a table does, so a write into it counts against every argument: build a nested table from whatgetreads back instead. A task argument read out of a variable —s.spawn(f, ps[0], out, 0), or either arm of a?:or??— hands that variable on, so a store into its elements or fields before the join is refused too: a record element is the address of its slot, and the task reads it at the join (tests/cases/cg_sec_scope_record_arg). Andr.orReturn()on aResultis refused in the same stretch, between the firstspawnand the end of the block, because its early return would leave the scope unjoined (NL2394,tests/cases/cg_sec_scope_or_return).
The rules are local because memory is reached only through parameters and what a function allocates: a module constant is a number, a
booleanor a string, and nothing isstatic. N tasks over N arrays of the destination's own type, built before the scope, in the spawning loop or after the first spawn, all compile (tests/link/thread_scope_many_arrays). -
A
-
The join is emitted at every exit of the block: its end, a
return— before the returned value is computed, so the value can read what the tasks stored — and abreakorcontinuethat leaves the block (tests/link/thread_scope_exit_paths). It sits inside every arena scope around it, which releases only after the join, and an argument a task is handed is never given back by a loop's pass before the join reads it (tests/link/thread_scope_nested_arena, which also nests one scope inside another). Since the join comes first, spawning gives up no memory: a function, a loop's pass or ausing a = arena()block that hands a task an array it made still gives that array back when it ends, after the join, unless the task keeps it — answers it, or stores it where it outlives the task — or the frame keeps it past its end by any other route (tests/link/thread_scope_reclaim, whose peak resident memorytests/run.jsbounds at every thread count, andthread_scope_reclaim_escapes, which reads each route after the memory is reused). A panic ends the process and joins nothing (throwis refused by Phase 0,reject_throw). -
A scope is introduced by
using, andusingtakes only a scope, aMutex's lock or anarena(). A scope bound any other way is`scope()` must be the initialiser of a `using` declaration: a scope joins its tasks when the block that declares it ends, so a scope bound any other way would be one nobody joins; declare it as `using s = scope()`(tests/cases/reject_thread_scope_not_using), and ausingof anything else is`using` takes only `scope()` or a `Mutex`'s `lock()` from `nish/threads`, or the builtin `arena()`, in this version: they are the values whose disposal the language defines — a scope joins its tasks, a guard releases its lock, an arena releases what its block allocated — so a `using` of anything else would promise a disposal nothing performs(tests/cases/reject_thread_using_not_scope). Ausingdeclaration is a statement of a block; as the body of anifor a loop it isA `using` declaration must be a statement of a block, `{ ... }`: its scope joins when that block ends, and a single-statement body is not a block(tests/cases/reject_thread_using_placement), and in acaseclause it is refused as every declaration there is. Outside a declarationusingis an ordinary identifier, as in TypeScript (tests/parser/using.ts). -
A scope never leaves its block. It is only ever the receiver of a
spawnstatement; passed, stored, returned or copied it isA `ThreadScope` can only be the receiver of a `spawn` statement: passed, stored, returned or copied, it could be given a task after its block has joined it(tests/cases/reject_thread_scope_escapes). That, and the join at every exit, is the whole proof that every task is joined: there is no handle to forget. -
[Symbol.dispose]isnish/threads's alone.ThreadScopeandMutexGuarddeclare one, so TypeScript and Node acceptusing s = scope()andusing g = m.lock(); the compiler emits the join and the release itself and never calls it.Channel's[Symbol.iterator], Node's way into afor...of, is the one other computed name, and isnish/threads's alone too: anywhere else it is refused as every computed name is (NL1041). Any other class's is`Handle` cannot declare `[Symbol.dispose]`: in this version `using` takes only `scope()` or a `Mutex`'s `lock()` from `nish/threads`, or the builtin `arena()`, whose disposal the compiler emits itself, so a disposal method of any other class would never be called(tests/cases/reject_thread_dispose_elsewhere). -
There is no thread count. Each spawn is one task and each task one thread; a scope with more tasks than cores leaves the scheduler to share them (wp29-thread-surface.md §9 declines the knob until a measurement asks for it).
What a task may do is what a data-parallel body may, less the rules about an
element's arena, which a task does not share, and each rule is reported at the
spawn:
-
The task writes nothing another can see, except through the guard of a
Mutex, the one shared write a task may make, by a method of an atomic (tests/link/thread_atomic_tasks), or by asendor a receive on aChannel, neither of which is a write the rule counts:`bump` writes memory its caller can see at main.ts:10:3, and `spawn` runs it beside the scope's other tasks: make scratch only this task uses inside it, because an object handed in as the argument is shared memory even when the `spawn` made it, since where an argument was made is not tracked. Write memory the tasks share only through the guard of a `Mutex`; hand a value the parent needs back as the task's answer, or run it on the thread that called it, `out[0] = bump(c)`(NL2471;tests/cases/reject_thread_task_shared_write, andreject_mutex_task_unguardedfor a task that also writes through a guard). Awipeis a write through its argument like any other, so it is shared exactly when its target is: an array the task, or a function it calls, allocated itself is private, and wiping it is allowed (tests/link/thread_task_wipe). So is aSecretthe function makes withsecret(v)and wipes, whenvis allocated in the call —new, an array literal or an object literal — and theSecretis held by aconst(tests/link/thread_task_wipe_secret). Wiping the task's argument, a captured local, a field, an element, an array a call answered, aSecretheld by alet, made from a call's answer, from a local moved into it or from a record whose constructor is handed an object, aSecreta call answers (wiped, or written by a helper first), or one a holder keeps in a field, is NL2471 (reject_thread_wipe_param,_capture,_field,_element,_helper,_call_result,reject_thread_wipe_secret_call_result,_moved,_holder,_let,_call_secret,_call_secret_helper,_ctor_param). A helper may write the task's ownSecretbefore the wipe. A borrowedSecretparameter is refused before it (reject_thread_wipe_secret_borrowed). The wipe itself is unchanged, one volatilellvm.memset. A lookup in a sharedMaporSet—get,has,size— is a read, so tasks of one scope may look keys up in the same table (tests/link/thread_scope_task_map_reads). A mutator —set,add,delete,clear,getOrInsert— writes it, a walk counts itself into it (walkOpen), and what a lookup answers is the table's memory, so each is refused (reject_thread_task_map_set,_map_delete,_map_clear,_map_get_or_insert,_set_add,_set_delete,_set_clear,_map_walk_keys,_map_walk_values,_set_walk,_map_readback_write). A task maypanic, as a parallel body may: the first panic of the program prints and exits 1, and a task that panics after it prints nothing. Output stays a write (reject_thread_task_console_error,reject_thread_task_process_exit), and so does the parent's: between the firstspawnand the join it may neither change aMaporSetit holds nor walk one —for (const k of s),m.keys(),m.values(), or a helper that does (reject_thread_region_parent_table_set,_set_walk,_map_keys_walk,_map_values_walk,_helper_walk). The rule is conservative: it judges the call by what it writes through its receiver, not by which tables a task holds, so a walk of a table no task can reach is refused too (reject_thread_region_parent_unreached_walk), and so is a mutator or a walk called straight on a table made in the block. Proving that no task reaches one table would mean following every pointer each task is handed; the walk moves before the firstspawn, after the block, or into a helper that makes and fills its own table (thread_scope_region_own_tables). A store into an array the task allocated and hands to nothing that keeps it is not counted, as for a parallel body: a sieve may fill its own table, and a helper may fill the array it is handed (tests/link/thread_task_private). Returned, stored into its argument's field, pushed onto its argument's array, kept by a helper or a constructor, or bound to a second name in a later declarator, the array is shared (tests/cases/reject_thread_task_private_returned,_field,_pushed,_kept,_new,_declarator). A constructor that handsthisout can lead to the array only by keeping it, so that route is_new's. A store through a?:with the array in one arm is a store into either (_ternary), and an element of a private array of arrays is the array it holds (reject_thread_task_private_nested). The same holds for the task's other own memory: a method that writes only the fields of an object the task made,const h = new Holder(); h.poke();set,addanddeleteon aMapor aSetit made; a store into an array field of an object it made, whose constructor filled it with a fresh array; a second name for its array bound in the same statement,xs = new Array<i32>(n), ys = xs; and a store through what a helper answers when the helper answers its argument, the task's own array, all write nothing shared (tests/link/thread_task_own_writes). What the argument reaches is not the task's own, even when thespawnmade it there, because where an argument was made is not tracked: an object the task makes that holds the argument's array from its constructor on, has a field re-pointed at it, or is handed it by a helper or a method, and a helper that may answer either its own array or the caller's, all write the caller's memory (reject_thread_task_own_holds_param,_repointed,_helper_stores_param,_method_adopts_param,_either_returned), and so does a second name bound in the same statement as a fresh array when it names the argument's (reject_thread_task_own_declarator_binds_arg). A helper's fresh answer that holds the caller's memory,[ys]ornew H(ys), is the task's at its top level only, so a write below it is refused (reject_thread_task_own_fresh_holds_array,_new,_object). That refusal starts with what to write:make scratch only this task uses inside it, because an object handed in as the argument is shared memory even when the `spawn` made it, since where an argument was made is not tracked. Write memory the tasks share only through the guard of a `Mutex`; hand a value the parent needs back as the task's answer, or run it on the thread that called it, `out[0] = bump(c)`. A task inside a task, or inside a data-parallel body, is refused by this rule, because a scope's join stores into memory (tests/cases/reject_thread_task_nested_spawn); and a data-parallel call inside a task is refused by it too (reject_thread_task_nested_map). -
It answers a number, a
boolean, an enum, astring, or an array of numbers, booleans or enums, because a task's thread frees its arena when it exits. A string or a scalar array crosses by copy-at-join: the worker copies its answer into amalloc'd transfer block before its arena goes (nish_cross_out), and the parent, once the scope has joined, copies each block into its own arena and frees it (nish_cross_in), so the block lives only between the two and is freed on every path out of the scope. An answer the task did not allocate — a literal, or a string its argument already held — passes through uncopied (tests/link/thread_task_string: a built label, a field of a made object, a substring and a slice, a returned fresh array, and a task that also stores into aMutex, all read afterchurn()). Anything else is`words` answers `string[]`, and `spawn` hands back only a number, a `boolean`, an enum, a `string` or an array of numbers, booleans or enums, which the join copies out of the worker's arena: anything else would point into memory the worker frees or reuses(NL2348,tests/cases/reject_thread_task_result_strings;reject_thread_task_result_typefor a class,reject_thread_task_result_nestedfori32[][]andreject_thread_task_result_mapfor aMap). What a task allocates on the way is freed with its thread. -
It leaves the arena alone, because the first task runs in the arena of
the thread that opened the scope:
`wipe` reads or moves the arena, and `spawn` runs it on several threads that each have an arena of their own: a parallel body may not call `Arena.mark` or `Arena.used`, nor open a `using a = arena()` block, nor call `region`, `regionWith`, `child`, `childWith`, `enter`, `enterWith` or `free` (`nish:region`)(tests/cases/reject_thread_task_arena). -
Its argument is not a
Resultsmall enough to travel as its parts:The argument of `spawn` is `Result<i32, i32>`, which a task cannot be handed: a `Result` is passed as its parts rather than as one value, so hand the task the value it holds(tests/cases/reject_thread_task_result_arg).
Under Node, scope() is an object whose spawn runs its task at once and
stores the answer there; the rule above is what makes that print the same as
the compiled program (RUN_UNDER_NODE.md). using needs
--js-explicit-resource-management on Node 22 and is native from Node 24.
parallelFor, non-scalar results and scope(n) are not in this stage
(wp29-thread-surface.md §7); the lock is the next
section, atomics the one after it, and
channels the one after that.
A lock that owns its data: Mutex<T>
A Mutex lets a scope's tasks share one value they all update: the data lives
inside the lock, and the only way to it is the guard lock() answers, bound by
using, which holds the lock until its block ends
(wp29-thread-surface.md §4.3). It is Rust's
Mutex<T>, narrowed until no borrow checker is needed.
import { Mutex, scope } from "nish/threads";
class Counter {
n: i32 = 0;
}
const addMany = (total: Mutex<Counter>): i32 => {
for (let k: i32 = 0; k < 100000; k++) {
using g = total.lock(); // waits for the lock, and holds it to the block's end
g.value.n = g.value.n + 1;
} // released here, on every pass
return 0;
};
export const main = (): i32 => {
const total = new Mutex<Counter>(new Counter());
const done: i32[] = [0, 0, 0, 0];
{
using s = scope();
for (let t: i32 = 0; t < 4; t++) {
s.spawn(addMany, total, done, t);
}
}
using g = total.lock();
console.log(`${g.value.n}`); // 400000
return 0;
};
-
Natively the lock is one word inside the guard, taken with an acquire
compare-and-swap in
lock()and given back with a release store, so a lock nobody holds costs one atomic instruction and allocates nothing; alock()that finds the word held calls the runtime'snish_mutex_wait, which spins briefly and then yields until it can take it (tests/link/thread_mutex_counter, whosethreads.llis the lowering, andtests/link/thread_mutex_histogram). Onwasm32a scope's tasks run one after another, as they do for every scope there, so no lock is ever found held: the same program links for--profile wasi, its atomics lowered for a module without shared memory, and prints the same total (tests/run.js, "the--profile wasibuild"). -
The release is emitted at every exit of the guard's block: its end, a
breakorcontinuethat leaves it, and areturnor anorReturn— after the returned value is computed and before control leaves, unlike a scope's join, which runs before the value, soreturn g.value.nreads under the lock (tests/link/thread_mutex_or_return, whosemain.llpins the order) — and inside anarena()block it is given back before the arena releases (tests/link/thread_mutex_exit_paths, which locks again after each exit and would wait on itself if one kept the lock).
Eight rules keep every path to the data behind the lock, and a program that breaks one is refused with that rule's own code:
-
R1: the data is fresh all the way down, and only the guard reaches it.
new Mutex<T>(init)takes a scalar, or anewexpression or a literal whose every argument, element or value is a number, aboolean, an enum, a string literal — immutable, and never stored over — or itself fresh by this rule, because a value the program already holds would be a path to the data that skips the lock:`new Mutex` takes a value that is fresh all the way down — a `new` expression or a literal whose every argument or element is a number, a `boolean`, an enum, a string literal or itself fresh — because the data a `Mutex` owns is reachable only through its guard, and a value the program already holds would be a path to it that skips the lock(NL2463,tests/cases/reject_mutex_not_fresh,reject_mutex_not_fresh_deepfornew Hist(bins)with a namedbins, andreject_mutex_not_fresh_literalfor[row]with a namedrow;new Mutex<Label>(new Label("total", 0))compiles,tests/link/thread_mutex_scalar). Andlockis the only member of aMutexa program may name:`guard` is not a member a program may name: the data a `Mutex` owns is reachable only through the guard its `lock()` answers, `using g = m.lock()`, so every path to it takes the lock(NL2464,reject_mutex_member). -
R2:
m.lock()is only the initialiser of ausingdeclaration, the third disposable besidescope()andarena(), and oneusingtakes one lock (a second declarator'slock()is R5's, below). A lock may share ausingwith a scope opened before it or with an arena on either side, and the declarators close in the reverse of the order they opened, each opening before the next declarator's initialiser runs (anorReturn()there closes the ones before it), so inusing a = arena(), g = m.lock()the lock is given back before the arena releases (tests/link/thread_mutex_using_mixed); a scope opened after the lock,using g = m.lock(), s = scope(), is R5's (reject_mutex_using_lock_then_scope). Alock()anywhere but ausinginitialiser is`lock()` must be the initialiser of a `using` declaration: the lock is released when the block that declares it ends, so a guard bound any other way would be one nobody releases; declare it as `using g = m.lock()`(NL2465,reject_mutex_lock_not_using;reject_mutex_lock_expr_bodyfor an arrow whose expression body ism.lock()). -
R3: a guard is used only as
g.value, and what it reaches only as the base of another access. Bound, stored, returned or walked, a guard isA guard is used only as `g.value`, and `g.value` only as the base of a field or element read or store: passed, bound, stored, returned or walked, what it reaches would outlive the block that holds the lock; write `g.value.n` where the value is used, or hand `g` to a helper whose parameter is a `MutexGuard<Counter>`(NL2466,reject_mutex_guard_helper_return,reject_mutex_guard_helper_field,reject_mutex_guard_helper_ternary). The one place a guard may go is to a guard helper: a function named at the call, neither a method, a constructor nor a generic, whose parameter in that position is declared as the guard is,MutexGuard<T>, with the guard named once among the call's arguments:const bump = (g: MutexGuard<Counter>): void => { g.value.n = g.value.n + 1; }; { using g = m.lock(); bump(g); }A guard helper is named, never held: naming it as a value is
`bump` takes a `MutexGuard`: a guard is handed only to a function named at the call, so a function reference cannot name it; call the helper by name(NL2739,reject_mutex_guard_helper_ternary_alias,reject_mutex_guard_helper_nullish), because the rules below follow the function a call names.The helper's parameter is held to R3–R5 in its body exactly as
gis in the block, so it can only read and store through it, and R5 follows the call, so a helper that locks, opens a scope, spawns or receives is refused at the call (reject_mutex_guard_helper_spawn,reject_mutex_guard_helper_receive). A helper that passes the guard on is judged by the same rule at that call (reject_mutex_guard_helper_forward), through helpers that call each other (reject_mutex_guard_helper_mutual) and across an import (link/reject_mutex_guard_helper_import); a helper that locks aMutexit is handed is refused by R5 at the call (reject_mutex_guard_helper_second_lock), andg.valuehanded beside the guard by NL2467 (reject_mutex_guard_helper_value_arg). A helper has no default, rest or destructured parameter, which Nish has none of (reject_mutex_guard_helper_default,_rest,_destructured).link/thread_mutex_helperis three helpers deep on four threads. Any other call is`g` is passed to `push`, a method: a guard is handed only to a function named at the call, neither a method, a constructor nor a generic, whose parameter is declared as the guard is, `MutexGuard<T>`, once per call, because that parameter is held to the rules of the guard's block; write the helper as `const useGuard = (g: MutexGuard<Counter>): void => { console.log(g.value.n) }` and call it as `useGuard(g)`(NL2536;reject_mutex_guard_helper_array,_map,_method,_new,_ctor_this,_generic,_twice,_nullable, and_callbackfor a helper that takes a function, which makes it a generic), because a method or a constructor hasthisas a second way out, a generic's body treatsTas any value, and a guard passed twice is two names for one lock. Anything read through it that is not a number, aboolean, an enum or astring—const c = g.value,g.value.xsas a value, afor...ofover it — isThis reads `Counter` through a guard, whose chains are only the base of a further field or element read or store: a value that is not a number, a `boolean`, an enum or a `string`, which the read copies, would outlive the block that holds the lock; do the work inside the guard's block, or read the numbers and strings it holds one at a time, `const n = g.value.n`(NL2467,reject_mutex_guard_alias;reject_mutex_guard_alias_arrayforconst b = g.value.bins, which would race a guarded write once the block ends;reject_mutex_guard_returnandreject_mutex_guard_return_arrayfor a return;reject_mutex_guard_store_fieldandreject_mutex_guard_store_elementfor a store;reject_mutex_guard_for_offor a walk). An arrow's expression body is its answer and is judged as areturnof it is, so a helper written(g: MutexGuard<Box>): Box => g.valueis refused as one that writesreturn g.value(reject_mutex_guard_helper_expr_value,_field,_paren,_index,link/reject_mutex_guard_helper_expr_import;_call,_newand_condforg.valuehanded to a call or anew, or written as an arm of? :, inside the body), andreject_mutex_guard_helper_expr_churnis the use-after-free it would open: the object kept past the lock, its string replaced under a second lock, andchurn()reusing the memory before the old one is printed. An arrow argument, which may read its function'sconstlocals, may not read a guard among them, whether it runs on this thread (reject_mutex_guard_capture) or is passed to a data-parallel form (reject_par_capture_guard). A scalar read through the guard is a value like any other,.lengthof an array it reaches included. WhenTis itself a scalar,g.valueis read and stored directly —g.value = g.value + 1on aMutex<i32>— and the store reaches theMutexunder Node as natively, because the guard is theMutex's own storage rather than a copy (tests/link/thread_mutex_scalar). -
R4: a store through a guard stores a scalar or a string, and no method is called through it. A number, a
boolean, an enum or astring, into a field or an element of an array that already exists:This stores `string[]` through a guard, which stores only a number, a `boolean`, an enum or a `string`, which the store copies, into a field or an element that already exists: anything else is a pointer, which may point into a task's arena, freed when the task is joined; build the value into the one the `Mutex` is made with, or store what it holds one at a time into slots that already exist, `g.value.names[i] = v[i]`(NL2468,reject_mutex_guard_store;reject_mutex_guard_store_classfor a class). Anullis stored through a guard only into astring | nullslot — a field, an element or aMutex<string | null>— where the store frees the copy the slot held and keeps nothing, so a later read through?.answersnull(tests/link/thread_mutex_null_store, natively and under Node, read back afterchurn()). Into any other nullable slot, aLink | nullfield, element orMutex<Link | null>, it is refused, since such a slot holds a pointer, which a guard stores in no form:This stores `null` through a guard into a slot that holds an object, and a guard stores only a number, a `boolean`, an enum or a `string`, and a `null` only into a `string | null` slot, where the store frees the copy the slot held; keep a `boolean` field beside the slot, `g.value.hasNext = false`, and test it before reading the object(NL2537,reject_mutex_guard_null_class,_link_element,_link_value,_link_nestedforg.value.inner.next = null, and_link_parenforg.value.next = (null)). Only the literalnullis stored: astring | nullvalue, a variable or a??, is refused as NL2468 above, since the store copies what it is given and a value that may benullwould need a test the store does not make. A string stored through a guard is copied into memory the mutex owns:nish_guard_keepmallocs the copy, records it, and frees the copy the slot held before, if that one was its own; a string read through a guard is copied out into the reader's arena (.lengthexcepted, which keeps nothing), so no reader holds the block the next store frees. The block a slot holds when theMutexitself is dropped is not freed — the arena that held theMutexdoes not say when it goes — and is reported by valgrind as still reachable, one block per guarded string slot (tests/link/thread_mutex_string). A program that makes and drops many suchMutexes keeps all of them: 100,000, each holding one 850-byte string, peaked at 89 MB against 11 MB for the same loop without the store. The record of those copies is one for the process, behind a lock of its own, so stores through two differentMutexes wait on each other while it is held.push— which can move the array into the task's arena — and every other method are`push` is called through a guard, which reads and stores fields and elements and calls no method: a method could keep what it reaches, or grow an array into a task's arena, which is freed when it is joined; write what it does as reads and stores through the guard, into slots that already exist, `g.value.xs[0] = 1`(NL2469,reject_mutex_guard_push). So aMutex<i32[]>or a class with ani32[]field holds a histogram, sized when it is made. -
R5: one lock at a time, and nothing waited on while it is held. Inside a guard's block — directly, or through any function it calls — a second
lock(),scope(),spawn,parallelMapIntoorparallelReduceisA second `lock()` is reached while `ga` holds its lock: inside a guard's block, directly or through any function it calls, there is no second `lock()`, no `scope()` or `spawn`, and no `parallelMapInto` or `parallelReduce`, so one lock is held at a time and nothing is waited on while it is held; move `b.lock()` below the end of the block that opens with `using ga = a.lock()`, which releases the lock(NL2470;reject_mutex_nested_lock, andreject_mutex_using_two_locksandreject_mutex_using_two_mutexesfor a second declarator of the sameusing;reject_mutex_lock_through_callfor a callee that locks, andreject_mutex_held_newfor a constructor that does, reached bynew;reject_mutex_held_scopeandreject_mutex_held_scope_callfor a scope;reject_mutex_held_mapandreject_mutex_held_reducefor the data-parallel calls). That is the whole argument that a program cannot deadlock: a task holding a lock never waits for anything, so every lock is given back. -
R6: a store through a guard is the one shared write a task may make. P2's rule is otherwise unchanged (above, NL2471).
-
R7: from the statement that holds a scope's first
spawnto the end of its block, the parent does not lock what the scope's tasks share, directly or through any function or constructor it calls — P2's own window, so a lock before thespawnin the same loop body is inside it: on the loop's second pass that lock runs after the first pass'sspawn(reject_mutex_region_lock_loop). Natively the tasks have not run yet there, and under Node each has run at its spawn, so the parent would see a different value:This `lock()` locks a `Mutex` a task of `s` can reach through `arg`, before the scope joins: natively the scope's tasks run when its block ends and under Node each runs where it is spawned, so the parent locks what they share only before the statement that holds the first `spawn` or after the join; move `m.lock()` above the block that opens with `using s = scope()`, or below the end of that block(NL2472,reject_mutex_region_lock;reject_mutex_region_lock_fieldfor aMutexthe task reaches as a field of its argument;reject_mutex_region_lock_callfor a call that takes the lock, andreject_mutex_region_lock_newfor anewwhose constructor does). It is asked by type: aMutex<T>of the type a task's argument reaches, or, for a call that takes a lock, anyMutexa task reaches. Before the firstspawnand after the join the parent locks freely, astests/link/thread_mutex_counterdoes. -
R8: a
Mutexreaches a task as its argument, or as a field of the class its argument is. An array of them, a deeper field, or aMutexinside another's data isThe argument of `spawn` is `Mutex<Counter>[]`, which reaches a `Mutex` through `ms[i]`: a `Mutex` reaches a task as its argument or as a field of the class its argument is, never deeper, so which tasks share a lock is a question the checker can answer; hand the task the `Mutex` itself, `s.spawn(first, ms[i], done, 0)`, and declare its parameter a `Mutex` of the same data(NL2473,reject_mutex_task_array;reject_mutex_task_nestedfor aMutexinside another's data,reject_mutex_task_deepfor one two classes deep,reject_mutex_task_nullablefor a nullable carrier,reject_mutex_task_mapfor aMapholding them). A task's destination holds numbers, so aMutexis never one.
And a lock is a scope's: the body of parallelMapInto or parallelReduce,
which runs on many threads at once, takes none
(`tally` takes a `Mutex`'s lock, and `parallelMapInto` runs it on several threads at once: a lock is taken by a scope's task or by the thread that opened the scope, never by a parallel body; run it as a scope's tasks instead, `{ using s2 = scope(); for (let i = 0; i < src.length; i++) { s2.spawn(tally, src[i], dst, i) } }`,
NL2474, reject_mutex_parallel_body; reject_mutex_parallel_body_call and
reject_mutex_parallel_body_new for a lock taken in a callee or a
constructor).
What a guarded program means. The tasks' critical sections run one at a
time, in some order. Natively the order is the scheduler's; under Node, where
lock() answers the guard at once and its [Symbol.dispose] does nothing, it
is spawn order, which is one of the native orders
(RUN_UNDER_NODE.md). A program prints the same both ways
when its guarded updates commute exactly — integer sums and counts, a minimum,
a histogram; an f64 sum does not, because the order changes its bits — and
no task's answer or control flow depends on guarded state it read. Anything
else is its author's obligation, as an associative operator is for a named
function given to parallelReduce; the checker does not try to prove it. It
is the one place the lock weakens the scope's promise above, that the checker
refuses anything that could tell the two orders apart.
Atomics: AtomicI32 and AtomicI64
An atomic is one integer a scope's tasks, and the bodies of the data-parallel
forms, update at once without a lock: each method is a single indivisible
operation on the word, so no update is lost to a race and none waits for
another. Where a Mutex guards a whole value, an atomic guards one counter.
The fast way to use one is the example's: each task counts into a local tally
no other core touches and adds it to the shared counters once, at its end, so
the atomics see one update per bin per task rather than one per value
("Measured", below).
import { AtomicI32, parallelMapInto, scope } from "nish/threads";
class Shard {
xs: i32[];
bins: AtomicI32[];
constructor(xs: i32[], bins: AtomicI32[]) {
this.xs = xs;
this.bins = bins;
}
}
const binShard = (s: Shard): i32 => {
const local: i32[] = [0, 0, 0, 0]; // this task's own counts: no core shares them
for (const x of s.xs) {
const bin: i32 = x & 3;
if (bin < local.length) {
local[bin] += 1;
}
}
for (let b = 0; b < local.length; b++) {
s.bins[b].add(local[b]); // one indivisible add per bin, no lock
}
return 0;
};
export const main = (): i32 => {
const bins: AtomicI32[] = [new AtomicI32(0), new AtomicI32(0), new AtomicI32(0), new AtomicI32(0)];
const done: i32[] = [0, 0];
{
using s = scope();
s.spawn(binShard, new Shard([1, 2, 3, 4, 5], bins), done, 0);
s.spawn(binShard, new Shard([6, 7, 8], bins), done, 1);
}
const big = new AtomicI32(0);
const xs: i32[] = [3, 9, 4];
const ys: i32[] = [0, 0, 0];
parallelMapInto(xs, ys, (x) => big.max(x)); // the one shared write a body may make
console.log(`${bins[0].load()} ${bins[1].load()} ${big.load()}`); // 2 2 9
return 0;
};
-
The methods.
load()andstore(v);add(n),sub(n),min(v),max(v)andexchange(v), each answering the value the word held before it; andcompareExchange(expected, desired), which storesdesiredwhen the word holdsexpectedand answers the value it saw either way, so a loop that retries until the answer is itsexpectedbuilds any other update (tests/link/thread_atomic_tasks).addandsubwrap at the width;minandmaxcompare signed.AtomicI64is the same overi64. -
Natively each method is one instruction, sequentially consistent: a
load atomicorstore atomic ... seq_cst, anatomicrmw add,sub,min,maxorxchg ... seq_cst, or acmpxchg ... seq_cst seq_cst, aligned to the width (align 4,align 8), the whole body of the method's instance innish/threads's module (tests/link/thread_atomic_tasks, whosethreads.llis the lowering; docs/IR_COOKBOOK.md, "an atomic"). Onwasm32a scope's tasks and a parallel call's chunks run one after another, and the same IR is lowered for a module without shared memory. -
Under Node each method is the plain operation on an ordinary field,
which is exact because Node runs every task at its
spawnand every parallel body in order. Node holds anAtomicI64as anumber, and it agrees with native while values stay within ±2^53; wrapping at 2^63 is native-only, the same limit as everyi64(RUN_UNDER_NODE.md), so a program meant to run under Node as well passes iti64-typed values, nottoI64of a number, which is a BigInt there.
Three rules keep every touch of the word indivisible and every program's answer the same natively and under Node:
-
R1: an atomic is reached only by calling its methods. Its word is a field,
value, that no program may name, read or store:`value` is not a member a program may name: an atomic is read and updated only by calling its methods, `load()`, `add(n)` and the rest, each one indivisible: a plain read or store of its word would race with the update a task makes at the same moment(NL2514,tests/cases/reject_atomic_member_read,reject_atomic_member_store, andreject_atomic_member_nestedforh.a.b.value). A method named without a call is refused by the rule every class's methods keep, not by NL2514:Unknown field `add` on class `AtomicI64` (it is a method; call it)(reject_atomic_member_unbound).new AtomicI32(v)takes a number, so an atomic is fresh when it is made, andnewis the only way one is made. An object literal of any class ofnish/threads— an atomic, aMutex, itsMutexGuard, aChannelor aThreadScope— would be a plain record under Node, without the methods, and is refused with`AtomicI32` is a class of `nish/threads`, and its objects are made only as the module makes them: `new AtomicI32(v)`, `new AtomicI64(v)`, `new AtomicI32Array(n)`, `new AtomicI64Array(n)`, `new Mutex(v)`, `new Channel()`, `m.lock()` and `scope()`. An object literal would be a plain record under Node, with none of the methods the class is used through(NL2515,tests/cases/reject_atomic_literal,reject_atomic_literal_i64,reject_thread_literal_channel,reject_thread_literal_mutexandreject_thread_literal_mutex_only, whoseMutexliteral is refused by this rule rather than bynew Mutex's fresh-value one, andreject_thread_literal_scope). -
R7: from the statement that holds a scope's first
spawnto the end of its block, the parent does not touch an atomic the scope's tasks can reach — with a method, or through any function, method or constructor it calls, or by a data-parallel call that does —Mutex's R7, for the same reason: natively the tasks have not run there, and under Node each has run at its spawn, so aload()would answer differently:This `load()` reaches an atomic a task of `s` can reach through `arg`, before the scope joins: natively the scope's tasks run when its block ends and under Node each runs where it is spawned, so the parent touches what they share only before the statement that holds the first `spawn` or after the join; move `a.load()` above the block that opens with `using s = scope()`, or below the end of that block(NL2513,reject_atomic_region_direct;reject_atomic_region_capturefor an arrow task that captures it;reject_atomic_region_field,reject_atomic_region_array,reject_atomic_region_mapandreject_atomic_region_genericfor one the task's argument reaches through a field, an element, aMap's value or a generic's field;reject_atomic_region_call,reject_atomic_region_methodandreject_atomic_region_newfor a function, a method and a constructor that touch it;reject_atomic_region_thisfor one reached throughthishanded out of a constructor;reject_atomic_region_declaratorsfor the second declarator of a declaration;reject_atomic_region_spawn_argfor a secondspawn's argument;reject_atomic_region_parallelfor a parallel call). It is asked by type, asMutex's is: an atomic of the type a task's argument or capture reaches, at any depth, so an alias through?:or a function's answer is the same atomic (reject_atomic_region_ternary,reject_atomic_region_return), and so is any other atomic of that type the task cannot in fact reach. An atomic of a type no task reaches is the parent's alone, and its methods are not counted as writes the scope's tasks may read (tests/link/thread_atomic_unreached). Before the firstspawnand after the join the parent touches it freely.The scopes of one
usingare judged against each other too: natively they join in reverse order, so a task oftinusing s = scope(), t = scope()runs before every task ofs, and under Node each runs at its spawn. Aspawnon one of them whose task shares a destination, an atomic or a lock with a task of another scope of the same statement is refused, naming the other scope and the path it shares:This `spawn` on `t` hands its task an atomic through `a`, which a task of `s` also reaches: the scopes one `using` statement opens join one after the other natively, the last declared first, while under Node each task runs where it is spawned, so the tasks of sibling scopes share no lock, no atomic and no destination; open each scope in a block of its own, so the first joins before the second opens, `{ using s = scope(); s.spawn(() => { a.add(1); }) } { using t = scope(); t.spawn(() => { b.store(a.load()); }) }`(NL2516). A destination is shared when two scopes' spawns name the same array, at the same slot or at different ones (reject_thread_sibling_destination,_slots,_arrowfor arrow tasks,_threefor three declarators). A task that reaches another scope's destination — to read the answer that scope's task returns, or to write it — hands the array on, which the destination rule refuses before this one:_voidwrites it,_readreads it through an arrow's capture,_newthrough a constructor reached bynew, and_thisthrough a field of an object whose constructor handsthisout. An atomic or a lock is told apart by theconstit was made in bynew: a task handed that binding, as its argument or a capture, reaches that object and no other, and two such bindings are two objects. Reached any other way — through another binding, a field, an element, a function's answer — it is read as any object of its type, which meets every one (reject_atomic_sibling_load,reject_atomic_sibling_exchange,reject_atomic_sibling_arrow,reject_atomic_sibling_three,reject_thread_mutex_sibling,reject_thread_mutex_sibling_aliasfor a second name of one lock,reject_thread_mutex_sibling_fieldfor one reached through a field). Every way a task reaches the object counts: through a field of a field (reject_atomic_sibling_field_field,reject_thread_mutex_sibling_field_field), through an object whose constructor handsthisout (reject_atomic_sibling_this,reject_thread_mutex_sibling_this), in a constructor reached bynew(reject_atomic_sibling_new,reject_thread_mutex_sibling_new), in a helper the task calls (reject_atomic_sibling_helper,reject_thread_mutex_sibling_helper), from an arrow's capture (reject_thread_mutex_sibling_arrow), and between any two declarators of the statement (reject_thread_mutex_sibling_three). Tasks that take no lock share none, and tasks that call no atomic method share no atomic. Sibling scopes whose tasks reach disjoint locks or atomics, of one type or of two, run as before (tests/link/thread_sibling_disjoint_mutex,_mutex_types,_atomics). -
R8: an atomic reaches a task at any depth — as its argument, a field of it, an element of an
AtomicI32[], or a capture — unlike aMutex, which stops one field down: an atomic holds no data a task could keep, so which tasks share it is a question R7 answers by type, without the exact sharing set a lock's R5 needs.
An atomic update is the one shared write a task or a parallel body may make
beside a guard. A task that calls add on an atomic its argument reaches
writes nothing the rule above counts, and neither does the body of
parallelMapInto, parallelMap, parallelFrom or parallelReduce, which
takes no lock but may update an atomic it captures or an element reaches
(tests/link/par_atomic). Any other write beside it is refused as before
(reject_atomic_par_shared_write). A parallel body answers a number, never
an atomic (reject_atomic_par_result).
What an atomic program means. The updates land in some order: natively
the scheduler's, under Node spawn order and element order. A program prints
the same both ways when its updates commute exactly — add and sub, min,
max, a count, a histogram — and no task's answer or control flow depends on
a value it read from a word another task updates. A store, an exchange or a
compareExchange that wins against another task is order-dependent. A
parallelReduce's blocks depend on n alone, the same on one core or sixty
four, but natively they run at once and under Node one after another, so what
a body does to an atomic must not depend on which block reaches it first. That is the
author's obligation, as it is for a Mutex's critical sections; the checker
does not try to prove it.
Measured. examples/histogram-atomic.ts bins 4 × 2,000,000 values per
round into one AtomicI32Array of 256 bins, four rounds, on four tasks. On a
four-core machine, over seven interleaved rounds whose seeds move every round,
the medians are: each task counting into a local tally and adding it with one
add per bin, 29 ms (a byte-identical copy of it, the noise band, 23 ms; the
two copies' runs 22–49 ms); one thread binning into plain counters, 80 ms
(2.8× slower); one atomic add per value, 554 ms (19× slower), because every
increment contends for a cache line another core is writing; and one Mutex
taken per value, 2,155 ms. An atomic
is far cheaper than a lock and still far dearer than a counter no other core
touches: update it once per batch, not once per value.
Atomic arrays: AtomicI32Array and AtomicI64Array
An atomic array is length atomic counters in one contiguous buffer. It is
AtomicI32 (or AtomicI64) with the element's index first: every method of
the one has a twin on the other, with the same meaning, the same ordering and
the same rules, R1, R7 and R8 above, and the same one shared write a task or a
parallel body may make. Where an AtomicI32[] is one object per counter, each
its own allocation, so sixteen counters land on several cache lines that the
cores writing them pass back and forth, an AtomicI32Array(16) is sixteen
adjacent i32s in 64 contiguous bytes, which is the layout Rust's
[AtomicI32; 16] and Go's [16]atomic.Int32 have. The buffer starts on a
64-byte boundary at least one line into its own allocation, which ends past the
line the last element is on, so no other object shares a line with the
elements, the lines the counters' writers fight over.
import { AtomicI32Array, AtomicI64Array, parallelMapInto, scope } from "nish/threads";
class Shard {
xs: i32[];
bins: AtomicI32Array;
constructor(xs: i32[], bins: AtomicI32Array) {
this.xs = xs;
this.bins = bins;
}
}
const binShard = (s: Shard): i32 => {
for (const x of s.xs) {
s.bins.add(x & 3, 1); // one indivisible add on element x & 3
}
return 0;
};
export const main = (): i32 => {
const bins = new AtomicI32Array(4); // four counters in one buffer, each 0
const done: i32[] = [0, 0];
{
using s = scope();
s.spawn(binShard, new Shard([1, 2, 3, 4, 5], bins), done, 0);
s.spawn(binShard, new Shard([6, 7, 8], bins), done, 1);
}
const bytes = new AtomicI64Array(2);
const one: i64 = 1;
const xs: i32[] = [3, 9, 4];
const ys: i32[] = [0, 0, 0];
parallelMapInto(xs, ys, (x) => {
bytes.add(x & 1, one); // the one shared write a body may make
return x;
});
console.log(`${bins.load(0)} ${bins.load(1)} ${bins.length} ${bytes.load(1)}`); // 2 2 4 2
return 0;
};
-
The methods.
load(i)andstore(i, v);add(i, n),sub(i, n),min(i, v),max(i, v)andexchange(i, v), each answering the value elementiheld before it; andcompareExchange(i, expected, desired).lengthis areadonlyfield, fixed bynew AtomicI32Array(n), which makesnelements, each0; a negativenstops the program asnew Arrayof one does.AtomicI64Arrayis the same overi64, its index still ani32. -
An index is checked as an array index is: one out of range stops the
program with
index out of range: 16 >= 16and exit status 1, and never touches memory outside the buffer (tests/link/atomic_array_out_of_range). The check is an ordinary one, which the prover may discharge. -
Natively each method is the bounds check and then one instruction on a
getelementptrinto the buffer: aload atomicorstore atomic ... seq_cst, anatomicrmw add,sub,min,maxorxchg ... seq_cst, or acmpxchg ... seq_cst seq_cst, aligned to the element (align 4,align 8); no element is an object of its own, so there is no per-element pointer to load (tests/link/thread_atomic_array, whosethreads.llis the lowering; docs/IR_COOKBOOK.md, "an atomic array"). -
Under Node each method is the plain operation on an element of an
ordinary array, exact for the reason an
AtomicI32's is, with the same ±2^53 limit on anAtomicI64Array's elements. -
The rules are
AtomicI32's. Its buffer,values, is a field no program may name (R1, NL2514,reject_atomic_array_member_read,reject_atomic_array_member_store), and a method named without a call is refused as any class's is (reject_atomic_array_member_unbound);lengthis assigned only by the constructor (reject_atomic_array_length_store); an object literal of either class is refused by NL2515 (reject_atomic_array_literal); the parent does not touch one a scope's tasks reach before the scope joins (R7, NL2513,reject_atomic_array_region_direct); the tasks of sibling scopes share none (NL2516,reject_atomic_array_sibling_load); and a parallel body may update one and write nothing else shared (reject_atomic_array_par_shared_write,tests/link/par_atomic_array).
Choose an atomic array over an AtomicI32[] when the counters are many or hot
— a histogram's bins, a counter per route or per worker — and an AtomicI32
for one counter, or for counters that live in different objects. A counter no
other core writes is still cheaper than either: count into a task's own tally
and add it once per batch, as the example under "Atomics" does.
Channels: Channel<T>
A Channel passes scalars from a scope's tasks to one receiver: the tasks that
produce call send, and the one that consumes walks the channel with
for...of, which waits for values and ends once every task that may send on
it has returned and it is drained
(wp29-thread-surface.md §4.4). Nobody closes a
channel, so nobody can close one too early, or twice.
import { Channel, scope } from "nish/threads";
class Pipe {
ch: Channel<i32>;
n: i32;
constructor(ch: Channel<i32>, n: i32) {
this.ch = ch;
this.n = n;
}
}
const produce = (p: Pipe): i32 => {
for (let i: i32 = 1; i <= p.n; i++) {
p.ch.send(i); // never waits
}
return p.n;
};
const consume = (p: Pipe): i32 => {
let sum: i32 = 0;
for (const x of p.ch) { // waits; ends once the sender has returned
sum = sum + x;
}
return sum;
};
export const main = (): i32 => {
const sent: i32[] = [0];
const sums: i32[] = [0];
const ch = new Channel<i32>(); // for one run of the scope below
{
using s = scope();
s.spawn(produce, new Pipe(ch, 100), sent, 0); // every sender first
s.spawn(consume, new Pipe(ch, 0), sums, 0); // then the one receiver
}
console.log(`${sums[0]}`); // 5050
return 0;
};
-
Natively a channel is a buffer of 8-byte slots outside every arena, behind
the channel's
stateword, with a lock, a condition to wait on and a count that keeps it open (runtime/runtime-parallel.c).sendwidens the value into a slot and never waits; the loop takes a slot and narrows it back, and waits while the channel is empty and open. The count starts at one for the scope: aspawnwhose task may send on the channel adds one before the task is filed and takes it away when the task returns, and the scope takes its own away at every exit of its block, just before the join — whether or not the join finds a task to run — so a channel whose only sender was never spawned closes too (tests/link/thread_channel_pipeline, whosemain.llandthreads.llare the lowering;tests/link/thread_channel_exit_pathsfor the end of the block, areturn, abreakand a scope that runs nothing). Onwasm32a scope's tasks run one after another in spawn order, every sender before the receiver, and the same programs print the same thing (tests/run.js, "the--profile wasibuild"). -
The buffer is given back when a loop drains the channel closed, and the
channel records that, so a second loop over it ends at once rather than
reading what was freed; a loop that stops early with
breakleaves the rest for the next one (tests/link/thread_channel_parent_drain). Only that drain gives anything back. A channel whose last loop stopped early withbreakorreturn, or that no loop ever walks, keeps its bookkeeping — a few dozen bytes for its lock, its count and its buffer's bounds — and the values left in its buffer until the program exits, one such allocation per run of the declaration. -
Under Node a channel is an array queue.
sendpushes, and the loop takes values off the front as it walks, so a second loop finds what the first left (RUN_UNDER_NODE.md). That is correct because a task runs at its spawn under Node and every sender is spawned before the receiver, so every value has been sent by the time the receiver's loop starts.
Seven rules keep "every sender has returned" a single moment and the one receiver from waiting on what never comes, and a program that breaks one is refused with a code of that rule's own: ten codes, NL2475 to NL2484, because C2 has one for where a channel is declared, one for how it is used and one for a channel inside an arrow, and C6 one for spawn order and one for a task that sends and receives:
-
C1: a channel carries a number, a
boolean, an enum or astring. A sender's arena is freed when its scope joins, so a string is copied on the way:sendcopies it into amalloc'd block the buffer slot holds (nish_cross_send, a copy in time that grows with the string alone:tests/link/thread_channel_string_manysends a million it also keeps), andreceive, or afor...ofstep, copies it into the receiver's arena and frees the block (nish_cross_in). A string a receiver never takes — the loop that left withbreak— stays in the buffer, which is itself given back only once it has been drained closed, so the two leak together. Anything else would be a pointer into the sender's arena:A `Channel` carries a number, a `boolean`, an enum or a `string`, and this one would carry `string[]`: a sender's arena is freed when its scope joins, and only a string is copied out of it on the way, so anything else would point into freed memory; keep the values in an array the receiver can read and send each one's index, `const words = new Channel<i32>()`(NL2475,tests/cases/reject_channel_element). AChannel<string>pipeline — two producers, a rewriting stage and a consumer that keeps what it receives in an array — matches Node (tests/link/thread_channel_string). Anf64channel (tests/link/thread_channel_two_producers), abooleanone (thread_channel_parent_drain), and ani64, an enum and au8one (thread_channel_kinds) compile and carry their values exactly. -
C2: a channel is a
constdeclared beside its scope, and used only as a channel.new Channel<T>()initialises aconstin the block that holds the scope'susing s = scope(), or in the block that has the scope's block as one of its statements, before it, with no loop, condition or function between, so each run of the declaration makes a channel for one run of one scope — a scope in a loop gets a fresh one every pass:`new Channel` must be the initialiser of a `const`: a channel is a `const` declared in the block that holds its scope's `using s = scope()`, or in the block that has the scope's block as one of its statements, before it, so each run of the declaration makes a channel for one run of one scope; write `const ch = new Channel<i32>()` on the line before `using s = scope()`(NL2476,reject_channel_not_const,reject_channel_let_second_declaratorfor aletthat declares it beside something else,reject_channel_scope_conditionalfor a scope inside anif, andreject_channel_scope_loopfor one inside a loop the declaration is outside; a function boundary between them is C4's, below). A channel is the receiver ofsend, the iterable of afor...of, an argument, or a field its class's constructor sets from a parameter — never bound to another name, stored, returned or compared, and no member butsendis named:A `Channel` cannot be bound, stored, returned or compared here: a channel is used only as the receiver of `send`, the iterable of a `for...of`, an argument, or a field its class's constructor sets from a parameter, so it is never bound, stored, returned or captured, and which tasks reach it is a question the checker can answer(NL2477,reject_channel_bound,reject_channel_member,reject_channel_member_take,_ready,_headand_state,reject_channel_returned_exprfor an arrow whose expression body is a channel, which is its answer, andreject_channel_storedfor a class that holds one made anywhere but at thespawn;reject_channel_nullablefor a task's argument whose channel may benull, which thespawncould not count as a sender). A type holds a channel only as exactlyChannel<T>: never as one that may benull, an array's element, aResult's arm or another generic's type argument, wherever the type is written — a local, a parameter, a result or a field, a type alias read as the type it names (reject_channel_nullable,reject_channel_array,reject_channel_map_value,reject_channel_result_type,reject_channel_alias_type,reject_channel_field_generic). An argument is one only to a parameter its function declares aChannel, not to a generic'sT(reject_channel_generic_argument). It reaches a task as R8 says aMutexdoes: as the task's argument, or as a field of the object made for it at thespawn,new Pipe(ch, 100), and by one path only, so a task never holds it in two fields (reject_channel_two_paths). A constructor uses a channel it is handed only to set a field from it,this.ch = ch(reject_channel_ctor_bound); one it declares itself it uses as any function does. And a channel is used only in a top-level function's own body, never inside an arrow, which is lifted into a function of its own that these rules do not follow:A `Channel` is used inside an arrow here: an arrow is lifted into a function of its own that the channel rules do not follow, so a channel is declared, sent on and received from only in a top-level function's own body; write the arrow as a module function, `const relayBody = (q: Pipe): i32 => { const c = q.ch; c.send(1); return 1; }`, and pass `relayBody` where the arrow was(NL2484,reject_channel_arrow, andreject_channel_arrow_nullablefor a channel type an arrow writes but never uses). -
C3: a channel is unbounded, so
sendnever waits, because under Node a scope's tasks run one at a time and nothing would drain a full one:`new Channel` takes no capacity: a channel is unbounded and `send` never waits, because under Node a scope's tasks run one at a time and nothing would drain a channel that was full; write `new Channel<i32>()`(NL2478,reject_channel_capacity). -
C4: one scope's tasks reach a channel, and the parent sends on it only
before that scope's block. After the join it only receives. "May send" is
the call graph from a task's entry reaching
sendon a channel its argument holds, so an over-count only closes the channel later:This sends on `ch` once its scope's block has begun: a channel is reachable from the tasks of one scope, the parent sends on it only before the block that holds that scope's spawns, and nobody closes it: it closes once the scope has started to join and every task that may send on it has returned; move `ch.send(7)` above the block that opens with `using s = scope()`(NL2479,reject_channel_parent_send,reject_channel_send_after_joinfor a send through a call after the join, andreject_channel_two_scopesfor the two scopes of oneusing). Aspawnhands its task only a channel the function declared, never one it was handed, so no function boundary comes between a channel and its scope (reject_channel_spawn_handed, andreject_channel_spawn_handed_directfor the channel as the argument itself). In the constructor of a class that holds a channel, directly or through a field's class,thisappears only as the base of its own field,this.f, used in one of three ways: as the target of a store (this.f = ...,this.f += ...); as any value, except as the callee of a call, when the field is a number, aboolean, an enum, a string or an array of those, from which nothing reaches a channel (this.total = this.a + this.b,this.width = this.name.length,this.items.push(x)); or as the base of a store into a field or element of what it holds, when that is no channel and holds none (this.span.lo = 1;tests/link/thread_channel_ctor_reads). A method called on it, an argument (to a function or to a constructor), a store elsewhere, a second name (in any declarator), an array element, a read of the channel or of an object that holds one, orthisstored even into its own field would carry a send the count would not see:The constructor of a class that holds a channel uses `this` other than to read or set its own fields, `this.f`, where a send made through it could not be counted: …(NL2479;reject_channel_ctor_this,reject_channel_ctor_this_stored,reject_channel_ctor_this_bound,reject_channel_ctor_this_array,reject_channel_ctor_this_new,reject_channel_ctor_this_declarator,reject_channel_ctor_this_self,reject_channel_ctor_this_deep_carrier,reject_channel_ctor_this_send,reject_channel_ctor_this_carrier). -
C5: a channel has one receiver — one task, from a
spawnoutside any loop, or the parent in a later statement of the block that has the scope's block as one of its statements, never both — and inside a scope's block the parent receives nothing, directly or through any function it calls, so its one wait is the join:`ch` has a second receiver here: a channel has one receiver, either one task spawned outside any loop or the parent in a later statement of the block that has the scope's block as one of its statements, and inside a scope's block the parent receives nothing, so the parent's one wait is the join(NL2480,reject_channel_two_receivers,reject_channel_parent_and_task,reject_channel_receiver_loop,reject_channel_parent_receive_insidefor a call that receives inside the block,reject_channel_receive_in_spawn_argumentfor one in aspawn's argument, which is the parent's code, andreject_channel_receive_before_usingfor one before theusingin the scope's own block). A call counts as a receive when its function may receive on a channel it is handed; one that receives only on a channel of its own has joined that channel's scope first, and is not one. One receiver is what makes "the receiver sees every value sent" true natively and under Node alike. -
C6: every task that may send on a channel is spawned before the one that
receives on it, in the block's statement order. Under Node a receiver
spawned first would find the channel empty, and the same order makes the
graph from senders to receivers acyclic, so channels cannot deadlock
either:
`produce` may send on `ch` and is spawned after its receiver: under Node a task runs where it is spawned, so every task that may send on a channel is spawned, in the block's statement order, before the one task that receives on it, which is also what keeps channels from deadlocking; move the `spawn` of the receiving task below every `spawn` that sends, `s.spawn(produce, new Pipe(ch, 3), out, 1)` above `s.spawn(consume, new Pipe(ch, 0), out, 0)`(NL2481,reject_channel_receiver_first). And no task both sends and receives on one channel:`echo` may both send and receive on `ch`: no task both sends and receives on one channel, because a channel closes only once every task that may send on it has returned, and this one would wait in its own loop for its own return; split it into a task that sends and a task that receives, or answer the result through a second channel(NL2483,reject_channel_send_and_receive,reject_channel_both_through_newfor a send made by a constructor anewruns, andreject_channel_both_untraceablefor a task that reaches a channel through a second name for what holds it, which counts as using every channel it holds, and says so). -
C7: nothing receives while a lock is held, directly or through any
function it calls — R5's "nothing is waited on while it is held". A
sendnever waits and may stay:A loop over a channel is reached while `g` holds its lock: a receive waits, and nothing waits while a lock is held, so there is no receive inside a guard's block, directly or through any function it calls (a `send` never waits, and may stay)(NL2482,reject_channel_loop_held, andreject_channel_receive_held_callfor a function that receives on a channel it is handed;reject_channel_receive_held_exprfor one reached through an arrow whose expression body calls it). A function that receives only on a channel of its own opens a scope to fill it, which R5 refuses under a lock already.
What a channel program means. The receiver sees every value sent, each
sender's in its send order. How two senders' values interleave is the
scheduler's natively and spawn order under Node, so a receiver whose answer
depends on it — not a count or an integer sum, but the first value, or an
f64 sum whose partial sums are not exact — is its author's obligation, as
for the lock. A task's channel is checked where it can be read, at the
spawn; a parallel body has no way to reach one, because a task calls no
parallelMapInto and a channel is made only beside a scope. select stays
declined (wp29 §9).
What nish --fix writes for a thread program
The thread refusals end with the code to write instead (each pinned whole by
its tests/wordings case), and six of them carry a fix that nish --fix applies, each only where the source alone decides the
replacement (tests/fix/thread-*; the refused shapes are its thread-nofix-*
cases):
| Refusal | The fix | Left alone |
|---|---|---|
NL2388, a scope() not bound by using |
const s = scope() becomes using s = scope() |
a let, which is assigned again |
NL2465, a lock() not bound by using |
const g = m.lock() becomes using g = m.lock() |
a let |
NL2511, a spawn with no destination, whose task answers a number or a boolean |
const answer = new Array<T>(1) above the scope's using, and , answer, 0 after the arguments |
a spawn in a loop, where every task would write one slot; an answer that is not a number or a boolean |
NL2506, a task's arrow that captures a let |
const kNow = k just before the statement, and the arrow reads kNow: the value it would have read when it ran |
a let another argument names; a let a for header declares, where no statement before the call is in the loop |
NL2500, the same in a parallelMap |
the same | the same |
NL2484, an arrow that uses a Channel |
the arrow moves to a const above the top-level statement it is written in, named for the function and Task (mainTask), and the call names it |
an arrow that captures, or leaves a parameter's or its result's type out, which would not check at the top level |
Shared memory between processes: nish/sync/shm and nish/sync/ring
Threads share an address space; processes share nothing unless they map the
same memory. nish/sync/shm maps a named region that every process of
the same user can open, and nish/sync/ring builds lock-free queues of byte
records in one, so prefork workers can hand work to an owner process without a
socket:
import { ACQ_REL, ACQUIRE, createRegion, unlinkRegion } from "nish/sync/shm";
import { initSpscRing, spscRingBytes } from "nish/sync/ring";
export const main = (): i32 => {
const region = createRegion("lang-demo", 64 + spscRingBytes(4096)).expect("create");
region.addI64(0, toI64(1), ACQ_REL); // a counter every process may bump
const ring = initSpscRing(region, 64, 4096, true); // a ring after it
const job: u8[] = [1, 2, 3];
ring.tryPush(job, 0, job.length); // false when full
const got: u8[] = new Array<u8>(16);
const n: i32 = ring.tryPop(got, 0); // 3, or RING_EMPTY
console.log(`${region.loadI64(0, ACQUIRE)} ${n}`);
unlinkRegion("lang-demo").expect("unlink");
return 0;
};
-
A region is named, never inherited. A Nish program has no
fork, and every descriptor is close-on-exec, so an anonymous mapping or amemfdcannot reach a child; a name can.createRegion(name, bytes)makes one, zeroed, mode 0600, and refuses a name that exists (SHM_EXISTS), so a name another user planted is never adopted.openRegion(name)maps one this user made, and refuses one owned by someone else, open to group or others, reached through a symbolic link, or (on Linux) not a regular file or with a second hard link (SHM_ACCESS). A name is 1 to 30 ofA-Z a-z 0-9 . _ -, not starting with.; a size is 1 byte to 1 GiB. Errors are aResultwith the errno (SHM_NOT_FOUND,SHM_INVALID,SHM_TOO_MANY, ...). -
Lifetime. The name lasts until
unlinkRegion, even past every process that mapped it, which is how a process started later finds it. A mapping lasts untilclose()or the process's exit, when the kernel unmaps it. A thread may hold 63 regions at once. -
Every access is checked, as an array index is.
loadI32,storeI32,addI32,exchangeI32,compareExchangeI32and theirI64twins take ani32byte offset; a slot must lie wholly inside the region and at a multiple of its width, or the access panics naming the offset (nish/sync/shm: slot offset 4 is not a multiple of 8, the slot's width).readandwritecopy a byte window, checked against both the region and the buffer. No offset reaches outside the mapping, whatever a peer wrote into it. -
Windows: a
u8[]that is the region's bytes.region.lend(at, len)answers au8[]of lengthlenwhose data is bytes[at, at + len)of the mapping, checked asread's window is (nish/sync/shm: byte window at 60 is outside the region of 64 bytes). An index store,copyBytesinto it ornetReadinto it writes the region itself, and every process mapping it sees the bytes, with no copy. The window is not atomic: order it with a slot (storeI32(.., RELEASE)after the bytes,loadI32(.., ACQUIRE)before reading them), as a ring does.region.giveBack(window)answers its length and empties it, andclose()empties every window the region lent, so an index after either isindex out of range: 0 >= 0, never a read of an unmapped page. Apopor a shorterlengthkeeps a window; a push past its length moves it off the region into the program's memory, as any push past capacity does, andgiveBackthen panics (window of length 17 is no longer the region's bytes: a push moved it), as it does for a window given back twice or lent by another region. -
The reuse rule. Each
lendwithoutreusetakes a header of 48 bytes that lasts until the process exits, since the program may hold the array past anygiveBackorclose. A loop that lends again passes the window it gave back,region.lend(at, len, window), which re-points that header and takes no memory; areusethat is still lent, or not this region's, panics. Windows belong to the thread that lent them, as the region does. -
Why an intrinsic, not FFI.
lendandgiveBackare the module's two compiler-known functions (src/shm-window.ts), whose bodies the compiler writes as a call ofnishshm_lend/nishshm_give_back: a foreign function may take au8[]but answers no array, and a window must be one the program indexes. Known by its package and path, asnish:regionis, the call is marked as writing through and keeping its array and as possibly changing any array's length, so no length the bounds proof knew survives it, and a region'sclose()is marked the same way. -
Orderings are C11's:
RELAXED,ACQUIRE,RELEASE,ACQ_REL,SEQ_CST, plusfence(order). Each access is one LLVMload atomic,store atomic,atomicrmworcmpxchgwith that ordering; an ordering an access cannot take (a load withRELEASE) panics. A compare-exchange answers the value it found, so it succeeded when that equalsexpected. -
Waking.
waitI32(at, expected, timeoutMs)sleeps while the slot holdsexpectedandwake(at, count)wakes sleepers, in any process: Linux's shared futex. Darwin and wasi answerSHM_UNSUPPORTED(-95) from both, and a program there polls. -
Rings.
SpscRing(one producer, one consumer, records of any length up to its capacity) andMpscRing(any number of producers, fixed slots) live at a 64-byte-alignedbasein a region:initSpscRing/initMpscRingwrite the header once,attachSpscRing/attachMpscRingread it in another process.tryPushanswersfalsewhen full;tryPopanswers the record's length,RING_EMPTYorRING_TOO_SMALL;popBatchtakes every record ready. A ring made with the doorbell letspopWaitsleep, and a push rings it only when the consumer is asleep, so a busy ring makes no system call. Two consumers, or two producers on anSpscRing, lose or repeat records but never leave the region; a length no record can have panics asthe ring is corrupt. -
What a program reaches. The modules call C, so a program that imports
them reaches
ffi(andclock, forpopWait's deadline) and--deny ffirefuses it; under--sandboxa region, being a file in/dev/shm, needsfs.readandfs.writeas well. They are native only:--emit-tsrefuses them, since Node'sSharedArrayBufferis shared between threads of one process, not between processes. A region belongs to the thread that mapped it. A program that imports neither module is unchanged. Tested bytests/link/sync_shm,tests/link/sync_shm_window,tests/link/sync_ringandtests/link/sync_shm_deny_ffi; the design, its measurements and the threads it was weighed against aredocs/wp47-shared-memory.md, and the threat model isdocs/security/shm.md.
Random numbers: Rng in nish/random
new Rng(seed) is a generator of pseudo-random numbers whose every answer
depends on its i32 seed alone, and is the same natively and under Node:
import { Rng } from "nish/random";
export const main = (): i32 => {
const r = new Rng(42);
const die: i32 = r.nextInt(1, 6); // 1 to 6, each with equal chance
const u: f64 = r.nextF64(); // in [0, 1)
console.log(`${die} ${u < 1.0}`);
return 0;
};
-
nextI32()answers the next 32 bits as ani32, andnextF64()a double in [0, 1) built from 53 bits of two steps. The state walks a Weyl sequence (an odd increment, wrapping) through all 2^32 values, and each answer is MurmurHash3's 32-bit finaliser of it, so every seed, 0 included, starts a full sequence, and seeds 1, 2 and 3 start unrelated ones. It is not for cryptography: a key comes fromcrypto.getRandomValues. -
nextInt(lo, hi)answers ani32in [lo, hi], both ends included, each with equal chance. It refuses the fewnextI32()draws below 2^32 mod the span and answersloplus the draw mod the span, so no answer is likelier than another; fewer than half the draws are refused, so a call takes under two steps on average.nextInt(-2147483648, 2147483647)is onenextI32(), andlo > hipanics withRng.nextInt: lo is greater than hi. -
The same bits under Node are the point of the design. The
operations that wrap are
wrappingAdd,wrappingSubandwrappingMulfromnish:unsafe, every shift has a constant count,nextIntreads its unsigned counts throughi32compares and remainders of non-negative values, andnextF64's double is exact, sotests/link/std_randomprints the same lines both ways. -
A parallel body or a task steps an
Rngit made itself, which is its own memory (const r = new Rng(i), "Nor is a write into an object's own slots" under data parallelism); a generator shared by several tasks would be a shared write, and is refused as one. Seed each task's own from its index.
Byte search: indexOfAny in nish/text
indexOfAny(text, bytes, from) answers the first index at or after from
whose byte is one of bytes, or -1 (wp38-simd.md §3.1). It is
the scan a parser spends its time in — the next " or \ in a JSON string,
the next delimiter in a line — and natively it reads 16 or 32 bytes at a time.
import { indexOfAny } from "nish/text";
export const main = (): i32 => {
const line = "key: \"value\", next";
console.log(`${indexOfAny(line, "\":", 0)}`); // 3, the colon
console.log(`${indexOfAny(line, ",", 99)}`); // -1: `from` is clamped to the length
return 0;
};
-
The set is 1 to 16 ASCII bytes, or the call panics. An empty set
(
indexOfAny: the set must hold 1 to 16 bytes) matches nothing and is a mistake rather than a search; seventeen bytes is more than one vector of the kernel compares; and a byte of 128 or more (indexOfAny: every byte of the set must be ASCII) would be one byte of a multi-byte sequence natively and a UTF-16 unit under Node, which find different things (tests/link/std_text_index_of_any_empty_set,_long_set,_non_ascii_set). -
fromis clamped to[0, text.length], assubstringclamps: a negativefromsearches from the start, and one at or past the length answers -1. The answer is a byte offset, likes.length; an ASCII byte never occurs inside a multi-byte UTF-8 sequence, so a non-ASCIItextis searched as it is and only its ASCII bytes match. Under Node the offset is a UTF-16 one, so the two agree on what was found and, once a non-ASCII character precedes it, not on where (tests/link/std_text_index_of_any). -
The module is ordinary Nish, and the walker is the meaning.
std/text.tswritesindexOfAnyas its checks and its clamp followed by one call to a private walker, a loop overcharCodeAt, which is what runs under Node. The compiler recognises that call by module and name (src/kernels.ts) and emits the runtime'snish_str_index_of_anyin its place; everything around it is emitted as written, so the kernel is only ever handed a clampedfromand a set of 1 to 16 bytes. The package is part of the test: a root-package file that happens to sit atstd/text.tsis an ordinary module, and its walker is called as written. The kernel picks a scalar, a 16-byte (SSE2 or NEON) or a 32-byte (AVX2) path on its first call, andNISH_SIMD=scalar|base|avx2pins it. -
It is variable-time. How long the search takes says where the first
match is, so it is never for secret data:
std/cryptodoes not call it on one, andnish:secret'sexposelets it through only because it touches nothing outside its two strings.
On a 1 MiB text with a match every 4,099 bytes, the kernel scans at 14 GB/s
with AVX2 and 7 GB/s on the 16-byte path, against 1.6 GB/s for the same
walker compiled as written (bench/index-of-any.ts).
Fixed digits and a split limit: toFixed and splitLimit in nish/text
Nish has no methods on a number and no optional parameters, so JavaScript's
x.toFixed(n) and text.split(sep, limit) are two functions of nish/text.
Each is the method it stands for, read as TypeScript: under Node the module
runs as written and answers what the method answers, and natively it answers
the same bytes (tests/link/std_text_to_fixed, tests/link/std_text_split_limit,
whose every line tests/run.js also holds to Node's own method).
import { splitLimit, toFixed } from "nish/text";
export const main = (): i32 => {
console.log(toFixed(1.005, 2)); // 1.00: 1.005 is stored a little below
console.log(toFixed(2.5, 0)); // 3: a tie goes away from zero
console.log(splitLimit("a,b,c", ",", 2).join("|")); // a|b: the rest is dropped
return 0;
};
-
toFixed(x, digits)rounds the exact binary value, as ECMA-262 does:1.005is stored as 1.00499999999999989…, so it gives"1.00", while8.345is stored a little above and gives"8.35"; an exact tie rounds away from zero, sotoFixed(-2.5, 0)is"-3".digitsis 0 to 100, and any other count panics (toFixed: digits must be 0 to 100) where JavaScript throws aRangeError.NaN, the infinities and anyxof magnitude 1e21 or more answer${x}, as JavaScript does;-0gives"0.00", and a negativexthat rounds to zero keeps its sign (toFixed(-0.0001, 2)is"-0.00"). It is written in Nish over an array of decimal digits, so it is exact for everyf64, the subnormals included, and it needs no runtime C. -
splitLimit(text, sep, limit)keeps the firstlimitpieces ofsplit(text, sep)and drops the rest, rather than joining them into the last; a limit of 0 gives[]. A negative limit keeps every piece, because JavaScript reads the limit throughToUint32and -1 becomes 4294967295; that makessplitLimit(text, sep, -1)the same assplit(text, sep). An empty separator panics (splitLimit: an empty separator) whatever the limit, assplit's does, even where JavaScript would answer[].
Byte runs: skipBlank, skipDigits, skipJsonText, skipJsonPlain and copyBytes in nish/bytes
The four runs answer the first index in [from, to) of a u8[] whose byte
is not in their class, or to when every byte there is; copyBytes copies a
window of one array into another, or into itself
(wp38-simd.md, S6a). They are the scans and the copy a parser
spends its time in, and std/json is written over them.
import { copyBytes, skipBlank, skipDigits, skipJsonPlain, skipJsonText } from "nish/bytes";
export const main = (): i32 => {
const text: u8[] = [32, 10, 52, 50, 44, 34, 104, 105, 34]; // " \n42,\"hi\""
const start = skipBlank(text, 0, 9);
console.log(`${start} ${skipDigits(text, start, 9)}`); // 2 4
console.log(`${skipJsonText(text, 6, 9)}`); // 8, the closing quote
console.log(`${skipJsonPlain(text, 0, 9)}`); // 1, the newline
const word: u8[] = [0, 0];
copyBytes(word, 0, text, 6, 2);
console.log(`${word[0]} ${word[1]}`); // 104 105
return 0;
};
-
The classes.
skipBlankskips JSON's four blanks (space, tab, newline, carriage return) andskipDigitsthe ASCII digits.skipJsonTextskips what a JSON string holds as itself: printable ASCII other than"and\(DEL included, which JSON does not escape), and every well-formed UTF-8 sequence (RFC 3629 §4) that ends beforeto. It stops at a quote, a backslash or a control byte, and at the lead byte of a sequence that is overlong, a surrogate, past U+10FFFF, a stray continuation byte or cut short byto, so where it stops is the byte a decoder has to look at.skipJsonPlainskips what a JSON writer copies without an escape: every byte from 0x20 up other than"and\, DEL and 0x80–0xff included, with no UTF-8 check. It stops at a quote, a backslash or a byte below 0x20. -
The runs never panic.
tois clamped into[0, b.length]andfrominto[0, to], asindexOfAnyclamps: a negativefromscans from the start, and afromat or pasttoanswers it (tests/link/std_bytes_runs). -
copyBytespanics on a range outside its array. A negativecount, or a window that does not lie insidesrcordst, panicscopyBytes: the range lies outside an arraybefore any byte is written (tests/link/std_bytes_copy_outside). Overlapping ranges of one array copy as if every byte were read before any was written, ascopyWithindoes. -
The walkers are the meaning. Each public function clamps or checks its
range, then makes one call to a private walker, a plain loop over the bytes,
which is what runs under Node. Natively the compiler recognises that call by
module and name (
src/kernels.ts), as it doesindexOfAny's, and emits the runtime'snish_bytes_skip_blank,nish_bytes_skip_digitsornish_bytes_skip_json_textornish_bytes_skip_json_plain(runtime/runtime-bytes.c) in its place, or onellvm.memmovefor the copy. The kernels pick a scalar, a 16-byte (SSE2, NEON or wasm SIMD) or a 32-byte (AVX2) path on their first call, andNISH_SIMD=scalar|base|avx2pins it, except in a freestanding wasm32 build, which has no environment to read it from. -
The runs are variable-time. How long one takes says how long the run is,
so they are for text, never for secret data:
std/cryptodoes not call them.
SQLite: nish/sqlite
nish/sqlite is the SQLite C library, linked into a program that imports the
module, with a Database and a Statement to drive it. A statement is
prepared once and bound, stepped and reset as often as wanted:
import { openDatabase, OPEN_READWRITE, OPEN_CREATE } from "nish/sqlite";
export const main = (): i32 => {
const db = openDatabase(":memory:", OPEN_READWRITE | OPEN_CREATE).expect("open");
db.exec("CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT)").expect("create");
const insert = db.prepare("INSERT INTO t (name) VALUES (?1)").expect("prepare");
insert.bindText(1, "ada").expect("bind");
insert.step().expect("insert");
insert.reset().expect("reset");
console.log(`${db.lastInsertRowid()}`);
db.close().expect("close");
return 0;
};
-
Every failure is a
Result.openDatabaseandprepareanswer aSqliteError, with SQLite's extended code (extendedCode, such as 2067 forSQLITE_CONSTRAINT_UNIQUE), its primary code (code, the low byte) and SQLite's message. Every other call answers the bare extended code, anderror(code)on theDatabaseorStatementthat answered it builds theSqliteError, so a failure allocates only for the caller who asks.step()answerstruewith a row to read andfalsewhen the statement has finished. -
A row allocates nothing. Binding,
step,reset,columnI64,columnF64andcolumnIntoanswer scalars;columnInto(c, buf, at)copies a text or blob column into the caller'su8[], and answers its length, copying only when all of it fits.columnText(c)is the call that makes a string, and it stops at the first NUL byte. Parameters count from 1 and columns from 0, as in SQLite. -
exec(sql)runs every statement ofsqland discards any rows, andbegin(),commit()androllback()areexecofBEGIN,COMMITandROLLBACK; pragmas, such asjournal_mode=WAL, are SQL too. A binding is copied by SQLite, andresetkeeps the bindings. -
A handle is an
i32into tables the C adapter keeps per thread.closefinalizes every statement the connection still has, and a handle used aftercloseorfinalizeanswersSQLITE_MISUSErather than reaching freed memory. A connection and its statements belong to the thread that opened them. -
It reaches
ffi. The module calls C throughdeclare function, so a program that imports it is reported withffiand--deny ffirefuses it, naming the call. The files SQLite opens are opened by C, which thefs.readandfs.writecapabilities do not see. It is tested natively and under--profile wasi(tests/link/std_sqlite*).
Interfaces and object literals
interface Pair { first: number; second: number; }
const swap = (p: Pair): Pair => ({ first: p.second, second: p.first });
-
A record type is always a declared name. There is no inline object type:
(p: { x: i32 }) => p.xissyntax error: expected a type name, found `{`(NL0001), and so is the same type aftertype Point =, in a field, a return type or a type argument (tests/cases/reject_type_object_literal_param,reject_type_object_literal_alias,_field,_return,_type_arg). A record type is identified by its declaration, not by its fields: two interfaces that declare the same fields are two types, and a value of one is refused where the other is wanted, the only widening being a class to an interface itimplements(Types). So a record's fields are declared once, in aninterfaceor aclass, and every annotation names it. -
An interface is a struct type with the same layout rules as a class, with fields only: a method, or an accessor signature
get x(): i32, isInterface `Shape` cannot declare methods (interfaces describe layout only)(tests/cases/reject_iface_method,reject_iface_accessor; a field may still be calledgetorset),extendsisInterface inheritance (`extends`) is not supported; list every field(tests/cases/reject_iface_extends,reject_generic_interface_extends), an index signature isIndex signatures are not supported in interface `Point` (object layout is fixed)(reject_iface_index_signature), a construct or call signature,new (): Pointor(x: i32): i32, isUnsupported interface member in `Point`: ConstructSignature(orCallSignature,reject_iface_member_unsupported),declare interfaceis`declare interface` is not supported, andnewon an interface isCannot `new` interface `Pair`(tests/cases/reject_cls_new_interface). -
An object literal allocates in the arena and stores every property in source order. It must set every field exactly once (
is missing field `second`,tests/cases/reject_cls_literal_missing;`Pair` has no field `third`,reject_cls_literal_extra), with plain identifier keys andkey: valueor shorthand members: a key written as a string or a number isObject literal keys must be plain identifiers(reject_cls_literal_key), and a method or an accessor isUnsupported object literal member: MethodDeclaration (only `key: value`)(GetAccessor,SetAccessor;reject_cls_literal_member) —get,setandasyncstay keys and shorthands ({ get: 1 },{ get }). It takes its type from the context: a variable annotation, the enclosing function's return type, the parameter it is passed to, the field or variable it is assigned to, or the field of an enclosing literal; otherwiseObject literal needs a contextual class or interface type(tests/cases/reject_cls_literal_no_context,cls_interface_literal,cls_nested). That context reaches a nested object literal and anull, and stops there: a numeric literal written for a field keeps the mode's default type (see "Numeric literals"), and[]has no element type at all in this position —{ xs: [] }isEmpty array literal needs a type annotationand wants a named array instead (tests/cases/reject_struct_field_empty_array). A value of the wrong type isField `code` of `IoError` expects a value of type i32, got f64(reject_struct_field_f64). -
A nullable context gives an object literal the struct inside it: wherever one of the contexts above expects
E | null— a declared local, areturn, an arrow's concise body, a ternary arm whose other arm isnull, a parameter, or a field — the literal is anE, checked againstE's fields as above, and thatEconverts toE | nullas any otherEvalue does, the same pointer with no wrapper (tests/cases/obj_lit_nullable_ternary,obj_lit_nullable_return,obj_lit_nullable_arrow,obj_lit_nullable_local,obj_lit_nullable_field;reject_obj_lit_nullable_field). A ternary with no declared type around it gives neither arm a struct, soconst z = c ? { v: 7 } : nullis stillObject literal needs a contextual class or interface type(reject_obj_lit_nullable_context). -
A class with methods is made only by
new. Natively a literal would take the class's layout and its methods, but under Node it is a plain object, and a call on it throws, so wherever a context above types a literal as such a class it is`Counter` has methods, so it is made only by `new`: an object literal would be a plain object under Node, with none of the methods the class is used through. Write `new Counter(...)`(NL2556;tests/cases/reject_class_literal_decl,…_return,…_arrow_body,…_call_arg,…_method_arg,…_new_arg,…_array_elem,…_push,…_index_assign,…_map_set,…_record_field,…_assign,…_field_assign,…_ternary,…_paren,…_alias,…_coalesce,…_coalesce_chain,…_coalesce_field,…_coalesce_call,…_coalesce_map,…_coalesce_optional,…_coalesce_ternary,…_equality,…_index_of,…_math_max,…_fn_param,…_unary,…_compound_bitwise,…_compound_arith,…_nullable,…_generic_class,…_generic_target,…_generic_method,…_imported). A generic method counts as a method; a constructor does not. A class with fields alone, and an interface, still take a literal (tests/cases/cls_literal_methodless). Thenish/threadsclasses keep their own refusal, NL2515 (see "Atomics"), and a class holding aSecretkeeps NL2545. -
An object spread
{ ...s, x: 1 }copiess's fields into the literal, one store per field, and is otherwise the literal written out: the spread and{ x: 1, y: s.y }emit the same IR, anallocawhere the literal stays local and an arena allocation where it escapes (tests/run.js"object spread",tests/cases/obj_spread_twin). Each spread is evaluated once, in source order with the entries around it, and the entry written later wins: a field set after a spread overrides it, and of two spreads the later one supplies the fields both have; only the fields no later entry overrides are stored (tests/cases/obj_spread_order). A field holding an object is copied as the reference it is (obj_spread_nested), a class instance spreads its fields and none of its methods, as Node's spread does (obj_spread_class), and a nullable source spreads once narrowed (obj_spread_narrowed); a write to the source after the spread leaves the copy as it was (obj_spread_class). The literal must still set every field of its type, by the spread or by name: a spread that supplies only some of them leaves the restObject literal for `Point` is missing field `z`(reject_object_spread_missing_field). Each form whose Node reading would differ from the copy is refused:Form Code Tests no contextual class or interface type, const o = { ...s }:An object literal with a spread needs a contextual class or interface type, whose layout the spread's fields are copied into: annotate the variable (`const p: P = { ...s, x: 1 }`)NL2709 reject_object_spread_no_target,reject_ffi_handle_walk_spreada source that is not a class or interface value, { ...n }for ani32NL2705 reject_object_spreada source that may be null, which Node spreads as no field at all: narrow it firstNL2703 reject_object_spread_nullablea union source, whose fields depend on the variant it holds NL2704 reject_object_spread_uniona class instance with a field the target does not declare, which Node would carry on the object NL2706 reject_object_spread_class_extraan interface value with a field the target does not declare NL2707 reject_object_spread_extra_fielda copied field whose type differs from the target's, i32intoi64: the copy converts nothing. A field a later entry sets by name is not copied, so its type is not checked (obj_spread_override_type); one a later spread supplies is still checkedNL2708 reject_object_spread_field_typea field set by name before a spread that always overwrites it, TypeScript's TS2783 NL2710 reject_object_spread_overwrittenAn interface value may hold any class that implements it, and such a class may declare fields after the interface's prefix (see
implementsbelow): Node copies every own field of the instance, the native copy only the interface's. So once the whole program is checked, a spread of an interfaceIinto a type that lacks a field of some class implementingIisSpread of `Named` into `Tagged` may spread an instance of `Item`, which implements it with field `weight` besides: …(NL3055,reject_object_spread_width); spread a value of the class type instead. A spread into the typed JSON options of a class is NL2711 (reject_json_options_spread). One gap remains: Nish acceptsprivateon a field and ignores it, so a class instance with aprivatefield spreads into a type that declares that field, wheretscrefuses the spread; under Node the copy is the same object either way. A computed key is still refused (NL1041). NL1061, Phase 0's old refusal of every object spread, is retired and never reused. -
readonlyon an interface field forbids every assignment (i.a = 2isCannot assign to readonly field `a` of `Config`,tests/cases/reject_cls_readonly_interface); literals still set it. The message drops the class form's "outside its constructor", because an interface has none to point at. -
class C implements IrequiresI's fields to beC's first fields, in the same order and with identical types;Cmay declare more after them (tests/cases/cls_implements_prefix). A missing one isdoes not implement `Point3`: it lacks field `z: i32`(reject_cls_implements_short) and a mismatched one isfield 1 is `width: i32` in `Shape` but `height: i32` in `Square`(reject_cls_implements_mismatch); both end(the interface's fields must be the class's first fields, in order).A
Cthen converts toIwherever anIis expected (initializer, return, argument, assignment, field store, ternary arm,I[]element) by onebitcast, because the two address the same bytes at the same offsets for every fieldInames; a class with the right fields that does not listIis not assignable (tests/cases/cls_implements,reject_cls_not_implements). Nothing converts back: there is no downcast, noas, noinstanceof. The class's own fields after the prefix reuse its tail padding, exactly as clang lays out the same C struct (tests/layout/structs.ts, classesK,L,M).This is the only widening in the language. Since WP25 a class is a prefix of nothing but the interfaces it names, which is what lets two classes with different tails be held in one
I[]and operated on by one function (tests/cases/cls_implements_prefix,tests/differential/corpus/class_prefix) — the jobextendsused to do, without a dispatch rule to get wrong.
Calling C
declare function name(params): T; declares a C function this program calls
but does not define. The symbol is the identifier — no mangling, no prefix —
and the call is an ordinary call:
declare function abs(n: i32): i32;
declare i32 @abs(i32)
%0 = call i32 @abs(i32 %n)
The spelling is TypeScript's own ambient declaration, and declare is a
contextual keyword: a variable may still be called declare.
A parameter is a scalar, a CPtr, a borrowed string, a u8[] buffer
with its count or a borrowed declare class; a return is a scalar, a CPtr or
a copied string — i32,
i64, u8, u16, u32, u64, f32, f64, boolean, void, the foreign
pointer below, string and string | null in either position, and
a buffer and
a C struct as a parameter. Any
other class or an interface is rejected with
and a declared C function takes scalars, `CPtr`, a `string` it borrows as `const char *` and a `u8[]` buffer followed by its `u64` count, or a `declare class` it borrows by pointer in a
parameter (reject_ffi_class_param), any other array with
the only array a declared C function takes is a `u8[]` or a `readonly u8[]`
(tests/cases/reject_ffi_string_array_param: a string[] too, which would be
a char ** the compiler built per call) and
returns a scalar, a `CPtr` or a `string` it copies from a `const char *`
in the return position (reject_ffi_string_array_return). This is
not a stylistic limit: the only pointer this compiler allocated that crosses
the boundary is a string C reads for the length of the call, and the only one
C hands back that the program keeps is a copy it made, so there is nothing for
a foreign function to capture or free, which is what lets the escape analysis
stay correct without knowing anything about the callee.
A u8, u16 or boolean crosses as C's unsigned char, unsigned short or
_Bool, and the caller widens it (#1755). The x86-64 SysV and Apple arm64
ABIs give the caller that job and a callee clang built reads the whole
register, so the declare and every call carry zeroext on each such
parameter and result; toU8(511) reaches C as 255
(tests/cases/ffi_narrow_zeroext):
declare i32 @widen_u8(i8 zeroext)
declare zeroext i1 @is_odd(i32)
%3 = call i32 @widen_u8(i8 zeroext %2)
Nish has no signed integer narrower than i32, so no argument takes signext.
A string parameter: borrowed for the length of the call
A string parameter is C's NUL-terminated const char *, and C borrows it:
a foreign function may read the bytes until it returns and may not keep the
address. A function that keeps its argument — putenv, setvbuf's buffer, a
callback's context — is declared to take a CPtr instead, and the program hands
it a copy C made (tests/cases/ffi_string_param, ffi_string_param_arena,
cookbook):
declare function strdup(s: string): CPtr | null;
declare function putenv(entry: CPtr): i32; // keeps the pointer: C's, not ours
const kept = strdup("MODE=fast");
if (kept !== null) {
putenv(kept);
}
declare i8* @strdup(i8*)
%0 = getelementptr inbounds i8, i8* bitcast ({ i64, [10 x i8] }* @.str.0 to i8*), i64 8
%1 = call i8* @strdup(i8* %0)
-
The argument is the string's own bytes, with no copy. A
nish_strstores them NUL-terminated, a literal included, so the call passes the address eight bytes into it. The escape analysis treats the argument as a read, the way it treats a stringconsole.logprints: an arena scope that holds it is released after the call returns, and the call is never atailcall. -
C may not write through it. A literal lives in read-only constant data and
an arena string is shared freely, so a C function that writes into its
argument (
strtok,mkstemp) corrupts a value the program still holds, or faults. Such a function takes aCPtrbuffer allocated through C. The compiler cannot check this rule; it is the declaration's author's word, like the borrow. -
A string that holds a NUL byte panics before the call, because C would
read the NUL as the end of the string —
"secret\0.txt"would pass a.txtcheck in the program and namesecretin C. The message names the callee, the argument's position and the parameter:`strlen`: argument 1 (s) holds a NUL byte, which C would read as the end of the string(tests/cases/panics_ffi_nul). That is theffi-nulpanic site. A literal with no NUL in it is proved at compile time, soputs("hello")has no check and no site; every other string is checked with one scan of the bytes C is about to read anyway. string | nullpassesNULLfornull, and checks only a string.-
The declaration carries no attribute for a string parameter:
nocaptureandreadonlywould restate the borrow as an LLVM fact the C body need not honour, which would be a miscompile rather than a broken contract.
A string result: copied out of C's const char *
A declare function that returns string or string | null answers C's
NUL-terminated const char *, and the program copies its bytes into the arena
at the call. C keeps its buffer — strerror's table, getenv's environment,
setlocale's static, which the next call may overwrite — and the copy is the
program's own string from then on, placed, kept and released like any other
allocation of the function that made the call (tests/cases/ffi_string_return,
cookbook):
declare function strerror(errnum: i32): string;
declare function getenv(name: string): string | null;
console.log(strerror(2)); // No such file or directory
const home = getenv("HOME");
%0 = call i8* @strerror(i32 2)
%1 = icmp eq i8* %0, null
br i1 %1, label %ffi.null, label %ffi.copy
ffi.copy:
%2 = call i8* @nish.ffi.str(i8* %0)
-
The copy is one scan for the NUL and one
nish_str_new, in a private helper the module defines once (@nish.ffi.str). C's pointer is never held past it, so a buffer C reuses or frees later cannot reach the program. -
Every call's copy is new, and is the calling function's allocation, as
s.substring(...)is: a loop that keeps one keeps it past the pass, and ausing a = arena()block that would release one held outside it is refused, as it is for any fresh string (tests/cases/ffi_string_loops,reject_ffi_string_arena). -
The copy follows the call, so a call with a
stringresult is never atailcall: a C function that answers a pointer into its argument (strchr,strstr) is read before that argument's arena scope is released. -
string | nullreadsNULLasnull, tested before the copy but with no panic site. -
A
stringresult that isNULLpanics after the call, because astringcannot hold it:`getenv` returned NULL for a `string` result; declare it `string | null`(tests/cases/panics_ffi_null). That is theffi-nullpanic site. Declare the resultstring | nullwhenever the C function's manual says it may answerNULL. -
A result C allocated for the caller to free (
strdup,realpathwith no buffer) is aCPtr, not astring: the copy would leave C's allocation with no one to free it. Declare itCPtr | null, copy it withreadCString, and pass the pointer back tofree.
CPtr: the pointer a C function hands back
CPtr is an address C owns. It is eight bytes and i8* in the IR, and that
is the whole of what the compiler knows about it: it is not arena memory, not a
%struct, not a nish_str, and its lifetime is the C library's business
(tests/cases/ffi_pointer, cookbook).
declare function calloc(count: u64, size: u64): CPtr | null;
declare function free(block: CPtr): void;
-
It may be written in a
declare functionsignature and on a local that holds one, and nowhere else. A field, an array element, aResultarm, a type argument, and the parameters or return type of a function this program defines are all`CPtr` cannot be ...: a foreign pointer may only appear in a `declare function` signature or on a local bound to one(reject_ffi_pointer_field,reject_ffi_pointer_array,reject_ffi_pointer_param,reject_ffi_pointer_return,reject_ffi_pointer_type_argument_fn). The first group would put a foreign address inside a value the arena owns and the escape analysis walks; the second would put one across a boundary--emit-header,--emit-dtsand--emit-napidescribe, and none of the three has a spelling for an address whose provenance and lifetime are unknown. The type-argument clause is stated where a generic function or class is instantiated (reject_ffi_pointer_type_argument), sonew Empty<CPtr>()is refused even when no member ofEmpty<T>namesT. The fields that may hold aCPtrare a handle class's and adeclare class's, below. -
A
nullreaches C only as the literal. Adeclare functionmay returnCPtr | nulland may take one, and a call passes aCPtr | nullparameter either the literalnull—realpath(path, null),getcwd(null, 0), the manual's "pass NULL to have it allocated" — or a pointer narrowed withif (p !== null). A nullable pointer the program holds isArgument 1 of `free` is a `CPtr | null` the program holds: a nullable foreign parameter takes the literal `null` or a pointer narrowed with `!== null`, so C is never handed a null by a path nobody tested(NL2539,reject_ffi_pointer_nullable_variable), wherever it is held: a local, a parenthesised one, a ternary or??that may answer it, inside a constructor reached bynewor a method, and through a function parameter adeclare functionis passed to (reject_ffi_pointer_nullable_specialised). So everyNULLC receives is one the source spells, and a pointer that may beNULLbecause C said so is narrowed before it goes back. The narrowing is the ordinary nullable one (WP6). The literal lowers toi8* null(tests/cases/ffi_owned_string). -
No arithmetic, no dereference, no index, no member.
p + 1isOperator `+` requires two operands of the same numeric type(reject_ffi_pointer_arithmetic) andp[0]isCannot index a value of type CPtr. What is left is=== null,!== null, comparing twoCPtrs, assignment, and passing one to another foreign function. -
It gets no pointer attributes.
dereferenceable,align,nonnull,nocaptureandreadonlyare each a claim about memory this compiler laid out, and it laid out none of this;isPointerParaminsrc/attributes.tsis an allow-list so that aCPtrfalls out of it by construction rather than by a case somebody remembered to write.
readCString: a string out of a buffer C owns
readCString(p: CPtr): string, from
nish:unsafe, copies the
NUL-terminated bytes at p into a new string of the program's own. It is the
copy a string result makes, offered for the pointers a string result cannot
be: the ones C allocated for the caller to free. The program copies, then frees
C's buffer itself (tests/cases/ffi_owned_string,
cookbook):
import { readCString } from "nish:unsafe";
declare function strdup(s: string): CPtr | null;
declare function free(p: CPtr): void;
const owned = (s: string): string => {
const p = strdup(s);
if (p === null) {
return "";
}
const copy = readCString(p);
free(p);
return copy;
};
if.end:
%6 = load i8*, i8** %p.addr, align 8
%7 = icmp eq i8* %6, null
br i1 %7, label %ffi.null, label %ffi.copy
ffi.null:
call void @nish_panic(i8* bitcast ({ i64, [34 x i8] }* @.str.3 to i8*))
unreachable
ffi.copy:
%8 = call i8* @nish.ffi.str(i8* %6)
store i8* %8, i8** %copy.addr, align 8
%9 = load i8*, i8** %p.addr, align 8
call void @free(i8* %9)
-
It is
nish:unsafebecause nothing proves the bytes end. The copy scans for the NUL with the@nish.ffi.strhelper astringresult uses, and a pointer that is dangling, already freed, or to a buffer with no NUL is undefined behaviour: the import is the author's word, asuncheckedGet's is. It is anunsafecall to--deny unsafe, and theCPtrit reads came from adeclare function, which isffi. -
A
NULLpointer panics before the copy, with theffi-nullkind, asreadStruct's does:`readCString` read a NULL pointer, exit 1 (tests/cases/panics_ffi_read_cstring_null). The parameter isCPtr, so an unnarrowed pointer is`readCString` expects an argument of type CPtr, got CPtr | null(reject_unsafe_read_cstring) andif (p !== null)comes first, but adeclare functiondeclared to answerCPtrmay still answerNULL, so the copy tests its pointer: theicmp eqand the branch toffi.nullabove. No proof removes the test, so--deny-panicsrefuses a call in its scope; the copy'soombeside it is allowed. Declare a result that may beNULLasCPtr | nulland narrow it, rather than lean on the panic. -
The copy is new on every call and is the calling function's allocation,
as a
stringresult's is: kept past a loop's pass, returned, stored by a constructor and released with its owner, and refused inside ausing a = arena()block that would release one held outside it (`readCString(p)` may be allocated inside the `using arena()` block,reject_ffi_owned_string_arena). C's buffer is never held past the call, sofree(p)straight after it is safe. -
On wasm32 it lowers to the same helper.
@nish.ffi.stris a byte scan and anish_str_new, which every runtime has, soreadCStringadds nothing to the link; whether the program links there is thedeclare function's question, answered by the libc the target has.
cErrno: the errno a C call left
cErrno(): i32, from
nish:unsafe, answers the
errno the most recent declare function call on this thread left, and 0
before any. In a program that names it anywhere, the compiler sets errno to 0
immediately before every declare function call and copies errno into a
per-thread runtime slot immediately after it returns, before any result is
marshalled; cErrno() reads the slot (tests/cases/ffi_errno,
cookbook,
design):
import { cErrno } from "nish:unsafe";
declare function strtol(s: string, end: CPtr | null, base: i32): i64;
const parse = (s: string): i64 => {
const n = strtol(s, null, 10);
return cErrno() === 0 ? n : -1;
};
ffi.ok:
%2 = getelementptr inbounds i8, i8* %s, i64 8
%3 = call i32* @nish_errno_ptr()
store i32 0, i32* %3, align 4
%4 = call i64 @strtol(i8* %2, i8* null, i32 10)
%5 = load i32, i32* %3, align 4
store i32 %5, i32* @nish_errno_slot, align 4
store i64 %4, i64* %n.addr, align 8
%6 = load i32, i32* @nish_errno_slot, align 4
-
Saved before anything else runs. The copy of a
stringresult, an arena allocation or aconsole.logmay call into libc and move errno, so the save is the first instruction after the call, ahead of theNULLcheck of astringresult. A C function that sets errno and answers a string is read correctly after the copy. -
Zeroed before, so a success reads 0.
strtolreports an out-of-range number only through errno; with the zeroing,cErrno()after an in-range call is 0 unless the callee set it. No setter is exposed. -
Per thread. errno is per thread in every libc, and the slot is
_Thread_localunder--threads(thread_local(initialexec)in the IR), so a thread reads what its own last call left (ffi_errno_threads). -
Program-wide, and free when unnamed. One
cErrnoanywhere makes everydeclare functioncall in every module of the program pay the zero and the save, and none of them is then a tail call. A program that never names it emits none of it. -
Not covered: a host-module import (
--host) is not a C call; the close ausinghandle runs at scope exit is the compiler's call, not the program's, and is not captured. -
Not a global. errno is a macro over a thread-local accessor in every
libc, so there is no
errnosymbol to declare, anddeclare let errno: i32is the parser'sa module holds only `function`, `class`, `interface`, `const`, `type`, `enum` and `import`, found `IDENT`(reject_ffi_errno_global). -
Refused: an argument (
`cErrno` expects exactly 0 arguments, got 1,reject_ffi_errno_arg), a call without the import (NL2456,reject_ffi_errno_import), and a freestanding wasm32 build, which has no operating system:`cErrno` reaches the operating system, and a wasm32 build has none to reach(reject_ffi_errno_wasm).--profile wasilinks wasi-libc and keeps it. -
It is
nish:unsafeand anunsafecall to--deny unsafe, like every export of the module; it reads a defined value and is no panic site.
A u8[] parameter: a buffer and its count
A u8[] or readonly u8[] parameter is a buffer C reads or writes, and a
run of one or more of them is followed by exactly one u64: the byte count C
may touch in every buffer of the run. Pairing the count by position is the
declaration's word, not a proof. fwrite(ptr, size, nmemb, stream) touches
size * nmemb bytes, so declared as fwrite(buf: readonly u8[], size: u64, nmemb: u64, f: CPtr) it pairs size with the buffer, and fwrite(buf, 1, 1000, f) on a 10-byte array overflows it. The safe spelling declares the size
as the count and passes 1 as nmemb: fwrite(buf: readonly u8[], n: u64, one: u64, f: CPtr), called as fwrite(buf, n, 1, f). Nothing stops a call from
passing another nmemb, which is the same class of mistake as declaring free
with two parameters.
declare function memcpy(dst: u8[], src: readonly u8[], n: u64): CPtr;
declare function getentropy(buf: u8[], n: u64): i32;
declare function compare(a: readonly u8[], na: u64, b: readonly u8[], nb: u64): i32;
-
memcpy(dst, src, n)is one run of two buffers sharing a count, and(a, na, b, nb)is two runs with a count each (tests/cases/ffi_buffer_declared). The same array passed twice in one run,memcpy(a, a, n), is not refused: C sees two pointers to one block, and overlappingmemcpyis undefined in C (memmoveis the one that allows it). -
readonly u8[]is C'sconst. Au8[]parameter says C may write the bytes; areadonly u8[]says it only reads them. C can cast theconstaway, and the compiler cannot check that. -
A run with no count after it is refused, with
takes the buffer `md` with no count after it: a run of `u8[]` parameters is followed by one `u64`(reject_ffi_buffer_length_missing,SHA256(d, n, md)'s fixed-size output), and so is a count of any other type,and the parameter after a run of `u8[]` buffers is their byte count, a `u64`(reject_ffi_buffer_length_type). -
Only
u8crosses.i32[]orf64[]isthe only array a declared C function takes is a `u8[]` or a `readonly u8[]`(reject_ffi_buffer_element). -
A buffer is never null and never a result.
u8[] | nullis refused (reject_ffi_buffer_nullable, andreject_ffi_buffer_nullable_in_runfor one inside a run): it has no length to hold the count to, so theNULL, 0call C allows is declared with aCPtr | nullparameter. Au8[]result is refused by the return rule above (reject_ffi_buffer_return). -
A call passes the array's data block, and checks the count against every
buffer of its run first. Each buffer is lowered to its data pointer, a
uint8_t *(i8*in the IR), and the count is compared, unsigned, with the buffer's length before C runs. A count larger than one of them stops the program with anffi-lengthpanic,`write`: argument 3 (n) counts more bytes than argument 2 (buf) holds, exit 1 (tests/cases/panics_ffi_length, andpanics_ffi_length_memcpyfor the shorter buffer of a run). The count is au64, sotoU64(0) - toU64(1)is the largest count there is, and fails (panics_ffi_length_wrapped). -
The pointer and the length are read after the last argument is
evaluated, so C sees the array as it is at the call: an argument that
pushes or pops before it is seen (
tests/cases/ffi_buffer_late_load,panics_ffi_length_shrink,panics_ffi_length_inline_pop). -
One count is proved:
toU64(b.length)of the buffer itself, wherebis aconstlocal or a parameter named alone and no later argument may resize an array. Such a call carries no check and is not a panic site, soread(fd, buf, toU64(buf.length))compiles under--deny-panics. A field path, an alias of the buffer, another array's length and a literal are checked (tests/cases/ffi_buffer_count_proved).--deny-panicsrefuses any other count and names the one it accepts:pass `toU64(buf.length)` of the buffer itself, which needs no check. -
C borrows the bytes for the call, and writes through a
u8[]. As with astring, the pointer is lent for the length of the call and C may not keep it, so a buffer made in the call's arena scope is released after it (tests/cases/ffi_buffer_read). Au8[]argument is written by the call and areadonly u8[]one is not, and the compiler records it as it records a store. An exported function that hands its ownu8[]parameter to such a call writes that parameter: its header drops theconst, the wasm glue copies the bytes back, and itsdefinelosesreadonly(tests/cases/ffi_buffer_export_written). A task a scope spawns may not reach such a call: C's body is unseen, so the call may write memory the parent can see, and the task is refused withwrites memory its caller can see(reject_ffi_buffer_call_spawn, andreject_ffi_buffer_scope_race,reject_ffi_buffer_scope_field_raceandreject_ffi_buffer_scope_element_racefor a call reached through a function). -
Every kind of call site lowers the same way: a call by name, through a
function parameter bound to the declaration, in a constructor reached by
newand in a method (tests/cases/ffi_buffer_call_sites), and a call that passes a buffer is refused by--deny ffilike any other foreign call. -
On wasm32 a
u64count is wider than C'ssize_t. The data pointer is 32 bits there and the length check is the same 64-bit compare, but a C function whose count is asize_ttakes 32 bits: declare its countu64only on a target wheresize_tis 64 bits, until a later release gives the count a target-width type. A freestanding wasm build (--target wasm32) refuses adeclare function, so a buffer declaration is for--profile wasithere (see "Host modules" below).
A handle class: a class that owns a CPtr
A class with a CPtr or CPtr | null field is a handle class: it owns the C
resource the pointer names, and its [Symbol.dispose] names the C function
that closes it (tests/cases/ffi_handle_positions, docs/wp53-ffi-buffers-handles.md
§4a–§4d).
declare function fopen(path: string, mode: string): CPtr | null;
declare function fclose(p: CPtr): i32;
declare function fgetc(p: CPtr): i32;
class File {
readonly file: CPtr | null;
constructor(path: string) {
this.file = fopen(path, "r");
}
read(): i32 {
const p = this.file;
if (p === null) {
return -1;
}
return fgetc(p);
}
[Symbol.dispose](): void {
const p = this.file;
if (p !== null) {
fclose(p);
}
}
}
const first = (f: File): i32 => f.read();
using of a handle class closes it at every exit of its block, once.
using f = new File(p) opens f as soon as its initialiser returns, before a
later declarator's initialiser runs, and calls the close its
[Symbol.dispose] names on every way control leaves the block: falling off
its end, return, break, continue, and an orReturn() that propagates,
including one in a later declarator's initialiser. A return computes its
value first and closes after it, so return twice(n) inside the block is not
a tail call. Each declarator opens before the next one's initialiser runs.
Falling off the end, break and continue close the declarators in the
reverse of the order they opened, beside an arena, a scope or a lock in the
same using. A return or a propagating orReturn() joins every scope()
before its value is computed (the value may read what the tasks stored), then
closes the handles and gives back the locks, innermost first, then releases
the arenas (ffi_handle_file, ffi_handle_exits, ffi_handle_arena,
ffi_handle_shapes, tests/link/ffi_handle_mutex,
tests/link/using_declarator_exit). The close is inlined as the dispose spells
it: the field is loaded and the C function called with it, behind a null
test only when the dispose writes one (const p = this.f; if (p !== null));
a dispose that is the single call close(this.f) calls it with the field as
it is, null included (ffi_handle_shapes). The using is where the program reaches ffi, so
--deny ffi refuses it and names the close (caps_ffi_handle,
reject_ffi_handle_deny).
A panic and process.exit do not close: the process ends, and the operating
system takes back the descriptor. What it does not take back is what a C
library still buffers. panic, expect and a failed range entry end through
exit(1), which flushes stdio, but a failed check and out of memory end
through _exit(1), which does not, so bytes written to a FILE * with
fwrite and not yet flushed are lost on a bounds panic. Flush (fflush)
before code that may panic when those bytes matter.
-
The class. Exactly one
CPtrorCPtr | nullfield,readonlyand with no initialiser (reject_ffi_handle_two_fields,_field_mutable,_field_initialiser). No type parameters, noimplements, noexport(_generic_class,_implements,_export), and nostatic readonly json(_json). -
The write. The constructor writes the field once, as
this.file = g(...)at its top level, withgadeclare functionand no exit before it (_field_copy,_field_copy_twice,_field_copy_branch,_field_copy_non_foreign). -
The read. The pointer is read only as
this.fileinside a method of the class, passed straight to adeclare functionor bound once to aconst(_field_read_outside,_field_other_instance,_field_alias_copy,_field_alias_let,_field_position):The pointer `file` of handle class `File` cannot be used here. -
The close.
[Symbol.dispose](): voidisclose(this.file);orconst p = this.file; if (p !== null) { close(p); }for adeclare functioncloseof one parameter (_no_dispose,_dispose_body,_dispose_arity,_dispose_arg_type,_dispose_unknown,_dispose_forms,_dispose_shapes). A class declares one, with no modifier (_dispose_twice,_dispose_static). Nothing else may call that close on the field, in a method or in the constructor (_close_call), and[Symbol.dispose]is never called by name (_dispose_call). -
Where a
Filemay be. It is opened byusing(once lowered), called (f.read()), or lent to a parameter of typeFile, which may lend it on. Everything else isA `File` cannot be used here(_position):newoutside ausinginitialiser,thisleaving the constructor, a copy into another binding, an array element, an object literal, a capture, a builtin's argument, a comparison (_new_outside_using,_this_escapes,_alias,_array,_object_literal,_closure_capture,_builtin_arg,_walk_compare). -
Where
Filemay be written. Only as the type of such a parameter. A local's annotation, a field, a return, aResultarm, a nullable, a type argument and an exported function's parameter are`File` is a handle class, and its name may be written only as the type of a parameter that borrows one(_type_position,_field,_return,_result,_nullable,_type_argument,_exported_param). -
Not after its close, and not kept past a lend. A
usingbinding's scope is its block, which ends where the close is emitted, so a handle named after its block — after it, in the loop's next iteration, in adoloop's condition, in aforloop's update, in a latercase, in theelse, later in a constructor — names nothing:Unknown builtin `f.read`orUnknown identifier `f`(_use_after_close,_use_after_close_loop,_use_after_close_do_while,_use_after_close_for_update,_use_after_close_switch,_use_after_close_else,_use_after_close_ctor). Ausingwritten straight into acaseclause is refused by the clause rule (_use_after_close_switch_clause). A lent handle cannot be kept by its callee. The value rule refuses a local (let saved = f,_use_after_close_keep), an alias (_alias), an array (_array) and ascope()task (_scope_task); the type rule refuses a return (_return) and a type argument (_type_argument), among them aMapor aChannelof the class (_use_after_close_keep_map,_use_after_close_keep_channel); both refuse a field of another class, set in a method, throughnewor ash.l = this(_use_after_close_keep_field,_walk_ctor_param,_use_after_close_keep_this). Some shapes are refused by a general rule first: a closure by the arrow-capture rule (_closure_capture), and a module binding by the absence of a top-levelletand the rule that a module constant is never assigned (_use_after_close_keep_module,_use_after_close_keep_module_expr). In its own constructorthisis half built, and is neither lent, called, stored nor aliased (_ctor_this_lend,_ctor_this_receiver,_ctor_this_store,_ctor_this_alias).
Two things stay C contracts, as a buffer's count does. The class's methods
pass the field only to functions that use it and return, and that neither close
what it owns nor hand it back: freopen, a close through a C wrapper with
another name, close(fileno(p)), or a C function returning its argument are
not seen by the rules above. And the constructor's foreign call returns a
pointer the object owns: two objects built from one shared pointer would both
close it.
A declare class: a C struct, lent by pointer
A declare class is a C struct: its fields are laid out in declaration order,
each at its natural alignment, and the size is rounded up to the widest
alignment, as a C compiler lays out the same fields. new Name() takes no
arguments and zero-fills it, and a declare function parameter typed Name
borrows it as a struct Name * (tests/cases/ffi_struct_param,
ffi_struct_layout, cookbook,
docs/wp27-ffi.md §9b):
declare class Timespec {
sec: i64
nsec: i64
}
declare function clock_gettime(clock: i32, ts: Timespec): i32
const now = new Timespec()
clock_gettime(1, now)
console.log(now.sec)
%struct.Timespec = type { i64, i64 }
declare i32 @clock_gettime(i32, %struct.Timespec*)
%Timespec.obj = alloca %struct.Timespec, align 8
%0 = bitcast %struct.Timespec* %Timespec.obj to i8*
call void @llvm.memset.p0i8.i64(i8* align 8 %0, i8 0, i64 16, i1 false)
store %struct.Timespec* %Timespec.obj, %struct.Timespec** %now.addr, align 8
%1 = load %struct.Timespec*, %struct.Timespec** %now.addr, align 8
%2 = call i32 @clock_gettime(i32 1, %struct.Timespec* %1)
-
newzero-fills every byte, padding included, with onellvm.memsetof the struct's size: C may compare or hash the whole struct, and azeroinitializerstore need not write the padding (ffi_struct_layout). -
A
CPtrfield is the target'svoid *: eight bytes natively, four under--profile wasi(wasm32), where the struct's size and offsets are the ones clang gives the same C struct for that target (tests/ffi-struct.js). -
A field is
i32,i64,u8,u16,u32,u64,f32,f64,CPtr,CPtr | null, or anotherdeclare class, embedded by value — C'sstruct timespec it_value;, not a pointer to one. Anumberfield is refused whatever the mode and however it is spelled — in parentheses, through atypealias or a chain of them, imported or not — since--number-modepicks its width and a C struct has one layout. The resolver decides it where it meetsnumber, so no spelling escapes; an alias ofi32stays a field:has type `number`, whose width `--number-mode` chooses(NL2755,reject_ffi_struct_field_number,_paren,_alias,_alias_paren,_alias_chain;tests/ffi-struct.js: the imported alias, thei32alias). Any other type isField `ok` of `declare class Flags` has type `boolean`, which has no C layout(NL2752,reject_ffi_struct_field_type,reject_cls_declare,reject_ffi_struct_field_array,reject_ffi_struct_field_class): aboolean, astring, an array and a Nish class have no C spelling the compiler could promise. A struct cannot embed itself, directly or through another (NL2749,reject_ffi_struct_self,reject_ffi_struct_cycle). -
A field is plain and has no initializer, and the class has nothing but
fields: no
readonly,staticor optional field (NL2747), no initializer, becausenewzero-fills (NL2748), no constructor, method, accessor or other member such as an index signature (NL2751,reject_ffi_struct_index_signature), no type parameter (NL2741,reject_generic_declare_class), noextendsorimplements(NL2750), and noexport(NL2746) — each pinned by areject_ffi_struct_*case. -
It stays in its module. It cannot be a parameter or the return type of an
exported function or an exported class's method, or a field of an exported
class or interface:
`declare class Point` cannot be a parameter of an exported function(reject_ffi_struct_export_param,_return,_method,_field,_interface). A type that holds one is refused there too, however deep — throughResult,T | null, an array, a generic instance or a class of this program:`Wrap` holds a `declare class Point` and cannot be a parameter of an exported function(NL2756,reject_ffi_struct_export_result,_generic,_holder,_holder_nullable). So no.d.tsor N-API binding ever describes one, and--emit-header, which lists every class of the program, declares a class that holds one only as an opaquetypedef struct Holder Holder;(tests/ffi-struct.js,docs/wp27-ffi.md§9g). -
It is a native build's. A wasm module a page loads — freestanding wasm,
or any build that writes
--emit-dts— links no C to lend a struct to, and its IR is laid out for wasm32 later:`declare class Point` declares a C struct, and a wasm module a page loads links no C to lend it to(NL2757,tests/wordings/nl2757_declare_class_emit_dts,nl2757_declare_class_wasm32). A--profile wasibuild is wasm32 from the start and keeps it. -
It is built with
newand nothing else.new Point(1)is refused as for any class without a constructor (reject_ffi_struct_new_args), and an object literal typed as one isan object literal cannot lay out a C struct(NL2744,reject_ffi_struct_literal). -
C borrows it for the length of the call, as it borrows a
string: it may read and write the fields until it returns, and may not keep the address. The escape analysis treats the argument as a read, so an instance that does not otherwise escape stays in the caller's stack frame; a C function that keeps the pointer (anaio_readcontrol block, a struct handed to a thread) is outside the contract. Its accesses carry no TBAA tag, so a field read after the call sees what C wrote, and a function of this program that passes its parameter on to C writes through that parameter, so the parameter is notreadonlyand its caller's read sees the write too (ffi_struct_param). -
Name | nullpassesNULLfornull, under the rule aCPtr | nullparameter follows: the literalnull, or a value narrowed with!== null(NL2539,reject_ffi_struct_nullable_held). -
An embedded struct is part of its owner.
stamp.at.secreads and writes a field in place, andclock_gettime(1, stamp.at)lends C the address insidestamp; those are the only two uses. Holding it —const at = stamp.at, or passing it to a function this program defines — iscannot be held as a value(NL2743,reject_ffi_struct_nested_bind,reject_ffi_struct_nested_nish_arg), and replacing it isa `declare class` embedded in its object is written field by field(NL2745,reject_ffi_struct_nested_assign). -
It is not a return type or an array element. C returning a struct by
value, or a pointer into memory it owns, leaves the copy with no owner on
this side:
a C struct is only lent to C by pointer: take a `Point` parameter for C to fill(NL2742,reject_ffi_struct_return); a struct C owns is copied out withreadStruct(below) from theCPtrC returns. An array of them would be a C array the program lays out element by element, and iscannot be an array element(NL2740,reject_ffi_struct_array,reject_ffi_struct_array_literal,reject_ffi_struct_array_new), however the array's type is spelled. -
Its fields are C's to fill. Typed JSON does not decode into one or
encode one:
a `declare class`: C reads and writes its fields(NL2753,reject_ffi_struct_json). A spread copies a flat one's fields into a class of this program, but not one that embeds another, whose embedded struct is not a value:which a spread cannot copy(NL2754,reject_ffi_struct_spread). -
--emit-tsrefuses a module that declares one, with the same message as for adeclare function(NL4010): it is C's layout, and JavaScript has none.
readStruct: a struct out of memory C owns
readStruct(p) from nish:unsafe copies the struct at a CPtr into a new
instance of the declare class its context names, and the copy is the
program's (tests/cases/ffi_struct_read,
cookbook,
docs/wp27-ffi.md §9h). It is how a struct C keeps comes back:
localeconv()'s struct lconv, getpwnam's struct passwd, each a buffer
the next call overwrites.
import { readStruct, readCString } from "nish:unsafe"
declare class Lconv {
decimalPoint: CPtr
thousandsSep: CPtr
}
declare function localeconv(): CPtr
const lc: Lconv = readStruct(localeconv())
console.log(readCString(lc.decimalPoint))
%struct.Lconv = type { i8*, i8* }
%0 = call i8* @localeconv()
%1 = icmp eq i8* %0, null
br i1 %1, label %ffi.null, label %ffi.copy
ffi.null:
call void @nish_panic(i8* bitcast ({ i64, [48 x i8] }* @.str.0 to i8*))
unreachable
ffi.copy:
%2 = call i8* @nish_alloc_struct(i64 16)
call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 8 %2, i8* align 1 %0, i64 16, i1 false)
%3 = bitcast i8* %2 to %struct.Lconv*
store %struct.Lconv* %3, %struct.Lconv** %lc.addr, align 8
-
The layout comes from context, as
Ok(v)'s does: an annotation, a parameter, a return type or an assignment's target,| nullstripped. With none it is`readStruct(p)` needs a contextual `declare class` type to copy into(NL2764), and a context that is not a layout is`readStruct(p)` copies into a `declare class` layout, and Plain is not one(NL2765,reject_unsafe_read_struct_context). A type argument is not written at a call site:readStruct<Lconv>(p)is refused with the annotated spelling in the message (NL2763,reject_unsafe_read_struct_type_args). -
One copy, the layout's size: the arena allocation
newmakes, then onellvm.memcpyof the struct's bytes, padding included, from the source atalign 1, since C promised the struct and not its address's alignment. The size is the target's: 16 bytes for{ CPtr; CPtr }natively, 8 under--profile wasi(tests/ffi-struct.js). After the call C may free or rewrite its struct; the copy is an instance like anynewone, and a copy made in a callee and returned is held by its caller. -
It does not copy deep. A
CPtrfield of the copy is still C's pointer, read withreadCStringwhile C keeps it live. -
A
NULLpointer panics before the copy, with theffi-nullkind:`readStruct` read a NULL pointer into a `Lconv`(tests/cases/panics_ffi_read_struct_null). The argument is aCPtr, so aCPtr | nullis narrowed first (`readStruct` expects an argument of type CPtr, got CPtr | null,reject_unsafe_read_struct_null), but adeclare functiondeclared to answerCPtrmay still answerNULL, so the copy tests its pointer: theicmp eqand the branch toffi.nullabove. No proof removes the test, so--deny-panicsrefuses a call in its scope. -
The rest of the pointer is not checked: not that it is live, not the
size it covers. The layout must be C's struct or a prefix of it. That is why
it is in
nish:unsafe: refused by--deny unsafeand recorded by--emit-checked. -
There is no
writeStruct. A struct C should fill is lent to it, as above.
What a declaration cannot say
Each of these is refused, and each refusal is pinned by a case:
-
A variadic C function, spelled variadically. A rest parameter is
Rest parameters are not supported herebefore its type is read,...bytes: u8[]included (reject_ffi_variadic,reject_ffi_buffer_rest). A variadic function declared with a fixed arity,printf(fmt: string, x: i32), is accepted and is the declaration's contract: the compiler cannot tellopenis variadic from its name. -
syscall, by name:`declare function syscall` is refused(reject_ffi_syscall). It is variadic and bypasses libc; declare the libc function that wraps the system call. -
A callback. C never calls into Nish: an arrow passed to a declared C
function is refused (
reject_ffi_callback), and a function reference is a tag only this program's dispatchers read, not an address, so it is not aCPtreither. -
A read through a
CPtr.p[0]isCannot index a value of type CPtr(reject_ffi_pointer_deref);readCStringandreadStructabove are the two reads. - Inline assembly.
asm("nop")is an unknown function (reject_ffi_asm). - An allocator from C memory.
arena()takes no argument, sousing a = arena(p)is refused (reject_ffi_allocator).
A foreign declaration has no body (reject_ffi_body) and cannot be
exported (reject_ffi_export), because export offers other modules a
function this module defines.
A declared function's symbol is its name as written, in whatever package
declares it. A function a package defines carries the package's prefix
(@probe.probeAbs), but a declare function names the C function, and C has
one namespace, so a package or a std/ module that declares abs calls @abs
exactly as the program's own modules do (tests/link/ffi_package_declare).
Two declarations of one name must agree: the same parameter and return
types, which is what a C prototype is, so parameter names and the aliases a
type is spelled through do not count (tests/link/ffi_package_declare_same).
Agreeing declarations are one C function, whether they are in two packages or
in two modules of the same package, which was refused as a duplicate symbol
before. Declarations that disagree are refused, naming both:
C has one namespace, so every `declare function` of one name must agree on its parameter and return types. A buffer's readonly, a | null and the
number of parameters each count (tests/link/ffi_package_declare_clash,
tests/link/ffi_package_declare_mismatch). So is a function whose emitted
symbol is a declared name: an exported function of the program's own package,
or a private one whose name is not reserved to C:
a `declare function` binds the C symbol of its name in every package, so rename the defined function (tests/link/ffi_package_declare_root_clash).
A private function named like a C library function is emitted as abs$, and a
generic one's instantiations as labs$i32, so neither takes the C symbol and
both are accepted (tests/link/ffi_package_declare_apart). The two refusals
are refused rather than linked because the whole-program analysis keeps one
set of facts per symbol, and the two would share it.
The declaration carries no LLVM attributes, and its callers pay for it.
Nothing about a body the compiler cannot see is provable, so a function that
calls C loses readnone and willreturn, and so does everything above it in the
call graph; a function that calls no C keeps both (tests/cases/ffi_scalar).
nounwind is the one exception, and it is a decision rather than a proof:
unwinding out of a foreign call is undefined in this language, the same position
clang takes compiling C, because there is no throw and no landing pad with
which to do anything else. See
docs/wp27-ffi.md §2 and the
cookbook entry.
A program that calls C is no longer one the compiler can reason about end to end. That is the trade the feature is: the whole-program fact fixpoint is the performance thesis, and a foreign call is a hole in it.
And it is outside the differential oracle by construction. The oracle
runs a program's frozen JavaScript rewrite against runtime/shim.mjs, on the
premise that the same source means the same thing in both worlds; that premise
does not hold for a source whose meaning is "whatever this C function does"
(wp27-ffi.md §5). The evidence for an FFI program is arranged the
other way round: a golden .ll, an llvm-as pass, a native round trip against
real libc, and the attribute assertions.
Host modules: the functions a page supplies
A wasm module calls a function of the page that loads it by importing it
from a host module: a TypeScript file of the page's, named by --host <file> or by the root package.json's "nish": { "host": [...] }
(tests/host-import, docs/wp50-wasm-host.md §3). An import from any other
file is an ordinary Nish import.
// host.ts, the page's
import type { f64, u32 } from "nish";
export type Canvas = HostHandle<"canvas">;
export function fill(c: Canvas, rgb: u32): void { canvases[c].fill(rgb); }
export function nowMs(): f64 { return performance.now(); }
declare void @fill(i32, i32) #1
attributes #1 = { "wasm-import-module"="host" "wasm-import-name"="fill" }
-
Only the exported signatures are read. Each
export function's parameter and result types, each exportedenum, and eachexport type X = HostHandle<"kind">. A function body is skipped as a balanced run of tokens, so it may use anything TypeScript has. Text the lexer cannot skip, a regex literal or JSX, refuses the file withmove that code into a module the host file imports(NL3049); other statements are skipped unread. -
What crosses. The integer and float widths,
boolean, a ranged integer, an enum, aHostHandle, astringparameter, andvoidas a result.i64andu64are abiginton the page's side (import type { i64 } from "nish"). Refused, each naming the explicit form: an object, a class, a function,unknownorany(NL2661), a generic function (NL2662), aPromise(NL2663) and astringresult (NL2667): the page writes text into module storage instead (nish/host/text). An array crosses only as a buffer window, below. -
A
stringargument is its UTF-8 bytes, lent for the call. It lowers to two wasm values, the address of the string's bytes and their count, read from the string's own header, so nothing is copied on the module's side; the loader's wrapper decodes them withTextDecoderinto a JS string before the page's function runs, and an invalid sequence becomes U+FFFD. The string may be a literal, a concatenation or a template; one the module built lives in the export's arena, which is released after the export returns (tests/cases/host_import_string;tests/run.js, sectionhost-import-text; cookbook):// web/host.ts export function say(s: string): void { console.log(s); } // app.ts say(`hello ${n}`);declare void @say(i8*, i32) #2 %1 = call i8* @nish_str_concat(i8* bitcast ({ i64, [7 x i8] }* @.str.0 to i8*), i8* %0) %2 = bitcast i8* %1 to i64* %3 = load i64, i64* %2, align 8 %4 = trunc i64 %3 to i32 %5 = getelementptr inbounds i8, i8* %1, i64 8 call void @say(i8* %5, i32 %4) -
Each host function is a declaration with the wasm import attributes: module
host, its own name. It carries no other attribute, and its callers losereadnoneandwillreturnas for a C declaration.wasm-ldmakes it an import with no--allow-undefined. -
A host module exists only in a freestanding wasm build. A native or
--target wasm32-wasibuild that imports one is refused (NL3050). A build that makes a page module (--target wasm32, or--emit-dtswith no--target) refuses adeclare function(NL2666): the module links no C, so the declaration would be an import the page does not supply.--target wasm32-wasi --emit-dtslinks C, so it keepsdeclare functionlegal. -
The loader and its types.
--emit-dtsaddsImports = { host: typeof import("./host") }to the.d.ts, andloadtakes it. The loader passes a function through by reference when every value crosses as it stands, and wraps it once, atload, when a value converts. Au32oru64argument arrives signed and is read unsigned (>>> 0,BigInt.asUintN(64, …)); au8oru16argument needs nothing, since the module zero-extends it. Au8oru16result is masked, abooleancrosses as 0 or 1, and a ranged integer result is checked by the loader'srangeOut(RangeErrornaming the function). -
--emit-tskeeps the import as written, pointing at the host file, so the TypeScript reading calls the same functions. -
A call to a host function is an
fficall:--deny ffirefuses it and--emit-capabilitiesnames it.
An array crosses as a buffer window: the array, then at: u32, n: u32
(tests/host-window, docs/wp50-wasm-host.md §5.3). The host lends the page
the elements [at, at + n) of the array the program passes, in place, with no
copy:
// host.ts, the page's
import type { u32 } from "nish";
export function doubleF32(xs: Float32Array, at: u32, n: u32): void {
for (let i = at; i < at + n; i++) xs[i] *= 2;
}
// the program
const DATA = new Float32Array(64)
export const tick = (): void => { doubleF32(DATA, 16, 8) }
-
Six elements lend:
u8[],u16[],u32[],i32[],f32[]andf64[], theirreadonlyforms and their typed-array names. Any other array is NL2681 (a buffer window lends only `u8[]`, `u16[]`, `u32[]`, `i32[]`, `f32[]` or `f64[]` (or the typed array of each): copy the values into one of those). The two parameters after it must be there (NL2682,a buffer window with no `at: u32, n: u32` after it) and beu32, the width of a wasm32 address (NL2683). NL2664, which refused every typed array before windows existed, is retired. -
The window is checked in the module, before the page runs.
atandnare zero-extended and added ini64, so a sum that wraps in 32 bits still fails, and held against the array's length; past the end is anffi-lengthpanic,`doubleF32`: argument 2 (at) plus argument 3 (n) reaches past the end of argument 1 (xs), numbered and named from the host function's own parameters (cookbook). A call that lends0, toU32(xs.length)of a local arrayxsis proven in bounds and emits no check. A window of module storage always keeps the check, even0, toU32(DATA.length): the range pass proves facts about locals, not about storage. -
What the page gets is a view of all of memory and an index. The
loader's adapter passes a whole-memory view of the element type, the
array's first element plus
atas the index, andn, so the host function'sxs[at]is the program'sDATA[at]. A page that indexes outside[at, at + n)reaches the rest of the module: the page is trusted, and it reads only its window (security record). - Module storage lends to a window like any array of a lending element, and so does a local array. A record storage does not.
Module storage is the page's view of the module. Under
--profile wasm-host the link exports each storage's data, and --emit-dts
types what the loader builds from it:
-
layout, aLayoutwith one entry per storage, keyed by its name (by its symbol when two modules export the same name):{ at, length, stride, fields? }, whereatis the byte address inmemory,stridethe element's size, andfieldseach record field's byte offset within the element.length,strideand each offset are literal types in the.d.ts, so a page that hard-codes one, or reads a field the module does not have, failstsconce the layout changes.atis read from the instance at load and never moves. A record field is read through the whole-memory view of its width:views().f32[(layout.bodies.at + i * layout.bodies.stride + layout.bodies.fields.x) >> 2]. -
views(), the whole memory asbufferand as each typed array (u8,i8,u16,i16,u32,i32,f32,f64,i64,u64), andstorage.<key>, a typed view of each scalar storage (abooleanone asUint8Array, an enum or range asInt32Array, aHostHandleasUint32Array).views()answers the same object until a growing memory replacesmemory.buffer, so a page calls it again after any call into the module rather than keeping one view across calls.
Under --profile wasm-host a panic is the page's to report, in a
program with module storage or a
host module
(tests/panic, docs/wp50-wasm-host.md §6). The nish.panic import,
NishPanic, the dead module, NishReentry (below), and the onPanic and
sites options of load and loadSync are for such a program only;
loadSync itself is in every loader.
Any other program gets the .wasm, loader and .d.ts it got before: a panic
is a bare RuntimeError trap, the module stays usable, and its next call
runs. The module of such a program imports one function
from module nish, panic, and every runtime panic calls it with its kind
and the words it would print natively; the loader turns that into an
exception:
-
The call that panicked throws
NishPanic, anErrorwhosemessageis the native text (index out of range: 9 >= 4,slice out of range: [a, b) of length len, an overflow's text,out of memory,the program exited with code 3, orpanic(…)'s own words) and which carrieskind(panic,index,slice,overflow,memory,exit,divide, ortrap,reentryorexceptionfor a module that died some other way) andsite, the panic site's number (-1 for none). An array'sxs.sliceout of range throws withkindpanic, notslice: its check isstd/array.ts's ownpanic(...), though the message reads as stringslice's does. -
Before it throws, the loader calls
onPanic.load(bytes, { onPanic, sites })takes the reporter; the default,reportPanicfromnish/page, writesnish: <message>toconsole.error, as a native program writes it to stderr. An exception the reporter throws is swallowed: theNishPanicis thrown either way. -
sitesnames where. The wasm-host link writes<out>.sites.json(app.wasm.sites.jsonfor-o app.wasm),[{ kind, at }]with one entry per panic site, indexed bysite. Passed toload(parsed), it ends each message withat path:line:col; without it the message is the native text alone. Under--profile wasmthe module imports nothing and each panic is a bare trap, with no message and no site; the same loader still marks the module dead on it, so later calls throw as above. -
A panic ends the module. Its memory survives, as it was, for a page
that reads it, but every later call throws
NishPanic(the module is dead: …) before the module runs. An exception a host import throws out of a call ends the module the same way, since the call it interrupted never finished. -
A bad argument does not end it. The loader checks every argument's
shape before the module runs: a number of the wrong type or range, a
Resultthat is not an object with a booleanokand a payload of its arm's type, an array argument whose brand (read off%TypedArray%.prototype, notinstanceof) is not the typed array it names. Each throws aTypeError; an error aResultargument's getter throws reaches the caller as it is. Either way the module never ran and stays alive (tests/run.js, sectionpanic:badarg,badshape).
One call at a time. A host import that calls an export while a call into
the module is still running gets NishReentry (a call into the module was made while another call into it was running), and the inner call never runs.
The reporter is different: the module is already dead when onPanic runs, so
an export it calls throws NishPanic (the module is dead: …). The outer call's state is half changed at that point, which
is why the loader refuses rather than runs it (docs/security/wasm-host.md,
WH-3). Left uncaught, the NishReentry escapes the outer call too, and like
any exception out of a call it ends the module. A page that needs the
module's answer schedules the call for after the import returns. In a module
that links the runtime, a call also gives back, once its result is read,
everything it allocated in the arena, so a long-lived page does not grow the
module's memory call by call.
HostHandle<K> is a nominal i32 for an object the page keeps
(docs/wp50-wasm-host.md §4). K is a string literal type, and
HostHandle<"a"> and HostHandle<"b"> are different types, here and under
tsc, where runtime/nish.d.ts makes it a branded number or 0, so tsc
also refuses any number but 0 where a handle is wanted. A handle supports === and
!== against a handle of its kind or 0 ("none"), and is stored wherever a
scalar is; it has no arithmetic and no conversion to or from a number, so it
is made only by a host function that returns one. A field's or a parameter's
default may only be 0 (NL2670). A second type argument, or
a kind that is not a string literal of letters, digits and _, is NL2669, and a string literal type
anywhere but a HostHandle's kind is NL2668. It lowers to
i32, so a program with no host module emits nothing new.

