nish/net/prefork
Run a server on every core: a supervisor process that starts one worker process per core, restarts one that dies, and stops them all on SIGTERM or SIGINT (docs/wp42-multicore-server.md §2.3).
import {
PREFORK_CHANNELS_FLAG,
PREFORK_LISTENER_FLAG,
PREFORK_SPAWN_FAILED,
PREFORK_STOPPED,
PREFORK_UDP_FLAG,
PREFORK_WORKERS_EXITED,
PREFORK_WORKER_FLAG,
PreforkConfig,
preforkArgs,
preforkIndex,
preforkListener,
preforkServiceChannel,
preforkServiceChannelOf,
preforkSupervise,
preforkSupervisorChannel,
preforkUdpCount,
preforkUdpListener,
preforkWith,
preforkWorkerCount,
preforkWorkers,
} from "nish/net/prefork";
import { tcpListenFlags } from "nish:net";
import { preforkWorkers } from "nish/net/prefork";
export const main = (): number => {
const worker: i32 = preforkWorkers(process.argv);
if (worker < 0) { return worker === -1 ? 0 : 1; } // the supervisor, done
const listener = tcpListenFlags("0.0.0.0", 8080, 1024, 1); // SO_REUSEPORT
// ... the single-threaded readiness loop, unchanged ...
};
preforkArgs
args (process.argv) without the worker flag and the channels, listener and UDP flags before it, the same in every process.
preforkIndex
The worker index args carries in its last element, or -1 when none.
preforkListener
In a worker, the listener the supervisor bound for it (listenPort); -1 when none.
preforkServiceChannel
In a worker, its end of its channel to the service worker; -1 in the service worker or when none.
preforkServiceChannelOf
In the service worker, its end of the channel to worker index; -1 for itself or elsewhere.
preforkSupervise
preforkWith, calling onChannel(state, index, fd, events) in the supervisor whenever worker index's supervisor channel fd is readable or hung up: events is pollWait's, 1 readable and 4 added on a hang-up or error.
preforkSupervisorChannel
In a worker, its end of its supervisor channel; -1 when it has none.
preforkUdpCount
In a worker, how many workers share its steered UDP port (listenUdp): the QuicServerConfig.workerCount to pair with preforkIndex as its workerIndex. 0 exactly when preforkUdpListener answers -1.
preforkUdpListener
In a worker, the UDP socket the supervisor bound for it (listenUdp); -1 when none, or when the flag is not the supervisor's <fd>,<N>.
preforkWith
In a worker, its index.
preforkWorkerCount
How many workers to start: env (NISH_WORKERS, null when unset) when it is a positive whole number, else requested when positive, else the platform's default.
preforkWorkers
preforkWith with the default PreforkConfig.
PreforkConfig
The supervisor's numbers; every one is checked by preforkWith.
One program, two roles
preforkWorkers(process.argv) answers in two kinds of
process. In a worker it answers the worker's index, 0 to N - 1, at once,
and the program goes on to bind its listener and serve. In the process the
user started it supervises, and answers only once every worker has exited:
-1 after a shutdown, -2 when the workers could not be started or the
supervisor could not listen for signals. Each worker binds its own listener
with tcpListenFlags' flag 1 (SO_REUSEPORT), so on Linux the kernel
spreads new connections across them and no state is shared:
a worker is the single-threaded server it was before, Arena.used() flat
per worker.
How a worker is told
The supervisor runs the program again with
--nish-prefork-worker=<index> appended to its arguments, through spawn;
on Linux it runs /proc/self/exe, so a program started through PATH or a
relative path finds itself. preforkArgs(process.argv) answers the
arguments without that flag, for a worker that reads its own; the module
takes process.argv from its caller because only a program with a main
has one. The environment is inherited whole, and every descriptor the
supervisor holds is close-on-exec, so a worker starts with none of them
but the channel ends below (docs/security/runtime.md, RT-17, RT-19).
Channels
With supervisorChannels on, each worker is started with one
end of a fresh unixSocketPair (channelKind: stream by default, since a
seqpacket pair needs Linux and the supervisor panics on one elsewhere) and
the supervisor keeps the other; preforkSupervise hands each readable end to
a callback. With serviceWorker set, every other worker gets a pair to that
worker. The ends go through spawnWithFds, from descriptor 3 in a fixed
order: the supervisor channel, then a worker's service channel, or in the
service worker its end to each other worker by index, skipping its own.
Where they landed rides in --nish-prefork-channels= just before the worker
flag, and preforkSupervisorChannel, preforkServiceChannel and
preforkServiceChannelOf read it back. A pair cannot be handed to a worker
already running, so service channels make the workers one group: when any of
them exits, the supervisor sends the rest SIGTERM (SIGKILL after graceMs),
and once none is left, starts them all again on fresh pairs, after the
backoff of the one that exited first. No worker keeps an end whose peer is
gone longer than that. With restart off nothing is signalled; a worker sees
its peer's exit as a hang-up.
Listeners and pinning
With listenPort set (0 or above), the supervisor
binds one SO_REUSEPORT listener per worker itself, in index order, keeps
them for its whole run and hands listener k to every start of worker k
after the channel ends; preforkListener(args) answers it in the worker
(--nish-prefork-listener=, before the channels flag), so a worker serves
that descriptor instead of binding its own. Port 0 binds the first listener
anywhere and the rest to its port.
With pin on as well, each listener steers by CPU (tcpListenFlags' flag
4): the kernel hands a connection that arrived on CPU k to listener k,
and a worker that is restarted takes the same listener, so the order holds
across restarts, which it would not if each worker bound its own. pin alone
makes each worker call setCpuAffinity(k) before preforkWith answers it,
so worker k runs on CPU k only; a worker the kernel refuses (CPU k
outside the process's mask) says so on stderr and runs unpinned.
More workers than availableParallelism() is refused with a panic, since a
worker past the last CPU would be pinned nowhere and, steered, handed
nothing. Pinning and steering are Linux's: elsewhere the supervisor says once
that the workers run unpinned, and the listeners are plain SO_REUSEPORT
ones. Both are off by default.
QUIC
listenUdp, with listenPort, binds one UDP socket per worker
on the same port before the TCP listeners, held like them, as a group the
kernel steers by connection ID (udpBind flag 4): a datagram whose
destination ID starts with byte k reaches worker k. Worker k reads its
socket with preforkUdpListener(args) and the group's size N with
preforkUdpCount(args) (--nish-prefork-udp=<fd>,<N>, before the listener
flag), and sets QuicServerConfig.workerIndex to k and .workerCount to N
so every ID it issues starts with k. A connection then stays with its
worker when the client's address or port changes. Port 0 is a port no UDP
or TCP socket holds when the group binds, never another reuseport group's,
which a port-0 bind with SO_REUSEPORT can be given on Linux. More than
255 workers panics; where the kernel refuses the steering (off Linux, or
under a seccomp profile that refuses it) the supervisor says so on stderr
and runs one worker.
How many
NISH_WORKERS when it is a positive whole number, else
PreforkConfig.workers when that is positive, else
availableParallelism() on Linux and 1 elsewhere: Darwin shares a
SO_REUSEPORT port without spreading connections across it. The count has
no upper bound, so NISH_WORKERS is the operator's to set, not a client's.
Restarts
A worker that exits, by a signal or with any code, is started
again at the same index while the supervisor is not stopping. One that dies
within quickExitMs of starting waits before its restart: backoffMs the
first time, twice the last wait after that, up to backoffCapMs; one that
ran longer starts again at once and clears its wait, so a crash loop costs a
spawn every backoffCapMs, not a core. A restart spawn refuses waits on
the same backoff and is tried again; only the first start of a worker that
fails ends the supervisor, with PREFORK_SPAWN_FAILED.
Shutdown
The supervisor listens through signalFd(), made before the
first spawn so no signal falls between. On SIGTERM or SIGINT it stops
restarting, sends every worker SIGTERM, reaps them for graceMs, sends what
is left SIGKILL and reaps that. A second signal during the grace period sends
SIGKILL at once. The workers get the default dispositions back from exec,
so a worker that never asked for signalFd() ends on the SIGTERM. A
supervisor that dies any other way (SIGKILL, the OOM killer, or the default
action of SIGHUP or SIGQUIT, which signalFd() does not catch) leaves its
workers running, reparented to init.
Private helpers share the importing program's flat symbol namespace
(docs/wp26-stdlib.md §3e), which is why each one carries the module's name.
Functions
preforkArgs
args (process.argv) without the worker flag and the channels,
listener and UDP flags before it, the same in every process.
const preforkArgs: (args: string[]) => string[]
argsstring[]requiredthe process's arguments, process.argv
returnsstring[]a new list of the arguments the user gave, without the flags the supervisor appended, so a worker reads the same ones the supervisor did
preforkIndex
The worker index args carries in its last element, or -1 when none.
const preforkIndex: (args: string[]) => i32
argsstring[]requiredthe process's arguments, process.argv
returnsi32
this worker's index, 0 to N - 1, or -1 in the supervisor (or any
process not started by one)
preforkListener
In a worker, the listener the supervisor bound for it (listenPort); -1 when none.
const preforkListener: (args: string[]) => i32
argsstring[]requiredthe worker's arguments, process.argv
returnsi32the listening socket's descriptor to serve instead of binding one, or -1 when the worker should bind its own
preforkServiceChannel
In a worker, its end of its channel to the service worker; -1 in the service worker or when none.
const preforkServiceChannel: (args: string[]) => i32
argsstring[]requiredthe worker's arguments, process.argv
returnsi32the descriptor this worker reaches the service worker through, or -1
preforkServiceChannelOf
In the service worker, its end of the channel to worker index; -1 for itself or elsewhere.
const preforkServiceChannelOf: (
args: string[],
index: i32,
) => i32
argsstring[]requiredthe service worker's arguments, process.argv
indexi32requiredthe other worker whose channel to look up, 0 to N - 1
returnsi32
the descriptor the service worker reaches worker index through,
or -1 for its own index, an index out of range, or any other process
preforkSupervise
preforkWith, calling onChannel(state, index, fd, events) in the
supervisor whenever worker index's supervisor channel fd is readable or
hung up: events is pollWait's, 1 readable and 4 added on a hang-up or
error. The callback reads what is waiting, since an end left readable wakes
the loop again at once, and may write back, but never closes fd: the
supervisor closes it after a hang-up, when the worker is reaped and when it
stops, and a restarted worker comes with a new one.
Panics, besides, when serviceWorker is not below the worker count or the
count is above 64 with a service worker (63 with listenPort), the most
spawnWithFds hands one child; when pin, on Linux, asks for more workers
than availableParallelism(); when listenUdp asks for more than 255
workers (62 with a service worker); and, through preforkInvalid, on
seqpacket channels off Linux. A listener or UDP socket that cannot bind,
or a refused steering that leaves one worker beside a serviceWorker past
0, answers PREFORK_SPAWN_FAILED before any worker starts.
const preforkSupervise: <S>(
args: string[],
config: PreforkConfig,
state: S,
onChannel: (state: S, index: i32, fd: i32, events: i32) => void,
) => i32
argsstring[]requiredthe process's arguments, process.argv
configPreforkConfigrequired
the supervisor's settings, every field checked (a bad one
panics) in the supervisor and each worker alike; past that check a
worker reads only pin
stateSrequiredthe program's own value, passed to every onChannel call
onChannel(state: S, index: i32, fd: i32, events: i32) => voidrequired
what the supervisor runs when worker index's channel
fd wakes it, with pollWait's events
returnsi32
in a worker, its index at once; in the supervisor, once it is done,
PREFORK_STOPPED, PREFORK_WORKERS_EXITED or PREFORK_SPAWN_FAILED
preforkSupervisorChannel
In a worker, its end of its supervisor channel; -1 when it has none.
const preforkSupervisorChannel: (args: string[]) => i32
argsstring[]requiredthe worker's arguments, process.argv
returnsi32
the descriptor the worker talks to the supervisor through, or -1
without supervisorChannels or outside a worker
preforkUdpCount
In a worker, how many workers share its steered UDP port (listenUdp): the
QuicServerConfig.workerCount to pair with preforkIndex as its
workerIndex. 0 exactly when preforkUdpListener answers -1.
const preforkUdpCount: (args: string[]) => i32
argsstring[]requiredthe worker's arguments, process.argv
returnsi32the group's size, 1 to 255, or 0 without a UDP socket
preforkUdpListener
In a worker, the UDP socket the supervisor bound for it (listenUdp); -1
when none, or when the flag is not the supervisor's <fd>,<N>.
const preforkUdpListener: (args: string[]) => i32
argsstring[]requiredthe worker's arguments, process.argv
returnsi32the bound UDP socket's descriptor to serve QUIC on, or -1
preforkWith
In a worker, its index. In the supervisor, run the workers until SIGTERM or
SIGINT and answer PREFORK_STOPPED; with restart off, also until every
worker has exited, answering PREFORK_WORKERS_EXITED; or
PREFORK_SPAWN_FAILED when a worker's
first start failed (those already started are stopped first) or no signal
listener could be made (no worker is started). Panics on a config
preforkInvalid names a field of. Supervisor channels are read and
dropped; preforkSupervise hands them to the program.
const preforkWith: (
args: string[],
config: PreforkConfig,
) => i32
argsstring[]requiredthe process's arguments, process.argv
configPreforkConfigrequired
the supervisor's settings, every field checked (a bad one
panics) in the supervisor and each worker alike; past that check a
worker reads only pin
returnsi32
in a worker, its index at once; in the supervisor, once it is done,
PREFORK_STOPPED, PREFORK_WORKERS_EXITED or PREFORK_SPAWN_FAILED
preforkWorkerCount
How many workers to start: env (NISH_WORKERS, null when unset) when it
is a positive whole number, else requested when positive, else the
platform's default.
const preforkWorkerCount: (
requested: i32,
env: string | null,
) => i32
requestedi32required
the program's own choice, PreforkConfig.workers; 0 or
less leaves it to the environment or the platform
envstring | nullrequired
the raw NISH_WORKERS text, or null when it is unset; text
that is not a positive whole number is ignored
returnsi32how many worker processes the supervisor starts, always 1 or more
preforkWorkers
preforkWith with the default PreforkConfig.
const preforkWorkers: (args: string[]) => i32
argsstring[]requiredthe process's arguments, process.argv
returnsi32
in a worker, its index at once; in the supervisor, once it is done,
PREFORK_STOPPED or PREFORK_SPAWN_FAILED
Classes
PreforkConfig
The supervisor's numbers; every one is checked by preforkWith.
class PreforkConfig
| Member | |
|---|---|
new PreforkConfig() |
|
backoffCapMs: i32 |
The longest wait before a restart. |
backoffMs: i32 |
The first wait before restarting a worker that died young. |
channelKind: i32 |
What every channel is: NET_UNIX_STREAM (1), or NET_UNIX_SEQPACKET (2), which keeps each write one message but only Linux has. Stream is the default because it works on Darwin too. |
graceMs: i32 |
How long the workers get to exit after SIGTERM before SIGKILL. |
listenBacklog: i32 |
Their backlog. |
listenFlags: i32 |
Their tcpListenFlags beyond SO_REUSEPORT, which is always set: 0, or 2 for TCP_NODELAY. |
listenHost: string |
The address the listeners of listenPort bind. |
listenPort: i32 |
The port the supervisor binds one listener per worker on, 0 for any; -1, the default, leaves each worker to bind its own. |
listenUdp: boolean |
With listenPort, also bind one UDP socket per worker on the same port, before the TCP listeners, as a group the kernel steers by QUIC connection ID (udpBind flag 4): a datagram goes to worker k when byte 0 of its destination connection ID is k, so a QUIC connection stays with the worker that issued its IDs when the client's address changes. Linux only, and at most 255 workers (more panics). Where the kernel refuses the steering, the supervisor says so on stderr and runs one worker. |
pin: boolean |
Pin worker k to CPU k (setCpuAffinity), and with listenPort, steer each connection to the listener of the CPU that received it. Linux only; elsewhere the workers run unpinned. k is the CPU number, not the k-th allowed CPU: under a mask that skips CPU k, worker k runs unpinned and a connection received past the listeners falls back to the kernel's hash. |
quickExitMs: i32 |
A worker that exits within this many milliseconds of starting died young. |
restart: boolean |
Start a worker that exited again. Off, a worker that exits stays gone, and the supervisor returns once none is left. |
serviceWorker: i32 |
The worker every other worker gets a socketpair to; -1 for none. |
supervisorChannels: boolean |
Give each worker a socketpair to the supervisor (preforkSupervise). |
tickMs: i32 |
How often, in milliseconds, the supervisor reaps when nothing wakes it. |
workers: i32 |
Workers to start; 0 takes NISH_WORKERS or the platform's default. |
Constants
"--nish-prefork-channels="
PREFORK_CHANNELS_FLAG: string
The argument just before the worker flag when the worker has channels: the
descriptor of its supervisor channel, of its service channel, then in the
service worker of its end to each worker by index, comma-separated, -1
for none.
"--nish-prefork-listener="
PREFORK_LISTENER_FLAG: string
The argument before the channels flag, or before the worker flag when there is none, when the supervisor holds the listeners: the worker's listener's descriptor.
-2
PREFORK_SPAWN_FAILED: i32
What preforkWith answers when a worker could not be started, or when the
supervisor could not listen for SIGTERM and so starts none.
-1
PREFORK_STOPPED: i32
What preforkWith answers in the supervisor once its workers are gone.
"--nish-prefork-udp="
PREFORK_UDP_FLAG: string
The flag that hands a worker its UDP socket and the size of the steered
group, <fd>,<N>, before the listener flag, when listenUdp is on: read
it with preforkUdpListener and preforkUdpCount.
-3
PREFORK_WORKERS_EXITED: i32
What preforkWith answers in the supervisor when restart is off and every
worker has exited.
"--nish-prefork-worker="
PREFORK_WORKER_FLAG: string
The argument a worker is started with, its index after the =.

