nish/net/prefork

Run a server on every core: a supervisor process that starts one worker process per core, restarts one that dies, and stops them all on SIGTERM or SIGINT (docs/wp42-multicore-server.md §2.3).

import {
  PREFORK_CHANNELS_FLAG,
  PREFORK_LISTENER_FLAG,
  PREFORK_SPAWN_FAILED,
  PREFORK_STOPPED,
  PREFORK_UDP_FLAG,
  PREFORK_WORKERS_EXITED,
  PREFORK_WORKER_FLAG,
  PreforkConfig,
  preforkArgs,
  preforkIndex,
  preforkListener,
  preforkServiceChannel,
  preforkServiceChannelOf,
  preforkSupervise,
  preforkSupervisorChannel,
  preforkUdpCount,
  preforkUdpListener,
  preforkWith,
  preforkWorkerCount,
  preforkWorkers,
} from "nish/net/prefork";
import { tcpListenFlags } from "nish:net";
import { preforkWorkers } from "nish/net/prefork";

export const main = (): number => {
  const worker: i32 = preforkWorkers(process.argv);
  if (worker < 0) { return worker === -1 ? 0 : 1; } // the supervisor, done
  const listener = tcpListenFlags("0.0.0.0", 8080, 1024, 1); // SO_REUSEPORT
  // ... the single-threaded readiness loop, unchanged ...
};
One program, two roles

preforkWorkers(process.argv) answers in two kinds of process. In a worker it answers the worker's index, 0 to N - 1, at once, and the program goes on to bind its listener and serve. In the process the user started it supervises, and answers only once every worker has exited: -1 after a shutdown, -2 when the workers could not be started or the supervisor could not listen for signals. Each worker binds its own listener with tcpListenFlags' flag 1 (SO_REUSEPORT), so on Linux the kernel spreads new connections across them and no state is shared: a worker is the single-threaded server it was before, Arena.used() flat per worker.

How a worker is told

The supervisor runs the program again with --nish-prefork-worker=<index> appended to its arguments, through spawn; on Linux it runs /proc/self/exe, so a program started through PATH or a relative path finds itself. preforkArgs(process.argv) answers the arguments without that flag, for a worker that reads its own; the module takes process.argv from its caller because only a program with a main has one. The environment is inherited whole, and every descriptor the supervisor holds is close-on-exec, so a worker starts with none of them but the channel ends below (docs/security/runtime.md, RT-17, RT-19).

Channels

With supervisorChannels on, each worker is started with one end of a fresh unixSocketPair (channelKind: stream by default, since a seqpacket pair needs Linux and the supervisor panics on one elsewhere) and the supervisor keeps the other; preforkSupervise hands each readable end to a callback. With serviceWorker set, every other worker gets a pair to that worker. The ends go through spawnWithFds, from descriptor 3 in a fixed order: the supervisor channel, then a worker's service channel, or in the service worker its end to each other worker by index, skipping its own.

Where they landed rides in --nish-prefork-channels= just before the worker flag, and preforkSupervisorChannel, preforkServiceChannel and preforkServiceChannelOf read it back. A pair cannot be handed to a worker already running, so service channels make the workers one group: when any of them exits, the supervisor sends the rest SIGTERM (SIGKILL after graceMs), and once none is left, starts them all again on fresh pairs, after the backoff of the one that exited first. No worker keeps an end whose peer is gone longer than that. With restart off nothing is signalled; a worker sees its peer's exit as a hang-up.

Listeners and pinning

With listenPort set (0 or above), the supervisor binds one SO_REUSEPORT listener per worker itself, in index order, keeps them for its whole run and hands listener k to every start of worker k after the channel ends; preforkListener(args) answers it in the worker (--nish-prefork-listener=, before the channels flag), so a worker serves that descriptor instead of binding its own. Port 0 binds the first listener anywhere and the rest to its port.

With pin on as well, each listener steers by CPU (tcpListenFlags' flag 4): the kernel hands a connection that arrived on CPU k to listener k, and a worker that is restarted takes the same listener, so the order holds across restarts, which it would not if each worker bound its own. pin alone makes each worker call setCpuAffinity(k) before preforkWith answers it, so worker k runs on CPU k only; a worker the kernel refuses (CPU k outside the process's mask) says so on stderr and runs unpinned.

More workers than availableParallelism() is refused with a panic, since a worker past the last CPU would be pinned nowhere and, steered, handed nothing. Pinning and steering are Linux's: elsewhere the supervisor says once that the workers run unpinned, and the listeners are plain SO_REUSEPORT ones. Both are off by default.

QUIC

listenUdp, with listenPort, binds one UDP socket per worker on the same port before the TCP listeners, held like them, as a group the kernel steers by connection ID (udpBind flag 4): a datagram whose destination ID starts with byte k reaches worker k. Worker k reads its socket with preforkUdpListener(args) and the group's size N with preforkUdpCount(args) (--nish-prefork-udp=<fd>,<N>, before the listener flag), and sets QuicServerConfig.workerIndex to k and .workerCount to N so every ID it issues starts with k. A connection then stays with its worker when the client's address or port changes. Port 0 is a port no UDP or TCP socket holds when the group binds, never another reuseport group's, which a port-0 bind with SO_REUSEPORT can be given on Linux. More than 255 workers panics; where the kernel refuses the steering (off Linux, or under a seccomp profile that refuses it) the supervisor says so on stderr and runs one worker.

How many

NISH_WORKERS when it is a positive whole number, else PreforkConfig.workers when that is positive, else availableParallelism() on Linux and 1 elsewhere: Darwin shares a SO_REUSEPORT port without spreading connections across it. The count has no upper bound, so NISH_WORKERS is the operator's to set, not a client's.

Restarts

A worker that exits, by a signal or with any code, is started again at the same index while the supervisor is not stopping. One that dies within quickExitMs of starting waits before its restart: backoffMs the first time, twice the last wait after that, up to backoffCapMs; one that ran longer starts again at once and clears its wait, so a crash loop costs a spawn every backoffCapMs, not a core. A restart spawn refuses waits on the same backoff and is tried again; only the first start of a worker that fails ends the supervisor, with PREFORK_SPAWN_FAILED.

Shutdown

The supervisor listens through signalFd(), made before the first spawn so no signal falls between. On SIGTERM or SIGINT it stops restarting, sends every worker SIGTERM, reaps them for graceMs, sends what is left SIGKILL and reaps that. A second signal during the grace period sends SIGKILL at once. The workers get the default dispositions back from exec, so a worker that never asked for signalFd() ends on the SIGTERM. A supervisor that dies any other way (SIGKILL, the OOM killer, or the default action of SIGHUP or SIGQUIT, which signalFd() does not catch) leaves its workers running, reparented to init.

Private helpers share the importing program's flat symbol namespace (docs/wp26-stdlib.md §3e), which is why each one carries the module's name.

Functions

preforkArgs

args (process.argv) without the worker flag and the channels, listener and UDP flags before it, the same in every process.

const preforkArgs: (args: string[]) => string[]
argsstring[]required

the process's arguments, process.argv

returnsstring[]

a new list of the arguments the user gave, without the flags the supervisor appended, so a worker reads the same ones the supervisor did

preforkIndex

The worker index args carries in its last element, or -1 when none.

const preforkIndex: (args: string[]) => i32
argsstring[]required

the process's arguments, process.argv

returnsi32

this worker's index, 0 to N - 1, or -1 in the supervisor (or any process not started by one)

preforkListener

In a worker, the listener the supervisor bound for it (listenPort); -1 when none.

const preforkListener: (args: string[]) => i32
argsstring[]required

the worker's arguments, process.argv

returnsi32

the listening socket's descriptor to serve instead of binding one, or -1 when the worker should bind its own

preforkServiceChannel

In a worker, its end of its channel to the service worker; -1 in the service worker or when none.

const preforkServiceChannel: (args: string[]) => i32
argsstring[]required

the worker's arguments, process.argv

returnsi32

the descriptor this worker reaches the service worker through, or -1

preforkServiceChannelOf

In the service worker, its end of the channel to worker index; -1 for itself or elsewhere.

const preforkServiceChannelOf: (
  args: string[],
  index: i32,
) => i32
argsstring[]required

the service worker's arguments, process.argv

indexi32required

the other worker whose channel to look up, 0 to N - 1

returnsi32

the descriptor the service worker reaches worker index through, or -1 for its own index, an index out of range, or any other process

preforkSupervise

preforkWith, calling onChannel(state, index, fd, events) in the supervisor whenever worker index's supervisor channel fd is readable or hung up: events is pollWait's, 1 readable and 4 added on a hang-up or error. The callback reads what is waiting, since an end left readable wakes the loop again at once, and may write back, but never closes fd: the supervisor closes it after a hang-up, when the worker is reaped and when it stops, and a restarted worker comes with a new one.

Panics, besides, when serviceWorker is not below the worker count or the count is above 64 with a service worker (63 with listenPort), the most spawnWithFds hands one child; when pin, on Linux, asks for more workers than availableParallelism(); when listenUdp asks for more than 255 workers (62 with a service worker); and, through preforkInvalid, on seqpacket channels off Linux. A listener or UDP socket that cannot bind, or a refused steering that leaves one worker beside a serviceWorker past 0, answers PREFORK_SPAWN_FAILED before any worker starts.

const preforkSupervise: <S>(
  args: string[],
  config: PreforkConfig,
  state: S,
  onChannel: (state: S, index: i32, fd: i32, events: i32) => void,
) => i32
argsstring[]required

the process's arguments, process.argv

configPreforkConfigrequired

the supervisor's settings, every field checked (a bad one panics) in the supervisor and each worker alike; past that check a worker reads only pin

stateSrequired

the program's own value, passed to every onChannel call

onChannel(state: S, index: i32, fd: i32, events: i32) => voidrequired

what the supervisor runs when worker index's channel fd wakes it, with pollWait's events

returnsi32

in a worker, its index at once; in the supervisor, once it is done, PREFORK_STOPPED, PREFORK_WORKERS_EXITED or PREFORK_SPAWN_FAILED

preforkSupervisorChannel

In a worker, its end of its supervisor channel; -1 when it has none.

const preforkSupervisorChannel: (args: string[]) => i32
argsstring[]required

the worker's arguments, process.argv

returnsi32

the descriptor the worker talks to the supervisor through, or -1 without supervisorChannels or outside a worker

preforkUdpCount

In a worker, how many workers share its steered UDP port (listenUdp): the QuicServerConfig.workerCount to pair with preforkIndex as its workerIndex. 0 exactly when preforkUdpListener answers -1.

const preforkUdpCount: (args: string[]) => i32
argsstring[]required

the worker's arguments, process.argv

returnsi32

the group's size, 1 to 255, or 0 without a UDP socket

preforkUdpListener

In a worker, the UDP socket the supervisor bound for it (listenUdp); -1 when none, or when the flag is not the supervisor's <fd>,<N>.

const preforkUdpListener: (args: string[]) => i32
argsstring[]required

the worker's arguments, process.argv

returnsi32

the bound UDP socket's descriptor to serve QUIC on, or -1

preforkWith

In a worker, its index. In the supervisor, run the workers until SIGTERM or SIGINT and answer PREFORK_STOPPED; with restart off, also until every worker has exited, answering PREFORK_WORKERS_EXITED; or PREFORK_SPAWN_FAILED when a worker's first start failed (those already started are stopped first) or no signal listener could be made (no worker is started). Panics on a config preforkInvalid names a field of. Supervisor channels are read and dropped; preforkSupervise hands them to the program.

const preforkWith: (
  args: string[],
  config: PreforkConfig,
) => i32
argsstring[]required

the process's arguments, process.argv

configPreforkConfigrequired

the supervisor's settings, every field checked (a bad one panics) in the supervisor and each worker alike; past that check a worker reads only pin

returnsi32

in a worker, its index at once; in the supervisor, once it is done, PREFORK_STOPPED, PREFORK_WORKERS_EXITED or PREFORK_SPAWN_FAILED

preforkWorkerCount

How many workers to start: env (NISH_WORKERS, null when unset) when it is a positive whole number, else requested when positive, else the platform's default.

const preforkWorkerCount: (
  requested: i32,
  env: string | null,
) => i32
requestedi32required

the program's own choice, PreforkConfig.workers; 0 or less leaves it to the environment or the platform

envstring | nullrequired

the raw NISH_WORKERS text, or null when it is unset; text that is not a positive whole number is ignored

returnsi32

how many worker processes the supervisor starts, always 1 or more

preforkWorkers

preforkWith with the default PreforkConfig.

const preforkWorkers: (args: string[]) => i32
argsstring[]required

the process's arguments, process.argv

returnsi32

in a worker, its index at once; in the supervisor, once it is done, PREFORK_STOPPED or PREFORK_SPAWN_FAILED

Classes

PreforkConfig

The supervisor's numbers; every one is checked by preforkWith.

class PreforkConfig
Member
new PreforkConfig()
backoffCapMs: i32 The longest wait before a restart.
backoffMs: i32 The first wait before restarting a worker that died young.
channelKind: i32 What every channel is: NET_UNIX_STREAM (1), or NET_UNIX_SEQPACKET (2), which keeps each write one message but only Linux has. Stream is the default because it works on Darwin too.
graceMs: i32 How long the workers get to exit after SIGTERM before SIGKILL.
listenBacklog: i32 Their backlog.
listenFlags: i32 Their tcpListenFlags beyond SO_REUSEPORT, which is always set: 0, or 2 for TCP_NODELAY.
listenHost: string The address the listeners of listenPort bind.
listenPort: i32 The port the supervisor binds one listener per worker on, 0 for any; -1, the default, leaves each worker to bind its own.
listenUdp: boolean With listenPort, also bind one UDP socket per worker on the same port, before the TCP listeners, as a group the kernel steers by QUIC connection ID (udpBind flag 4): a datagram goes to worker k when byte 0 of its destination connection ID is k, so a QUIC connection stays with the worker that issued its IDs when the client's address changes. Linux only, and at most 255 workers (more panics). Where the kernel refuses the steering, the supervisor says so on stderr and runs one worker.
pin: boolean Pin worker k to CPU k (setCpuAffinity), and with listenPort, steer each connection to the listener of the CPU that received it. Linux only; elsewhere the workers run unpinned. k is the CPU number, not the k-th allowed CPU: under a mask that skips CPU k, worker k runs unpinned and a connection received past the listeners falls back to the kernel's hash.
quickExitMs: i32 A worker that exits within this many milliseconds of starting died young.
restart: boolean Start a worker that exited again. Off, a worker that exits stays gone, and the supervisor returns once none is left.
serviceWorker: i32 The worker every other worker gets a socketpair to; -1 for none.
supervisorChannels: boolean Give each worker a socketpair to the supervisor (preforkSupervise).
tickMs: i32 How often, in milliseconds, the supervisor reaps when nothing wakes it.
workers: i32 Workers to start; 0 takes NISH_WORKERS or the platform's default.

Constants

"--nish-prefork-channels="

PREFORK_CHANNELS_FLAG: string

The argument just before the worker flag when the worker has channels: the descriptor of its supervisor channel, of its service channel, then in the service worker of its end to each worker by index, comma-separated, -1 for none.

"--nish-prefork-listener="

PREFORK_LISTENER_FLAG: string

The argument before the channels flag, or before the worker flag when there is none, when the supervisor holds the listeners: the worker's listener's descriptor.

-2

PREFORK_SPAWN_FAILED: i32

What preforkWith answers when a worker could not be started, or when the supervisor could not listen for SIGTERM and so starts none.

-1

PREFORK_STOPPED: i32

What preforkWith answers in the supervisor once its workers are gone.

"--nish-prefork-udp="

PREFORK_UDP_FLAG: string

The flag that hands a worker its UDP socket and the size of the steered group, <fd>,<N>, before the listener flag, when listenUdp is on: read it with preforkUdpListener and preforkUdpCount.

-3

PREFORK_WORKERS_EXITED: i32

What preforkWith answers in the supervisor when restart is off and every worker has exited.

"--nish-prefork-worker="

PREFORK_WORKER_FLAG: string

The argument a worker is started with, its index after the =.