Networking

nish:net gives a program TCP and UDP sockets and a loop that waits for whichever of them is ready. It is a thin layer: every socket is non-blocking, there is no callback, and an async function cannot wait on a socket yet, so your main is the loop. The protocol stack on top of it (TLS, HTTP/1.1, HTTP/2, HTTP/3) is in the standard library; this page is about the sockets themselves, and Unix sockets covers the Unix-domain listeners, connections and pairs, passing a descriptor over one, and naming the process at the other end. The rules are in the reference, nish:net.

Every function is also a global, as every nish: export is, but importing them says where they come from. They are native only: a wasm32 build has no sockets, and the compiler refuses a call to any of them there.

Errors are numbers

Every call answers an i32. Zero or more is success: a descriptor, a count of bytes, or 0. A negative number is the failure's error number, negated, and the ones a loop branches on are the same on every platform:

Answer Meaning
-11 would block: nothing to read, no room to write, nobody waiting to be accepted. Call again once the loop says the socket is ready
-32 the peer has gone. Writing to it never kills the process with SIGPIPE
-98 the port is already in use
-111 the connection was refused
-22 a bad argument: a host that is not a numeric address, a flag that does not exist

Hosts are numeric literals, "127.0.0.1" or "::1": the socket calls never look a name up. To reach a name, resolve it first with nish/net/dns. "::" listens on every address, IPv4 and IPv6 both.

A server

tcpListen(host, port, backlog) opens a listening socket; port 0 lets the kernel choose one, and netLocalPort(fd) tells you which. tcpAccept(fd, peer) takes the next waiting connection and writes its address into peer, an 18-byte u8[]. netRead and netWrite move bytes between a socket and a range of a u8[], and netClose closes it.

To wait, make a loop with pollCreate(), watch each socket with pollAdd(loop, fd, events, token) — 1 for readable, 2 for writable, and a token you choose to tell the sockets apart — and call pollWait(loop, ready, timeoutMs). It fills ready with pairs, a token and its events, and answers how many, 0 when the timeout passed.

import { netClose, netLocalPort, netRead, netWrite, pollAdd, pollCreate, pollWait, tcpAccept, tcpListen } from "nish:net";

export const main = (): i32 => {
  const server = tcpListen("127.0.0.1", 0, 16);
  if (server < 0) {
    return 1;
  }
  console.log(`listening on ${netLocalPort(server)}`);
  const loop = pollCreate();
  pollAdd(loop, server, 1, 0);
  const ready: i32[] = [0, 0];
  const peer = new Array<u8>(18);
  const buf = new Array<u8>(4096);
  // Wait up to a second for one client, and echo back what it sends first.
  if (pollWait(loop, ready, 1000) > 0) {
    const conn = tcpAccept(server, peer);
    if (conn >= 0) {
      pollAdd(loop, conn, 1, 1);
      if (pollWait(loop, ready, 1000) > 0) {
        const n = netRead(conn, buf, 0, buf.length);
        if (n > 0) {
          netWrite(conn, buf, 0, n);
        }
      }
      netClose(conn);
    }
  }
  netClose(server);
  netClose(loop);
  return 0;
};

A real server keeps accepting until tcpAccept answers -11, keeps reading until netRead does, and holds on to what netWrite could not send yet. The loop is level-triggered: anything you leave unread is reported again on the next pollWait.

The protocol servers in nish/net keep no timer of their own: your loop owns the clock. Call their expire(monotonicNanos()) once a turn, with a pollWait timeout no longer than the shortest deadline, and they close what has gone on too long. Http1Server closes an idle slot, a head past headTimeout, and a refused client still draining past lingerTimeout (it shuts its write side and reads what the client still sends, up to lingerBytes, so the error response is not lost to a reset). Http2TlsServer closes a client that has not acknowledged its SETTINGS within settingsTimeout, or is quiet past idleTimeout. Http3Server needs no expire: its timer wheel already runs on the clock your loop hands tick, so it closes a slot with no request open past Http3Config.idleTimeout (60 s), and one whose client trickles a request's head past headTimeout (30 s), by itself. A QUIC PING keeps the connection, not the slot. A WebTransport layer ends a session quiet past its idleTimeout and drops datagrams past maxDatagramsPerSecond; dropping is the whole defence, as datagrams may be lost anyway, so a burst never closes the session. Each limit has a default and is checked at start-up; the standard library's nish/net section has the details.

Keeping a request loop's memory flat

A server's loop runs until it is stopped, so whatever a pass allocates and does not give back grows for as long as it serves. A pass that calls accept or respond stores into the server's memory, and the compiler then keeps everything the pass allocated until the loop ends: a reply text built with a template on every request piles up. The compiler warns (NL9018) and names the line. Build the temporary inside a block that opens with using a = arena(), copying what must outlive it into a buffer you made before the loop, and answer after the block:

while (event !== H1_NEED_MORE && event !== H1_ERROR) {
  if (event === H1_END) {
    {
      using a = arena();
      fill(heads, `request ${heads.served}\n`); // copies into heads.reply
    }
    answer(server.connection(slot), heads); // calls respond, outside the block
  }
  event = server.next(slot);
}

Over 20,000 requests the first version grew 1,252 kB and this one none. The compiler refuses the block if anything allocated in it could outlive it, so the rewrite cannot lose a value you still need. --deny-retention turns the warning into an error, as it does for the other retention warnings.

The same holds when the text is built in a handler the loop calls. A helper that builds the reply and also answers stores into the server's memory, so it has no scope of its own either, and what it drops piles up in the loop. The warning sits on the loop and names the helper's line. Put the block inside the helper, around the work that builds the text:

const readPost = (conn: Http1Connection, heads: Heads): void => {
  {
    using a = arena();
    fill(heads, `request ${heads.served}\n`);
  }
  answer(conn, heads);
};

How many connections: the descriptor limit

Every connection a server holds is a file descriptor, and the operating system caps how many one process may have open with two limits: a soft one, which the process may raise itself, and a hard one above it, which only the operator may. Shells and service managers often start a program with a soft limit of 1024 (256 on macOS), far below a hard limit of 65536 or more.

A native program's first nish:net call that makes a descriptor (tcpListen, tcpConnect, udpBind, unixListen, unixConnect, unixSocketPair or pollCreate) raises the soft limit to the hard one, once for the whole process, as Node and Go do at start-up. On macOS it stops at OPEN_MAX (10240); where kern.maxfilesperproc is set lower, the raise fails and the program keeps the limit it had. A program that never calls nish:net keeps the limit it was given, and so does one that only accepts on a listener it was handed (recvFd, socket activation) without making a descriptor of its own.

Past the hard limit is the operator's to set: ulimit -Hn in the shell that starts the server, LimitNOFILE= in a systemd unit, or the container's nofile limit (docker run --ulimit nofile=…). At the limit, a call that wants a descriptor answers -24 (EMFILE), and nish/net/handler waits for one to close rather than spin (on -23, a full set of ring slots, too). On Linux a listener can serve past the limit instead: see More connections than descriptors.

A child the program starts with spawn inherits the raised soft limit, as it would under Node. A child that relies on the lower one should lower it itself.

Sharing a port and turning Nagle off: tcpListenFlags

tcpListenFlags(host, port, backlog, flags) is tcpListen with one more argument, a set of flags added together. Flag 1 sets SO_REUSEPORT, which lets several listeners that all ask for it open the same port, each with its own queue of waiting connections:

  • On Linux the kernel spreads new connections across them. Start the same server several times, each process with its own listener on one port, and the connections are shared between the processes without a load balancer in front. Only listeners started by the same user may share.
  • On macOS they share the port, but one listener takes the connections. It still lets a new copy of a server open the port before the old one closes, so a restart drops nothing.

Flag 2 sets TCP_NODELAY on every connection tcpAccept takes from the listener, on every platform. Without it the kernel may hold a small write back until the peer acknowledges the last one (Nagle's algorithm), which saves packets for a stream of tiny writes and costs a round trip for a server that answers each request with one small write. The bytes that arrive are the same either way; only when they leave changes.

A socket tcpConnect opens holds TCP_NODELAY already, as libpq and Go set it on every client socket: a client sends a whole request and waits on the answer, so there is nothing for Nagle to save.

Flag 0 is tcpListen exactly and 3 is both. Flag 4 steers connections for nish/net/prefork, and flag 8 is the next section's. Any other bit answers -22. A listener without flag 1 cannot open a port a listener with it holds, nor the other way round: both answer -98.

import { netClose, netLocalPort, tcpListenFlags } from "nish:net";

export const main = (): i32 => {
  const first = tcpListenFlags("127.0.0.1", 0, 64, 1);
  if (first < 0) {
    return 1;
  }
  const port = netLocalPort(first);
  // The same port again: allowed, because both listeners asked for it.
  const second = tcpListenFlags("127.0.0.1", port, 64, 1);
  console.log(`${second >= 0}`); // true
  console.log(`${tcpListenFlags("127.0.0.1", port, 64, 4)}`); // -22
  netClose(second);
  netClose(first);
  return 0;
};

A program can watch both listeners in one loop, as in the server above, and accept from whichever is ready. A server that wants both the shared port and its small answers sent at once opens each listener with 3.

More connections than descriptors: NET_LISTEN_RING

On Linux a listener opened with the flag NET_LISTEN_RING (8) takes its connections into io_uring rings instead of the descriptor table. What tcpAccept answers is then a slot handle, a number from 1048576 up, and the descriptor limit no longer bounds how many a process holds: one listener serves thousands of connections under a nofile limit of 1024, with only a few dozen descriptors open.

A slot handle goes wherever a server loop puts a connection. netRead, netWrite, netWrite2, netClose, netLocalPort and the poll calls take it as they take a descriptor, with the same answers: -11 until pollWait reports the handle ready, then progress, and 0 at the end of the stream. A loop written for descriptors runs unchanged. What differs:

  • netShutdown queues the shutdown behind the bytes already written, so the peer reads them and then the end of the stream. After the write side is shut, netWrite answers -32; after the read side is, netRead answers what is queued and then 0. Http1Server's lingering close works on a slot as on a descriptor.
  • netMss answers -1: the ring cannot ask the socket for its segment size, so take your fallback.
  • netSetOption answers -95, and every call that needs a real descriptor (sendFd, peerCredentials) answers -9, as does any call on a handle after its netClose.
  • tcpAccept writes the peer's address as zeros.
  • A slot handle is in one poll loop at a time: pollAdd to a second loop answers -17 until the first removes it or is closed.
  • Once a connection has failed, by a peer's reset or a failed send, netWrite answers the errno (-104 for a reset), and netRead answers it on every read once the queued bytes are read, where a descriptor answers it once and then 0.
  • When every slot the listener may hold is taken, tcpAccept answers -23. Each ring listener holds up to 64 rings of as many slots as the descriptor limit, at most 16,384 a ring. Close a connection and the next waiting one takes its slot.
  • If the kernel is short of memory for a new connection, tcpAccept answers -12 or -105 once, as it does for a plain listener. The listener then pauses for a moment (10 ms, doubling to 640 ms while the shortage lasts) and goes on accepting; waiting connections stay queued.
  • If the listener's accept fails with any other error but -11, -4, -103 or -71, the listener is broken: it stops accepting and tcpAccept answers that error from then on. Close the listener and listen again.
  • A process opens at most 31 ring listeners in its life, closed ones included; the 32nd tcpListenFlags with NET_LISTEN_RING answers -23.
  • Each connection holds at most 256 KiB that the peer has not taken yet. A netWrite past that copies what fits; a full connection answers -11 until the peer reads, as a full socket buffer does.

The kernel may refuse rings: a seccomp filter (some container runtimes), a kernel without io_uring, or io_uring_disabled set. Then the listener serves descriptors as if the flag were not there, and netRingState() says what happened: 1 for rings, 0 until a listener has found rings or a refusal, or the refusal as a negative errno (-1 when a filter or io_uring_disabled forbids it, -38 for a kernel without io_uring). The first listener that finds rings or a refusal decides for the whole process. A listener whose rings fail for want of descriptors or memory serves descriptors itself, and a later listener asks again. On macOS the flag is accepted and does nothing.

import { NET_LISTEN_RING, netClose, netRingState, tcpListenFlags } from "nish:net";

export const main = (): i32 => {
  const listener = tcpListenFlags("127.0.0.1", 0, 1024, 1 | NET_LISTEN_RING);
  if (listener < 0) {
    return 1;
  }
  const state = netRingState();
  if (state === 1) {
    console.log("connections go into rings");
  } else {
    console.log(`descriptors, because rings answered ${state}`);
  }
  netClose(listener);
  return 0;
};

Use the flag when a server needs more connections than its limit allows and nothing it does with a connection needs a descriptor. A server that hands connections to another process with sendFd keeps descriptors.

nish/net/handler takes the flag through its limits. Set HandlerLimits.ring and open the listener with limits.listen(host, port, backlog, flags), which is tcpListenFlags with NET_LISTEN_RING added when ring is set, then hand it to new HttpHandler(limits, listener, router) as any listener. HandlerLimits.slots is then the cap on connections, not the descriptor limit: set it to what the server should hold. When every ring slot is taken, the handler rests its listener on the -23, as it does on -24 for descriptors, until a connection closes or 100 ms pass, so a full server waits rather than spin. h.resting() is true while it does, for a server that wants to log or count the times it is full:

import { HandlerLimits, HttpHandler } from "nish/net/handler";
import { Router, RouterLimits } from "nish/net/router";
import { netClose, netRingState } from "nish:net";

export const main = (): i32 => {
  const limits = new HandlerLimits();
  limits.ring = true;
  limits.slots = 100000;
  const listener = limits.listen("127.0.0.1", 0, 4096, 0);
  if (listener < 0) {
    return 1;
  }
  const h = new HttpHandler(limits, listener, new Router(new RouterLimits()));
  console.log(netRingState() === 1 ? "connections go into rings" : "descriptors");
  h.drain();
  netClose(listener);
  return 0;
};

Rings are off by default. What one ring process costs against five processes on descriptors, with an SQLite lookup per request, is measured in the benchmarks.

Socket options: netSetOption

netSetOption(fd, option, value) sets one option on a socket. The option is one of five ids nish:net exports, and any other number answers -22:

Option value
NET_TCP_NODELAY 1 sends small writes at once, 0 lets Nagle batch them
NET_TCP_QUICKACK 1 acknowledges what arrives at once, 0 delays the ACK; Linux only, -95 on macOS
NET_SO_KEEPALIVE 1 probes a connection that has gone quiet
NET_SO_SNDBUF the send buffer, in bytes (Linux stores twice the value)
NET_SO_RCVBUF the receive buffer, in bytes (Linux stores twice the value)

It answers 0, or a negative error: -9 for a closed descriptor, -88 for one that is not a socket.

On Linux a keep-alive connection already delays its ACKs: once the server has sent its first response, the kernel waits to acknowledge the next request until it can ride on that request's response. Most servers need no call at all. Setting NET_TCP_QUICKACK changes that, and the trade runs both ways:

  • 1 after each response turns delayed ACKs off until the kernel next turns them on, so every request is acknowledged at once in a segment of its own. That is one more system call and one more packet per request, worth it only for a peer that waits on the ACK before it sends more.
  • 0 once, after accept, keeps delayed ACKs on from the first request. It costs one call per connection. The kernel still switches them off for the next request when a response took longer than its ACK timer (tens of milliseconds) or a segment arrived out of order, and nothing short of a call after each response turns them back on that early. On a connection answered quickly, that call after each response changes nothing.
  • No call behaves like 0 once from the second request on.

bench/quickack/ in the repository measures each way. On four cores running Linux, at 256 keep-alive connections over loopback, 1 after each response added one system call and one segment per request and served 13 to 18% fewer requests per second than no call. 0 after each response and 0 once stayed within the noise, since 0 saves an ACK only after a slow response. A server that wants the delayed ACKs from the first request sets the option once, as it accepts:

import { NET_TCP_QUICKACK, netSetOption, tcpAccept } from "nish:net";

export const acceptOne = (listener: i32, peer: u8[]): i32 => {
  const conn = tcpAccept(listener, peer);
  if (conn >= 0) {
    // Delay ACKs from the first request, so each one rides on its response.
    // On macOS this answers -95, which a server can ignore.
    netSetOption(conn, NET_TCP_QUICKACK, 0);
  }
  return conn;
};

The ids are plain i32s you pass straight to the call. They are not module constants, so const MODE = NET_TCP_QUICKACK; at the top of a module is an error; write the id where you call netSetOption.

A connection's segment size: netMss

netMss(fd) answers how many bytes of payload one TCP segment on a connected socket carries, the maximum segment size the kernel is sending with now. A program that batches writes can size each batch to whole segments. It answers -1 when there is no MSS to read: a socket that is not connected yet, a listener, a UDP socket, a closed descriptor or one that is not a socket, and a NET_LISTEN_RING slot handle. -1 is not an error code, so check for a positive answer and fall back to a size of your own:

import { netMss } from "nish:net";

// Bytes to gather before one write: four full segments, or 4096 when the
// socket has no MSS to tell.
export const batchSize = (conn: i32): i32 => {
  const mss = netMss(conn);
  return mss > 0 ? mss * 4 : 4096;
};

The kernel can change the MSS during a connection, as it learns the path, so read it again rather than keeping the first answer. On Linux it comes from TCP_INFO, on macOS from TCP_MAXSEG; over loopback it is large (tens of kilobytes), and over Ethernet about 1,448. A wasm build, which has no sockets, compiles netMss to -1, so the same code builds there and takes the fallback.

A server on every core: nish/net/prefork

A Nish server is one thread. To use every core, run one copy of it per core, each with its own SO_REUSEPORT listener on the same port: that is what nish/net/prefork does. Call preforkWorkers(process.argv) at the top of main, before you open a listener:

import { netClose, tcpListenFlags } from "nish:net";
import { PREFORK_STOPPED, preforkWorkers } from "nish/net/prefork";

export const main = (): i32 => {
  const worker = preforkWorkers(process.argv);
  if (worker < 0) {
    // The process you started: every worker has exited.
    return worker === PREFORK_STOPPED ? 0 : 1;
  }
  // A worker, numbered from 0: open its own listener and serve.
  const listener = tcpListenFlags("0.0.0.0", 8080, 1024, 1);
  if (listener < 0) {
    return 1;
  }
  // ... the readiness loop of a one-process server, unchanged ...
  netClose(listener);
  return 0;
};

The program runs in two roles. The process you start becomes the supervisor: it starts the program again once per worker, and preforkWorkers answers it only when they have all exited, with -1 (PREFORK_STOPPED) after a shutdown or -2 (PREFORK_SPAWN_FAILED) when a worker's first start failed or the supervisor could not set up its signal handling, in which case no worker starts.

In each worker it answers that worker's index, 0 up to one less than the count, at once. A worker is told its index by one extra argument at the end of its arguments; preforkArgs(process.argv) gives you the arguments without it, the same list in every process, so read your port and options from that. Workers share nothing, so each one is the single-threaded server it was before.

  • How many workers. The NISH_WORKERS environment variable when it is a positive whole number, such as NISH_WORKERS=2; anything else in it is ignored. Otherwise one per core on Linux, and one on macOS, which lets the listeners share the port but does not spread connections across them. The count has no upper bound, so set NISH_WORKERS yourself; never pass it on from a client.
  • Restarts. A worker that exits, whatever its exit code, or is killed, is started again at the same index. One that dies within a second of starting waits first: 100 ms, then twice as long each time it does so again, up to 5 s. A server that crashes on start costs one start every few seconds, not a core. A restart the system refuses (out of processes or memory) waits the same way and is tried again.
  • Stopping. SIGTERM or SIGINT (Ctrl-C) to the supervisor sends SIGTERM to every worker and waits up to 5 s for them, then kills the rest; a second signal kills them at once. The supervisor then exits with the -1 path above. A supervisor stopped any other way (SIGKILL, the out-of-memory killer, SIGHUP or SIGQUIT) leaves its workers running; stop it with SIGTERM.

preforkWith(process.argv, config) takes a PreforkConfig to change those numbers: workers, backoffMs, backoffCapMs, quickExitMs (what counts as dying young), graceMs and tickMs (how often the supervisor looks for a restart that is due). A negative number, a tickMs or backoffMs of zero, or a backoffCapMs below backoffMs stops the program with a panic naming the field. examples/prefork-server.ts is a whole HTTP server built this way: each answer names the worker that sent it, so a few curls show the spread.

By default the workers are not pinned to cores. Each one binds the port with SO_REUSEPORT, the kernel picks a worker for each new connection by hashing its address, and the scheduler runs each worker on any core it allows. To keep each worker on its own core and steer each connection to it, use a config-owned listener with pin, below.

One core per worker

Set listenPort and the supervisor binds the listeners itself, one per worker, before any worker starts: worker k takes its own with preforkListener(process.argv) and accepts from it, and gets the same one back after a restart. listenHost, listenBacklog and listenFlags (0, or 2 for TCP_NODELAY) shape them; port 0 picks a free port for the whole group, and with listenUdp one that no UDP or TCP socket holds, so two groups on port 0 never share a port and never receive each other's datagrams.

Add pin = true and worker k also pins itself to CPU k (setCpuAffinity), and on Linux the listeners steer each connection to the worker on the CPU that received it, so a connection's packets and the worker that answers it share a core's caches:

import { PREFORK_STOPPED, PreforkConfig, preforkListener, preforkWith } from "nish/net/prefork";

export const main = (): i32 => {
  const config = new PreforkConfig();
  config.listenPort = 8080;
  config.pin = true;
  const worker = preforkWith(process.argv, config);
  if (worker === PREFORK_STOPPED) {
    return 0;
  }
  if (worker < 0) {
    return 1;
  }
  const listener = preforkListener(process.argv);
  // Accept and serve on `listener` here.
  return listener >= 0 ? 0 : 1;
};

Pinning is for a machine the server has to itself: each worker owns a core, so workers should be at most availableParallelism(), and on Linux more pinned workers than CPUs stops the program with a panic rather than doubling two workers up on one core. That count is availableParallelism(), which a container's CPU quota or NISH_NUM_THREADS lowers.

A CPU outside the ones the program may run on (taskset, a container's cpuset) is never taken, and worker k asks for CPU number k, not the k-th CPU allowed: under taskset -c 4-7 every pin fails, each worker says so on stderr and runs unpinned, and a connection received on CPU 4 or above, past the listeners, falls back to the kernel's hash. Run a pinned server on CPUs that start at 0. Off Linux the workers run unpinned and unsteered, and the supervisor says so on stderr. A listenPort outside -1..65535, a listenBacklog of 0 or less, or other listenFlags stop the program with a panic.

QUIC on every core: listenUdp

A QUIC connection is named by its connection ID, not by the client's address, so it survives the client moving to a new port or network. Over several workers that only holds if each datagram reaches the worker that holds its connection. Set listenUdp with listenPort and the supervisor binds one UDP socket per worker on that port, as a group the kernel steers by connection ID: a datagram whose destination connection ID starts with byte k goes to worker k. Worker k takes its socket with preforkUdpListener(process.argv) and the group's size with preforkUdpCount(process.argv), and puts both in its QuicServerConfig, so every connection ID it hands out starts with k:

import { PREFORK_STOPPED, PreforkConfig, preforkUdpCount, preforkUdpListener, preforkWith } from "nish/net/prefork";
import { QuicServerConfig } from "nish/net/quic";

/** Worker `worker`'s QUIC settings: every connection ID it issues starts with its index. */
export const steered = (config: QuicServerConfig, worker: i32): QuicServerConfig => {
  config.workerIndex = worker;
  config.workerCount = preforkUdpCount(process.argv);
  return config;
};

export const main = (): i32 => {
  const config = new PreforkConfig();
  config.listenPort = 4433;
  config.listenUdp = true;
  const worker = preforkWith(process.argv, config);
  if (worker === PREFORK_STOPPED) {
    return 0;
  }
  if (worker < 0) {
    return 1;
  }
  const udp = preforkUdpListener(process.argv);
  // Serve HTTP/3 here: `new Http3Server(steered(quicConfig, worker), h3Config, udp, slots, entropy)`.
  return udp >= 0 ? 0 : 1;
};

A server that is not part of a group leaves workerIndex at -1 and workerCount at 0. Any other pair outside 0 <= workerIndex < workerCount <= 255 stops the program with a panic when the QuicListener or Http3Server is built, naming the pair:

nish/net/quic: workerIndex 2 and workerCount 2, not 0 <= k < n <= 255

Two more things change in a group. A worker that receives a packet for a connection ID that another worker issued drops it instead of answering with a stateless reset, since that reset would end the other worker's live connection. And the group holds at most 255 workers, as the steering reads one byte: more stops the program with a panic.

Steering needs Linux. Where the kernel refuses it (off Linux, or under a seccomp profile that forbids the program), the supervisor says so on stderr, nish/net/prefork: QUIC steering refused (-95), running one worker, and runs a single worker on a plain socket, with preforkUdpCount answering 1. A serviceWorker other than 0 has no place in that one-worker run, so there the supervisor starts nothing and answers PREFORK_SPAWN_FAILED. The underlying call is udpBind with flag 4 (nish:net).

QUIC congestion control: congestion

QuicServerConfig.congestion picks the congestion controller every connection runs: QUIC_RECOVERY_NEWRENO (0, RFC 9002) or QUIC_RECOVERY_CUBIC (1, RFC 9438), both from nish/net/quic-recovery. The field is required; pass QUIC_RECOVERY_DEFAULT, which names CUBIC, unless you have a reason to pick. Any other value is refused: a connection built with it closes at once, as one with any other limit out of range does. CUBIC regrows the window faster after a loss on a long, lossy path, and on a transfer of one bandwidth-delay product the two finish level. The measurement is in the QUIC security record.

QUIC and ECN: arrivalEcn

Http3Server uses Explicit Congestion Notification (ECN) in both directions without any setup. It marks its packets ECT(0) wherever the platform sends the ECN bits, and halves its window when the client reports congestion (CE). Its ACKs also report the bits each client packet arrived with: from the first marked packet on, the ACKs of that packet-number space are ACK_ECN frames, with ECT(0), ECT(1) and CE counts for the space (RFC 9000 §13.4.1). A client that validates its own path needs those counts, or it stops marking.

A program that drives QuicListener itself reads the arrival bits from udpRecvFrom's meta[1] and sets them on the connection before it hands the datagram over:

conn.arrivalEcn = meta[1] & 3;
quicListenerReceiveSegments(conn, buf, 0, n, segment, now);

If arrivalEcn stays 0, which means Not-ECT, the connection counts nothing and its ACKs stay plain. That is always the case on macOS, where meta[1] is always 0. A packet delivered twice is counted once.

Workers that are not restarted

Set restart to false when something else restarts the service, such as systemd or a container runtime, and a worker that exits should stay down rather than come back on its own. It then stays gone, and the others carry on. Once every worker has exited, the supervisor returns -3 (PREFORK_WORKERS_EXITED), so the program can exit and let its own supervisor decide:

import { PREFORK_STOPPED, PreforkConfig, preforkWith } from "nish/net/prefork";

export const main = (): number => {
  const config = new PreforkConfig();
  config.restart = false;
  const worker = preforkWith(process.argv, config);
  if (worker < 0) {
    return worker === PREFORK_STOPPED ? 0 : 1;
  }
  // ... serve as worker `worker` ...
  return 0;
};

SIGTERM and SIGINT still stop the workers that are left, and still give -1.

Channels to the supervisor and to a service worker

Workers share nothing, but some servers need one thing in one place: a cache, a connection pool, a counter. Two more PreforkConfig fields give the processes a Unix socketpair to talk over (Unix sockets):

  • supervisorChannels, true: each worker gets a pair to the supervisor. Supervise with preforkSupervise(process.argv, config, state, onChannel) instead of preforkWith to hear from them; preforkWith reads what a worker sends and drops it.
  • serviceWorker, a worker's index: every other worker gets a pair to that one. -1, the default, gives none.
  • channelKind: NET_UNIX_STREAM (1, the default), or NET_UNIX_SEQPACKET (2) to keep each write one message. Only Linux has seqpacket pairs, so stream is the default: it works on macOS too. On macOS a seqpacket channel stops the program with a panic, seqpacket channels need Linux, before any worker starts.

A worker finds its ends with three calls, each taking process.argv and answering a descriptor, or -1 when it has no such end:

Call Answers
preforkSupervisorChannel(process.argv) its end to the supervisor
preforkServiceChannel(process.argv) its end to the service worker
preforkServiceChannelOf(process.argv, index) in the service worker, its end to worker index

The ends start at descriptor 3, the supervisor channel first. A worker holds those ends and no other descriptor of the supervisor's. Like every nish:net socket they never block, so wait for one in the readiness loop alongside the listener. Where they are rides in one more argument, --nish-prefork-channels=, just before the worker's index; preforkArgs drops it with the index.

import { NET_UNIX_STREAM, netRead } from "nish:net";
import {
  PREFORK_STOPPED,
  PreforkConfig,
  preforkServiceChannel,
  preforkServiceChannelOf,
  preforkSupervise,
} from "nish/net/prefork";

class Log {
  lines: i32 = 0;
}

// The supervisor: worker `index` wrote on `fd`, or hung up (`events & 4`).
const heard = (log: Log, index: i32, fd: i32, events: i32): void => {
  const buf = new Array<u8>(256);
  while (netRead(fd, buf, 0, buf.length) > 0) {
    log.lines = log.lines + 1;
  }
};

export const main = (): number => {
  const config = new PreforkConfig();
  config.supervisorChannels = true;
  config.serviceWorker = 0;
  config.channelKind = NET_UNIX_STREAM;
  const worker = preforkSupervise(process.argv, config, new Log(), heard);
  if (worker < 0) {
    return worker === PREFORK_STOPPED ? 0 : 1;
  }
  if (worker === 0) {
    // The service worker: its end to worker 1, say.
    const one = preforkServiceChannelOf(process.argv, 1);
    // ... serve the others over their ends ...
    console.log(`worker 1 is on descriptor ${one}`);
  } else {
    const service = preforkServiceChannel(process.argv);
    // ... ask the service worker over `service`, serve clients ...
    console.log(`the service worker is on descriptor ${service}`);
  }
  return 0;
};

The callback runs in the supervisor each time a worker's end is readable or has hung up, with events as pollWait gives it: 1 readable, 4 added on a hang-up. Read everything waiting, since an end left readable calls it again at once; write back if you like; never close the end. The supervisor closes it when the worker hangs up or is reaped, and a restarted worker comes with a new one. The state is whatever you pass, a class instance to keep counts in, since a function cannot capture.

A restart restarts the group. A socketpair cannot be handed to a process that is already running, so with a service worker the workers live and die together: when any of them exits, the supervisor sends the rest SIGTERM (SIGKILL after graceMs), and once none is left starts them all again on fresh pairs, after the backoff of the one that exited first. No worker is left holding an end to a peer that is gone. With only supervisor channels, a worker restarts on its own with a new pair. With restart off nothing is signalled: a worker sees its peer's exit as a hang-up on its end.

A serviceWorker below -1 or not below the worker count, more than 64 workers with a service worker (the most descriptors one child is handed: 63 with listenPort, 62 with listenUdp too), listenUdp with a negative listenPort, a channelKind that is neither kind, or seqpacket channels off Linux stop the program with a panic.

Serving a file

A file bigger than you want in memory goes out the way it is read: a window at a time. Open it once with fileOpen, answer with fileSize as the length, and move each window from fileReadAt into the response with Http1Connection.write. Each connection keeps only an i64, how far into the file it has got, and every connection shares one window, so a server that sends a 1 GiB file holds what it holds for a 64 KiB one.

The output is the bound. write takes as much of the window as there is room for and answers how much, or H1_AGAIN when there is none; stop there, keep the position, and go on from it when next(slot) answers H1_WRITE, which it does once half the output is free. A window taken in part is read again from the new position:

const pump = (conn: Http1Connection, file: i32, size: i64, window: u8[], at: i64): i64 => {
  let pos: i64 = at
  while (pos < size) {
    const left: i64 = size - pos
    const want: i32 = left < toI64(window.length) ? toI32(left) : toI32(window.length)
    const n: i32 = fileReadAt(file, window, toI32(0), want, pos)
    if (n <= 0) {
      conn.abort() // the file shrank or failed: the length can no longer be met
      return toI64(-1)
    }
    const took: i32 = conn.write(window, toI32(0), n)
    if (took === H1_AGAIN) {
      return pos // the output is full: wait for H1_WRITE
    }
    if (took < 0) {
      conn.abort()
      return toI64(-1)
    }
    pos = pos + toI64(took)
    if (took < n) {
      return pos
    }
  }
  conn.end()
  return toI64(-1) // done
}

This pump moves the body only, and answers -1 alike when the body is sent and when it aborted, since either way the connection owes nothing more. The example also keeps a position for a response head that found no room in the output and retries respond at the next H1_WRITE.

respond takes the body's length as an i32, so one response carries at most 2^31 - 1 bytes; refuse a larger file before you listen. A read that comes back short of the length you announced means the file changed under you, and abort() closes the connection rather than leave the client waiting for bytes that will not come. A HEAD request (conn.parser.methodIs("HEAD")) needs the head and no reads at all. examples/file-server.ts is the whole server: file-server big.mp4 8080, then curl -s http://127.0.0.1:8080/ | cmp - big.mp4. Serving a 64 MiB file to a slow reader, its peak memory is the same as serving a 64 KiB one.

Names: nish/net/dns

A socket call takes an address. nish/net/dns turns a name into addresses without leaving your loop: a DnsClient joins the loop you made with pollCreate, under a range of tokens of its own, sends its queries on sockets it opens, and answers each lookup with the 18-byte addresses tcpConnect and udpSendTo take, the port you gave already in them. Your loop hands it the tokens it owns and the time; it never waits and never reads the clock itself, so a slow lookup costs your other sockets nothing.

import { DnsClient } from "nish/net/dns";
import { DNS_RESOLVE_PENDING, DnsResolver } from "nish/net/dns-resolver";
import { DnsConfig, dnsParseResolvConf } from "nish/net/dns-config";
import { DNS_ADDRESS_BYTES } from "nish/net/dns-message";
import { pollCreate, pollWait, tcpConnect } from "nish:net";
import { monotonicNanos } from "nish:process";

export const main = (): i32 => {
  // No file read here: an empty resolv.conf means the servers on 127.0.0.1 and ::1.
  const config = new DnsConfig();
  dnsParseResolvConf(new Array<u8>(0), 0, 0, config);
  const loop: i32 = pollCreate();
  const dns = new DnsClient(new DnsResolver(config, null, 4), loop, 1000);
  const lookup: i32 = dns.start("example.com", 443, monotonicNanos());
  if (lookup < 0) {
    console.log(`not started: ${lookup}`);
    return 1;
  }
  const out: u8[] = new Array<u8>(16 * DNS_ADDRESS_BYTES);
  const ttls: i32[] = new Array<i32>(16);
  const ready: i32[] = new Array<i32>(16);
  let found: i32 = dns.result(lookup, out, ttls);
  while (found === DNS_RESOLVE_PENDING) {
    const count: i32 = pollWait(loop, ready, dns.timeoutMs(monotonicNanos(), -1));
    const now: i64 = monotonicNanos();
    for (let k: i32 = 0; k < count; k++) {
      if (dns.owns(ready[2 * k])) {
        dns.ready(ready[2 * k], ready[2 * k + 1], now);
      }
    }
    dns.tick(now);
    found = dns.result(lookup, out, ttls);
  }
  if (found <= 0) {
    console.log(`no address: ${found}`);
    return 1;
  }
  // The first address, port 443 already in it, is what tcpConnect takes.
  const addr: u8[] = new Array<u8>(DNS_ADDRESS_BYTES);
  for (let i: i32 = 0; i < DNS_ADDRESS_BYTES; i++) {
    addr[i] = out[i];
  }
  const fd: i32 = tcpConnect(addr);
  console.log(`${found} addresses, connecting: ${fd >= 0}`);
  return 0;
};

start answers a lookup number, or a negative code when it cannot start one: -44 every slot is in use, and -1 to -6 (DNS_ERR_* of nish/net/dns-message) the name is not one, such as "" or a label over 63 bytes. Check it before you pass it to result, which panics on a number that is not a lookup.

A lookup answers its address count, A records first, or a negative code: -41 the name has no address (it does not exist, or exists with no A or AAAA record), -42 no server answered, -43 an alias chain ran past eight aliases or led back to the name asked (a loop inside one reply reads as -41), and from resolveHost -45 its time ran out. Another negative number is the errno of a socket or of pollWait that failed.

resolveHost answers start's codes too, and waits for both the A and the AAAA answer until the time runs out; then a family that answered with addresses is returned on its own, and only a lookup with none yet is -45. localhost, an address literal and a name in the hosts table answer at once, with no packet. For a command-line tool or a program's start-up, resolveHost(name, timeoutMs, port, out, ttls) reads /etc/resolv.conf and /etc/hosts and blocks until the lookup ends. The standard library page has both in full.

TLS to a server: tlsConnect

nish/net/tls-tcp's tlsConnect(addr, serverName, alpn, timeoutMs) takes an address like tcpConnect's, connects, runs a TLS 1.3 handshake and checks the server's certificate for serverName, blocking until it has or the time has run out. Like the socket calls it never looks a name up: resolve one first, as above, and pass the name you meant as serverName. The client's writeAll and readWait then write and read under a timeout, for a program with no loop of its own. Their answers, and how to drive the client from your own loop instead, are on TLS client, with how to trust roots other than the system's.

HTTP/2 without TLS: nish/net/http2-cleartext

Cleartext has no confidentiality and no integrity. Anyone on the path reads every request and response, cookies included, and can change them. Serve h2c only behind a proxy that terminates TLS, on loopback, or on a private network. For an open network use nish/net/http2-tls.

Http2CleartextServer serves HTTP/2 over plain TCP to a client that already knows the port speaks it ("prior knowledge", RFC 9113 §3.3): the client's first bytes are HTTP/2's 24-byte connection preface. There is no Upgrade: h2c, which RFC 9113 deprecates, and no HTTP/1.1 on the same port. It is nish/net/http2-tls's loop with TCP in place of TLS: the same accept, readable, writable, next, flush, close and rest, and the same interest bits (read 1, write 2, done 16).

import { pollAdd, pollModify, tcpListen } from "nish:net";
import { H2_ERROR, H2_NEED_MORE, H2_REQUEST, Http2Config, Http2Connection } from "nish/net/http2";
import { H2C_DONE, Http2CleartextServer } from "nish/net/http2-cleartext";

const LISTENER: i32 = 64; // past every slot's token

const start = (loop: i32): Http2CleartextServer => {
  const listener: i32 = tcpListen("127.0.0.1", 8080, 128);
  pollAdd(loop, listener, 1, LISTENER);
  // 64 slots; a client has 10 seconds to send its preface.
  return new Http2CleartextServer(new Http2Config(), listener, 64, 10000);
};

const serve = (loop: i32, server: Http2CleartextServer, slot: i32, events: i32): void => {
  let wants: i32 = (events & 5) !== 0 ? server.readable(slot) : server.writable(slot);
  const none: u8[][] = [];
  let event: i32 = server.next(slot);
  while (event !== H2_NEED_MORE && event !== H2_ERROR) {
    const conn: Http2Connection = server.connection(slot);
    if (event === H2_REQUEST) {
      conn.respond(conn.stream, 200, none, none, true);
    }
    event = server.next(slot);
  }
  wants = server.flush(slot);
  if ((wants & H2C_DONE) !== 0) {
    server.close(slot);
  } else {
    pollModify(loop, server.fd(slot), wants & 3, slot);
  }
};

When the listener's token comes back, server.accept() answers the new slot, which you pollAdd under its own number as the token, or -105 (H2C_POOL_FULL) when every slot is busy, having closed that connection. A client that sends anything but the preface, an HTTP/1.1 request line included, is closed at the first byte that differs and is sent nothing. One that sends part of it and stops holds its slot until you call server.expire(monotonicNanos()) past the timeout the constructor took; call it every so often, say whenever pollWait returns. Once the preface is in, the slot has no clock of its own: close one you judge idle yourself. curl --http2-prior-knowledge http://127.0.0.1:8080/ talks to it. What it refuses and why is its security record.

Trusting a peer's address

A TCP peer's address is proven: the handshake that tcpAccept completes needs answers sent to it. A UDP datagram's source address is only what its sender wrote, so anyone can send one that names another host. A server that caps or allows clients by address keys that on a UDP address only once a handshake has proven it.

For QUIC, Http3Server.addresses[slot] is the source of the slot's first datagram, and it is proven once server.quic(slot).addressValidated is set: by a Handshake packet from the client, or by a Retry token when the server's config turns retry on. The relay example in examples/relay counts a client against its per-address cap only from that point.

Under Node

A program that uses nish:net type-checks with npm run check, but it has no reading under Node: Node's sockets become ready only to Node's own event loop, which a program that owns its loop never returns to, so each call throws there, netMss included. Run it natively.