nish/net/http3-server

HTTP/3 over QUIC on one UDP socket: a pool of slots, each a reusable QuicConnection with an Http3Connection beside it, a QuicListener in front for the datagrams no slot owns, for a program that owns its readiness loop.

import {
  H3_SERVER_ANSWER_BURST,
  H3_SERVER_ANSWER_INTERVAL,
  H3_SERVER_RECEIVE_BURST,
  H3_SERVER_WHEEL,
  Http3CidIndex,
  Http3Server,
  Http3Wheel,
} from "nish/net/http3-server";
import { pollAdd, pollCreate, pollWait, udpBind } from "nish:net";
import { monotonicNanos } from "nish:process";
import { secret, wipe } from "nish:secret";
import { H3_ERROR, H3_NEED_MORE, Http3Config, Http3Connection } from "nish/net/http3";
import { Http3Server } from "nish/net/http3-server";
import { QuicServerConfig } from "nish/net/quic";
import { QUIC_LISTENER_ENTROPY_SIZE } from "nish/net/quic-listener";

// `quicConfig.alpn` is ["h3"]; `privateKey` is the P-256 key of its certificate.
const run = (quicConfig: QuicServerConfig, privateKey: u8[], rounds: i32): void => {
  const fd: i32 = udpBind("0.0.0.0", 4433, 0);
  const listenerEntropy: u8[] = new Array<u8>(QUIC_LISTENER_ENTROPY_SIZE);
  crypto.getRandomValues(listenerEntropy);
  const server = new Http3Server(quicConfig, new Http3Config(), fd, 64, listenerEntropy);
  const loop: i32 = pollCreate();
  pollAdd(loop, fd, 1, 0);
  const ready: i32[] = new Array<i32>(8);
  const key = secret(privateKey);
  for (let round: i32 = 0; round < rounds; round++) {
    pollWait(loop, ready, server.timeout(monotonicNanos() / 1000000));
    const nanos: i64 = monotonicNanos();
    const now: i64 = nanos / 1000000;        // milliseconds
    server.clock(nanos / 1000);              // the same instant in microseconds, for the pacer
    server.receive(now, key);                // every datagram waiting; signs handshakes with `key`
    server.tick(now);                        // the connections' timers
    let slot: i32 = server.ready();
    while (slot >= 0) {
      const h3: Http3Connection = server.connection(slot);
      let event: i32 = h3.next();
      while (event !== H3_NEED_MORE && event !== H3_ERROR) {
        // answer the event here: `h3.stream`, `h3.fields`, `h3.respond`, ...
        event = h3.next();
      }
      slot = server.ready();
    }
    server.flush(now);                       // what every touched slot has to send
  }
  wipe(key);
  server.stop();                             // releases every slot, wipes the static keys
};
Routing

A datagram goes to the slot that owns its first packet's Destination Connection ID, found through a hash index of every slot's connection IDs — its active local IDs, and the client's original DCID until the handshake is done — never by asking each slot. The index is keyed with bytes from the listener's entropy, so a client choosing its original DCIDs cannot pick ones that collide. A datagram no slot owns goes to the listener, which answers Version Negotiation, Retry or a stateless reset, or accepts a new connection into a free slot. With no slot free, the datagram is dropped and counted, and the client's own timers retry.

ALPN

The QUIC configuration's ALPN list is what the TLS handshake offers; a connection whose handshake chose anything but h3 is closed by its Http3Connection, before a byte of HTTP/3, with CRYPTO_ERROR no_application_protocol (0x0178; RFC 9114 §3.1).

Sending

flush writes each touched slot's datagrams with quicListenerTakeFlight, a paced flight at a time in one udpSendTo with GSO where the platform has it and one datagram at a time where it does not. Each flight goes with the ECN codepoint its connection marked it with (QuicFlight.ecn, RFC 9000 §13.4), so the peer's ACK_ECN counts are validated and a CE mark shrinks the window. Where the platform sends no ECN (Darwin, whose udpSendTo answers -95 to a marked send), the first marked send goes again unmarked and no slot marks after it.

A -95 is also what a platform without segmentation offload answers, so a refused GSO send goes again one datagram at a time, still marked, and only a -95 to one of those turns marking off. The pacer counts the microseconds clock gives it, so on a path whose round trip is under a millisecond it neither waits for the next millisecond to earn credit nor holds a slot back for a whole one; without clock it counts whole milliseconds. A slot whose HTTP/3 side sent GOAWAY and whose requests have all finished, every byte acknowledged, is closed with H3_NO_ERROR; a closed slot is freed once its CONNECTION_CLOSE is out.

Timers

Timers are a wheel of H3_SERVER_WHEEL one-millisecond buckets, a slot in at most one: after every datagram and every flush the slot is filed under its next deadline (its idle, loss and probe timers, and its pacer), and tick runs the buckets that are due. timeout finds the next filled bucket from a bitmap. A deadline past the wheel's horizon is filed at the horizon and filed again when it comes round. So no call scans the slots.

The HTTP/3 side's clock (Http3Connection.clock, H3-5) runs on every flush and files its deadline with the rest: a slot that goes idleTimeout without progress while no request is in hand is closed with H3_NO_ERROR, and one whose request head has waited headTimeout (a client trickling its HEADERS, the slowloris of HTTP/3) with H3_EXCESSIVE_LOAD, as the program writes no timer of its own. A client that only acknowledges or sends PING keeps its QUIC connection alive but not its slot. The clock times a slot from the first next that finds the handshake chose h3, whether or not the client has let this side's streams open; a slot the program runs another protocol on, such as hq-interop, is left to QUIC's idle timeout.

Caps, fixed at start-up: the slots, and inside each the QUIC configuration's streams and buffers and the Http3Config's field section, body chunk, DATA frame and two timeouts.

A slot's connections and address are built on the first Initial accepted into it and kept, so a pool sized for its peak costs at start-up only its per-slot tables (the index, the wheel, the queues), and holds at most size connections once the peak has come; a datagram an established connection reads, and every datagram it sends, allocates nothing; nor does a datagram no slot owns, which the listener reads where it lies and answers into scratch made once, nor a handshake into a slot built before, whose state is the slot's — docs/security/http3.md (H3-1, H3-3).

The Date

Every response carries one date field (RFC 9110 §6.6.1) from the pool's HttpDate, which formats the IMF-fixdate at most once a second for every slot; trailers carry none, and a program that sends its own date sends that one instead. server.date.pin(unixMs) fixes it, for a test that compares bytes.

Written from RFC 9114 §3 and RFC 9000 §5.2, not ported from another implementation. Private names carry the http3Server prefix (docs/wp26-stdlib.md §3e).

Classes

Http3CidIndex

The connection IDs of every slot, by hash: an open-addressed table keyed with salt, each ID copied into the entry it was filed as. Each slot has QUIC_CONN_LOCAL_CIDS + 1 entries, slot s's from s * (QUIC_CONN_LOCAL_CIDS + 1) up: its local connection IDs, then the client's original DCID. The table has at least twice as many places as entries, so it never fills.

class Http3CidIndex
Member
new Http3CidIndex(slots: i32, salt: u8[]) An empty index for slots slots.
salt: u8[] The hash's key: eight bytes from the listener's entropy.
find(buf: u8[], at: i32, length: i32): i32 The slot owning the ID buf[at .. at + length).
insert(e: i32, buf: u8[], at: i32, length: i32): void Files entry e as the ID buf[at .. at + length); an ID already filed under any entry is left there, and nothing is filed.
remove(e: i32): void Takes entry e out, moving later entries of its probe run back so none is lost.

Http3Server

The pool. slots are indices from 0 to size() - 1; connection(slot) is the HTTP/3 side of each and quic(slot) the QUIC side.

class Http3Server
Member
new Http3Server(quicConfig: QuicServerConfig, config: Http3Config, fd: i32, size: i32, listenerEntropy: u8[]) A pool of size slots serving HTTP/3 under config over QUIC under quicConfig on the UDP socket fd. listenerEntropy is the listener's QUIC_LISTENER_ENTROPY_SIZE random bytes, whose first eight also key the connection-ID index (wiped in the caller's array, as the listener does). No slot's connections are made here: each slot's are built on the first Initial accepted into it. From here on quicConfig's statelessResetKey and retryTokenKey are the server's: stop wipes them, so the configuration is this server's alone.
accepted: i32 Connections accepted into a slot, for a log or a test; saturating at 2^31 - 1.
addresses: u8[][] Where each slot sends now: its client's address as the first datagram came from, then wherever the connection follows the client to (QuicConnection.sendAddress, copied here by flush before each slot's flight once quic(slot).pathKnown). Not validated until quic(slot).addressValidated is set (RFC 9000 §8.1): a Retry token the listener checked, with quicConfig.retry on, or a Handshake packet from the client. Before then it is the source of one UDP datagram, which anyone can forge, so a cap or an allow-list keyed on it waits for that. After the handshake it changes when the client's address does (a NAT rebinding, §9.3), first to an address not yet validated, so a cap keyed on the address a session started from keeps its own copy.
answer: QuicListenerAnswer What the listener answers into, made once: its reply and IDs are refilled in their own room.
arrivalEcn: i32 The ECN bits of the datagram being handed on (udpRecvFrom's second metadata word, 0 where the platform does not report them), which serve gives the slot's connection as its arrivalEcn, so that the server's ACKs echo the counts (RFC 9000 §13.4.1). receive sets it for every datagram it reads; a caller of datagram sets it before the call.
conns: Http3Connection[] The HTTP/3 side of each slot in quics, at the same index: what connection answers.
dataPool: BufferPool The bodyChunk-byte read buffers every slot's Http3Connection borrows, one at most each.
date: HttpDate The date every slot's responses carry, formatted once a second for all of them: each slot's Http3Connection.date is its bytes, refreshed by connection and ready. date.pin fixes its clock.
ecn: boolean Whether the platform sends ECN marks: true until a marked send answers -95, and every slot's ecnMarking with it.
entropy: u8[] QUIC_CONN_ENTROPY_SIZE bytes of scratch, refilled from crypto.getRandomValues for each slot built and each connection accepted; it holds the last one's until the next, and stop zeroes it.
fd: i32 The UDP socket every datagram is read from and sent on, as the constructor was given it.
index: Http3CidIndex The connection IDs of every slot holding a connection, which route each datagram to its slot.
limited: i32 Datagrams no slot owns dropped past the answer budget before the listener read them, saturating.
listener: QuicListener Answers the datagrams no slot owns, and accepts new connections into free slots; stop releases it.
quicConfig: QuicServerConfig The QUIC configuration of the listener and every slot's connection; stop wipes its two keys.
quics: QuicConnection[] The slots built so far, 0 up: each built on the first Initial accepted into it, or the first ask for it.
refused: i32 Initials dropped because no slot was free, saturating at 2^31 - 1.
sectionPool: BufferPool The maxFieldSectionSize-byte HEADERS buffers every slot's Http3Connection borrows around a decode.
streamPools: QuicStreamPools | null The stream buffers every slot's connection borrows from, made by the constructor for the stream table quicConfig gives every slot: null when its streams have none. Each pool makes a buffer only when a stream first needs one, so an idle server holds none (QUIC-3). It is made even for a configuration quicConnConfigFits refuses; a slot's connection shares it only when its stream buffers are the pools' size.
wheel: Http3Wheel Each slot's next deadline, filed by flush and run by tick; timeout reads it.
busy(): i32 How many slots hold a connection.
clock(micros: i64): void Gives the server the caller's monotonic clock in microseconds: the instant the now of the calls that follow names, monotonicNanos() / 1000 where now is monotonicNanos() / 1000000. The pacer counts it; every timer still counts now. One that is not inside now's millisecond is not read.
connection(slot: i32): Http3Connection The HTTP/3 connection of slot, its date brought up to now; a slot out of range names the first. A slot not built yet is built here, with every slot before it, so a program that binds something to each slot's connection up front gets the one its slot keeps.
datagram(buf: u8[], at: i32, len: i32, address: u8[], now: i64, key: Secret<u8[]>): void One datagram, buf[at .. at + len), from address: to the slot that owns its first packet's Destination Connection ID, or to the listener. One too short to carry its DCID, or whose DCID is longer than QUIC_MAX_CID_LENGTH, or a window outside buf, is dropped.
flush(now: i64): void Sends what every touched slot has, as far as its pacer lets it, closes a slot whose HTTP/3 side is done after GOAWAY, frees a slot whose connection has closed, and files each in the wheel under its next deadline.
holds(slot: i32): boolean Whether slot holds a connection.
quic(slot: i32): QuicConnection The QUIC connection of slot, built as connection builds it; a slot out of range names the first.
ready(): i32 The next slot with news for the program: call its connection's next until it has none.
receive(now: i64, key: Secret<u8[]>): i32 Reads every datagram waiting on the socket, in at most H3_SERVER_RECEIVE_BURST receives, and hands each to its slot or the listener, signing a handshake's CertificateVerify with key (ECDSA P-256) when one asks. A GRO receive is cut into its datagrams in place. Answers how many datagrams it read.
serve(slot: i32, buf: u8[], at: i32, len: i32, address: u8[], now: i64, key: Secret<u8[]>): void Hands slot's connection a datagram from address, signs when the handshake asks, and refiles its IDs and timer. The connection decides from the address whether the client moved (RFC 9000 §9.3); an address that is not 18 bytes long leaves it where it is.
size(): i32 How many slots the pool has, built or not.
stop(): void Stops the server for good and wipes the keys it holds: every built slot's connection is released (QuicConnection.release, which sends nothing), as is one built after, by connection or quic; the listener is released; and the configuration's statelessResetKey and retryTokenKey, the index's salt, the connections' entropy scratch and the listener's last answer (a Retry's or a reset's token) are zeroed. From then on every datagram is dropped and no connection is accepted.

The configuration's two keys are the server's once it is made with them: a program that keeps a copy of either, to start the next server with, wipes its own copy. The keys are zeroed in place, so give each server a QuicServerConfig of its own: any other listener, connection or server made with the same one would go on MACing Retry tokens and deriving stateless reset tokens under an all-zero key, which anyone can forge.
tick(now: i64): void Runs the timers due by now, and marks each slot it ran for flush.
timeout(now: i64): i32 Milliseconds until tick has a timer to run, for pollWait.
touch(slot: i32): void Marks slot as having news for the program, so that ready answers it, and something to send, so that the next flush sends it; a slot already marked is not marked twice.
unmark(): void The platform sends no ECN: no slot's connection marks from now on.

Http3Wheel

Slots filed by the millisecond of their next deadline: H3_SERVER_WHEEL buckets, each a linked list of its slots, and a bitmap of the buckets that hold any, so next passes 64 empty buckets at a time. A slot is filed in at most one bucket.

class Http3Wheel
Member
new Http3Wheel(slots: i32) An empty wheel whose cursor is not started.
cursor: i64 Every bucket before this time has been run.
dueAt: i64[] Per slot, the deadline in milliseconds file last filed it for, before any clamp to the horizon.
file(slot: i32, due: i64, now: i64): void Files slot under due, taking it out of any bucket it was in first. A due before the cursor is filed at the cursor, one past the horizon (H3_SERVER_WHEEL - 1 milliseconds after the cursor) at the horizon.
next(now: i64): i32 Milliseconds from now to the first filled bucket, at most the horizon.
take(time: i64): i32 The slot at the head of the bucket for time, taken out of it.
unfile(slot: i32): void Takes slot out of its bucket.

Limits

256

H3_SERVER_RECEIVE_BURST: i32

The most datagrams one receive reads, so one busy socket cannot hold the loop.

Constants

16

H3_SERVER_ANSWER_BURST: i32

The answer budget's size: how many datagrams no slot owns that could get a stateless reset or a Version Negotiation packet may reach the listener at once.

100

H3_SERVER_ANSWER_INTERVAL: i64

The milliseconds that earn the answer budget one more, up to H3_SERVER_ANSWER_BURST.

2,048

H3_SERVER_WHEEL: i32

The timer wheel's buckets, one millisecond each: its horizon.