nish/net/upstream

Outbound HTTP/1.1 calls from a route handler that never block its worker: a pool of keep-alive connections to one origin, registered on the HttpHandler's own poll loop and read through the nish/net/http1-client-codec codec.

import {
  UPSTREAM_E_BLOCKED,
  UPSTREAM_E_CLOSED,
  UPSTREAM_E_CONNECT,
  UPSTREAM_E_FULL,
  UPSTREAM_E_LOOKUP,
  UPSTREAM_E_NONE,
  UPSTREAM_E_ORIGIN,
  UPSTREAM_E_REQUEST,
  UPSTREAM_E_RESET,
  UPSTREAM_E_TIMEOUT,
  Upstream,
  UpstreamLimits,
  UpstreamResponse,
} from "nish/net/upstream";
const h = new HttpHandler(new HandlerLimits(), listener, router)
const users = new Upstream(h, "http://127.0.0.1:9000", new UpstreamLimits())
while (true) {
  h.wait(100) // one turn: the handler's sockets and the upstream's
  while (h.ready()) {
    const req = h.take()
    switch (req.route) {
      case USER: if (users.get(req, "/v1/users/7") !== UPSTREAM_E_NONE) { h.problem(req, 503, "Busy", "") } break
      default: h.refuse(req)
    }
  }
  while (users.ready()) { // the answers that arrived this turn
    const r = users.take() // r.req is the request that asked
    if (r.ok) { h.send(r.req, r.status, JSON, r.body, 0, r.length) } else { h.problem(r.req, 502, "Bad Gateway", "") }
    users.release(r)
  }
}
`get` returns at once

It writes the request head into a call slot taken from a pool, then onto a free connection, or queues it until one is free, and the program goes on serving. The asking request stays held, as nish/net/handler allows. Each connection is watched on the handler's loop under a watch id (UpstreamLimits.firstId and up), so the program's next h.wait wakes for the upstream as well as for its clients; ready then reads what arrived, once per turn, and hands back every finished call.

Every call ends exactly once, as a call

A response the codec reads to its end is ok with its status and body, whatever the status. Anything else is ok === false with an error: one of the codec's HTTP1_E_ refusals (a malformed status line, a bare LF, conflicting framing, a body past maxBody, a response cut short), or one of the UPSTREAM_E_ codes below (the origin was unusable, the connection failed or closed, the deadline passed, the path could not be written). The program answers its own client either way and gives the call back with release.

A kept-alive connection is checked before it is reused

One the origin closed or sent bytes on while it was idle is closed instead, so no call is written to a dead socket or handed bytes it did not ask for. A call the origin drops unanswered on a reused connection anyway (it closed as the request crossed) is made once more, on a new connection; a GET is safe to repeat. Two Upstreams on one handler must be given watch ids that do not overlap: nothing checks it, and two with the default limits would both take ids 8 to 11, each taking the other's wake-ups for its own.

The origin is fixed at start-up

The constructor parses it, looks its host up (blocking, once, as a program's start-up may), and keeps one address; nothing is looked up again, so no later answer can move the pool somewhere else. When the origin comes from input — configuration, an environment variable, a request — set UpstreamLimits.outbound: the nish/net/outbound policy then checks the host name before the lookup and the address after it, and a refused origin fails every call with UPSTREAM_E_BLOCKED. Only http: is spoken; TLS is not, so https: is UPSTREAM_E_ORIGIN.

Every cap is set at start-up. The connection count, the call slots (calls queued, in flight and finished but not yet released), the request head's room, the response limits the codec enforces, and a deadline per call. The slots, their head buffers and the connections' read buffers are made by the constructor; a call's body buffer grows to the largest body it has held and is kept. Each connection keeps one codec for its life and the next connection's, reset between them, in its spans mode: a response's fields are read as spans of a buffer the codec reuses, so a warm pool allocates nothing per call.

docs/security/upstream.md is the record. Private helpers share the importing program's flat symbol namespace (docs/wp26-stdlib.md §3e), which is why each one carries the module's name.

Written from RFC 9110 and RFC 9112, not ported from another implementation.

Classes

Upstream

A keep-alive pool to one origin, on an HttpHandler's loop.

class Upstream
Member
new Upstream(h: HttpHandler, origin: string, limits: UpstreamLimits) Parses origin, looks its host up once (this blocks for at most limits.lookupMs), and makes every slot and connection buffer. Panics when a cap is below 1 or the watch ids run past the handler's. An unusable origin is not a panic: error says why, and every call fails with it.
error: i32 Not UPSTREAM_E_NONE when the origin is unusable: every call then fails with it.
errorText: string Why the origin is unusable; empty when it is not. For a log.
get(req: HttpRequest, path: string): i32 Starts GET path for req and returns at once: UPSTREAM_E_NONE when the call is the pool's, which hands it back through ready and take however it ends; UPSTREAM_E_FULL when every slot is taken, and then nothing is held and req is still the program's to answer. path is copied out before this returns.
openConnections(): i32 How many connections to the origin are open.
pending(): i32 How many calls hold a slot: queued, in flight, or finished and not yet released.
ready(): boolean Whether a finished call is waiting for take; reads the upstream's sockets once per handler turn.
release(r: UpstreamResponse): void Gives a taken call's slot back to the pool. Panics for a call not taken or already released: two calls sharing a slot would hand one request's answer to another.
take(): UpstreamResponse The next finished call, which the program gives back with release. Panics when none is ready.

UpstreamLimits

The caps of one Upstream, all read once by its constructor.

class UpstreamLimits
Member
new UpstreamLimits()
calls: i32 Call slots: calls queued, in flight, and finished but not yet released, at most.
connections: i32 Connections to the origin, at most; each takes one handler watch id.
firstId: i32 The first handler watch id the connections take: they use firstId to firstId + connections - 1.
lookupMs: i32 How long the constructor's one lookup of the origin's host may take.
maxBody: i64 The largest response body the codec accepts, at most 1 GiB; past it the call fails HTTP1_E_BODY_LIMIT.
maxHead: i32 Room for one request head: the request line, Host and the empty line.
maxHeaderBytes: i32 The most bytes a response's header section may take.
maxHeaders: i32 The most field lines a response may carry, trailers included.
maxStatusLine: i32 The longest status line a response may have, in bytes.
maxTrailerBytes: i32 The most bytes a response's trailer section may take.
outbound: OutboundPolicy | null Checks the origin's host and address when set; set it whenever the origin comes from input.
timeoutMs: i32 A call fails UPSTREAM_E_TIMEOUT this long after get if its response has not ended.

UpstreamResponse

One call: the request that asked, then, once ready hands it back, how it ended. The object is the pool's again after release.

class UpstreamResponse
Member
new UpstreamResponse(index: i32, headRoom: i32, placeholder: HttpRequest) An empty call slot; Upstream makes its pool of them.
body: u8[] The body, in body[0 .. length); the buffer is reused, so read length, not body.length.
error: i32 UPSTREAM_E_NONE, an HTTP1_E_ refusal of the codec, or an UPSTREAM_E_ code.
length: i32 How many bytes of body the response filled.
ok: boolean Whether the response was read to its end; error says why not.
req: HttpRequest The handler's request this call answers.
status: i32 The final status code, once the response's head has arrived.

Constants

34

UPSTREAM_E_BLOCKED: i32

The outbound policy refused the origin's host name or every address it has.

37

UPSTREAM_E_CLOSED: i32

The connection failed or closed before a response began.

35

UPSTREAM_E_CONNECT: i32

The connection to the origin could not be made.

39

UPSTREAM_E_FULL: i32

Every call slot is taken: get answers this and holds nothing.

33

UPSTREAM_E_LOOKUP: i32

The origin's host could not be looked up at start-up.

0

UPSTREAM_E_NONE: i32

The call was read to its end: ok.

32

UPSTREAM_E_ORIGIN: i32

The origin is not an http: URL with a host and nothing past its authority but /.

38

UPSTREAM_E_REQUEST: i32

The path does not start with / (an absolute-form or asterisk target is not this pool's to send), the codec refused to write it, or the head did not fit maxHead.

40

UPSTREAM_E_RESET: i32

The connection failed (it was reset, say) while the response was being read: what came is not the whole of it.

36

UPSTREAM_E_TIMEOUT: i32

The call's deadline passed before its response ended.