nish/net/http1-client-codec

The client side of HTTP/1.1, sans-IO: a request writer and an incremental response parser, bytes in, events and bytes out, and no socket anywhere. nish/net/http1-codec is the server side; the event constants and the chunk writers are shared and imported from there.

import {
  HTTP1_E_BARE_LF,
  HTTP1_E_BODY_LIMIT,
  HTTP1_E_CHUNK,
  HTTP1_E_CHUNK_LINE_LIMIT,
  HTTP1_E_CODING,
  HTTP1_E_COUNT_LIMIT,
  HTTP1_E_FIELD,
  HTTP1_E_FRAMING,
  HTTP1_E_HEADER_LIMIT,
  HTTP1_E_INFORMATIONAL_LIMIT,
  HTTP1_E_LENGTH,
  HTTP1_E_NONE,
  HTTP1_E_STATUS_LINE,
  HTTP1_E_STATUS_LINE_LIMIT,
  HTTP1_E_SWITCH,
  HTTP1_E_TRAILER_LIMIT,
  HTTP1_E_TRUNCATED,
  HTTP1_E_VERSION,
  HTTP1_INFORMATIONAL,
  HTTP1_MAX_CHUNK_LINE,
  HTTP1_MAX_INFORMATIONAL,
  Http1ResponseParser,
  http1RequestHead,
  http1UpgradeRequestHead,
  http1WriteRequestHead,
} from "nish/net/http1-client-codec";
import { HTTP1_NEED_MORE, HTTP1_HEAD, HTTP1_BODY, HTTP1_END, HTTP1_ERROR } from "nish/net/http1-codec";
import { Http1ResponseParser } from "nish/net/http1-client-codec";

const p = new Http1ResponseParser(maxStatusLine, maxHeaderBytes, maxHeaders, maxBody, maxTrailerBytes);
p.expect(false, false);
p.feed(received, 0, n); // and p.finish() when the server closes
let event: i32 = p.next();
while (event !== HTTP1_NEED_MORE) {
  // HTTP1_HEAD: p.status, p.reason, p.header("content-type"), ...
  // HTTP1_BODY: p.body[p.bodyOff .. p.bodyOff + p.bodyLen)
  // HTTP1_ERROR: p.error and p.errorText say why; close
  event = p.next();
}

The writer is http1RequestHead, which answers a request line and header section with the framing field the body needs, and http1WriteRequestHead, which writes the same head into a caller's buffer with nothing allocated. A chunked body is written with http1Chunk and http1LastChunk from nish/net/http1-codec. The writer refuses anything that would let a caller's string end a line early or frame the body itself — the request-splitting and request-smuggling shapes. http1UpgradeRequestHead is the one way to ask the server to switch protocols: it writes Upgrade and Connection: Upgrade itself, and every other writer refuses a Connection that lists upgrade.

The parser

The parser is a push-pull machine, like the server's. A caller tells it what it asked (expect), feeds whatever slice the socket gave it, cut anywhere, and calls next() until it answers HTTP1_NEED_MORE:

The events are HTTP1_INFORMATIONAL (an interim 1xx response was read; the final one follows), HTTP1_HEAD (the final response's status line and header section are in), HTTP1_BODY (some body bytes, de-chunked), HTTP1_END (the response is complete; with keepAlive() the next one follows), HTTP1_UPGRADE (a CONNECT was answered with 2xx, or a request that expectUpgrade named was answered with 101, and the connection is a tunnel now: tunnelBytes() answers what was fed past the head), HTTP1_CLOSED (the connection is done: the server closed it between responses, or the last response was not kept alive) and HTTP1_ERROR.

The last three repeat on every later call. Only complete lines are acted on, body bytes are counted, never guessed, and every refusal is decided by the same byte wherever the input was cut, so the sequence of events, the concatenated body and the error code do not depend on the split — tests/link/net_http1_client feeds a corpus at every split point.

How a response ends

How a response ends (RFC 9112 §6.3): a response to HEAD, a 1xx, 204 or 304 has no body whatever its fields say; a 2xx to CONNECT is a tunnel; a Transfer-Encoding: chunked response is de-chunked; a Content-Length response is that many bytes; anything else is read until the server closes, which finish() reports. A connection closed anywhere else is HTTP1_E_TRUNCATED, so a cut-short body is never taken for a whole one.

What it refuses, each with its own code in error: a malformed status line, an HTTP major version other than 1, a bare LF, a malformed field line (an obs-fold, whitespace before the colon, a control byte in a value), a 101 to a request that did not ask to switch protocols, Transfer-Encoding together with Content-Length, a transfer coding other than one chunked or one in an HTTP/1.0 response, a Content-Length that is not a decimal or whose repeats disagree, a malformed chunk, a status line, header section, field count, body or trailer section over the caller's limit, a chunk-size line over HTTP1_MAX_CHUNK_LINE and more than HTTP1_MAX_INFORMATIONAL interim responses.

Each of these is a response-smuggling or a resource shape, and each is refused rather than repaired: a client that repairs one differently from a proxy disagrees with it about where the next response starts. An error is final.

Private helpers share the importing program's flat symbol namespace (docs/wp26-stdlib.md §3e), which is why each one carries the module's name.

Written from RFC 9112 and RFC 9110, not ported from another implementation.

Functions

http1RequestHead

http1WriteRequestHead's head in an array of its own, sized to fit, so there is no room to run out of.

const http1RequestHead: (
  method: string,
  target: string,
  names: string[],
  values: string[],
  bodyLength: i64,
) => u8[] | null
methodstringrequired

The request method, a token such as GET.

targetstringrequired

The request target in the form the method may send.

namesstring[]required

The field names; exactly one is Host, which is written first, and the rest follow in order.

valuesstring[]required

The value of each field in names, at the same index.

bodyLengthi64required

The body's length in bytes, or HTTP1_CHUNKED (-1) or HTTP1_NO_BODY (-2).

returnsu8[] | null

The head, ending in the empty line, or null for everything http1WriteRequestHead refuses and for a head over 1 GiB.

http1UpgradeRequestHead

A request that asks the server to switch the connection to upgrade (RFC 9110 §7.8), such as a WebSocket opening handshake: http1RequestHead's head with no body, and Upgrade: <upgrade> and Connection: Upgrade written after the caller's fields. Answers null for everything http1RequestHead refuses, an upgrade that is not one token, and an Upgrade or Connection among the caller's fields, which would contradict the two this writes. A response to it is read after expect(false, false) and then expectUpgrade(), so that its 101 answers HTTP1_UPGRADE.

const http1UpgradeRequestHead: (
  method: string,
  target: string,
  names: string[],
  values: string[],
  upgrade: string,
) => u8[] | null
methodstringrequired

The request method, a token; a WebSocket handshake sends GET.

targetstringrequired

The request target in the form the method may send.

namesstring[]required

The caller's field names; exactly one is Host, which is written first, and the rest follow in order, before the two upgrade fields.

valuesstring[]required

The value of each field in names, at the same index.

upgradestringrequired

The protocol asked for, one token such as websocket; an empty one answers null.

returnsu8[] | null

The head, ending in the empty line, or null when any of the above is refused or the head is over 1 GiB.

http1WriteRequestHead

An HTTP/1.1 request line and header section (RFC 9112 §3, §5), ending in the empty line, written into out at at with nothing allocated, and answers where it ends. Host goes first, as RFC 9110 §7.2 asks, then the caller's other fields in order, then the framing field bodyLength asks for: Content-Length: n for n >= 0, Transfer-Encoding: chunked for -1 (HTTP1_CHUNKED), and neither for -2 (HTTP1_NO_BODY).

Answers HTTP1_REFUSED for a method that is not a token; a target that is not a form the method may send (see the guide) or that holds a space, a control byte, a non-ASCII byte or a #; a field name that is not a token or a value holding CR, LF or another control byte (which would end the line early and let the rest be read as a field or a second request); names and values of different lengths; not exactly one Host; a Content-Length or Transfer-Encoding among the caller's fields, since the framing is decided here; a Connection that lists upgrade, which only http1UpgradeRequestHead may write; and a bodyLength below -2.

Answers HTTP1_NO_ROOM when it is well formed but does not fit. Either way out is untouched.

const http1WriteRequestHead: (
  out: u8[],
  at: i32,
  method: string,
  target: string,
  names: string[],
  values: string[],
  bodyLength: i64,
) => i32
outu8[]required

The buffer the head is written into; nothing outside the head's bytes changes.

ati32required

The index in out the request line starts at; a negative one answers HTTP1_NO_ROOM.

methodstringrequired

The request method, a token such as GET, written as given.

targetstringrequired

The request target in the form the method may send, written as given.

namesstring[]required

The field names, matched case-insensitively; exactly one is Host, which is written first, and the rest follow in order.

valuesstring[]required

The value of each field in names, at the same index.

bodyLengthi64required

The body's length in bytes, or HTTP1_CHUNKED (-1) or HTTP1_NO_BODY (-2).

returnsi32

The index in out just past the empty line, or HTTP1_REFUSED or HTTP1_NO_ROOM.

Classes

Http1ResponseParser

One connection's response parser: responses on a kept-alive connection follow one another through the same parser, each after its expect.

class Http1ResponseParser
Member
new Http1ResponseParser(maxStatusLine: i32, maxHeaderBytes: i32, maxHeaders: i32, maxBody: i64, maxTrailerBytes: i32) Each i32 limit is clamped into 0 to 2^30 and maxBody into 0 to 2^53, so that the sums the parser makes of them cannot overflow.
body: u8[] The buffer the last HTTP1_BODY's bytes are in, at bodyOff, bodyLen long.
bodyLen: i32 How many bytes the last HTTP1_BODY holds.
bodyOff: i32 Where the last HTTP1_BODY's bytes start in body.
bodyTotal: i64 De-chunked body bytes so far, against maxBody.
chunked: boolean Whether the response's body is chunked.
closed: boolean Whether finish() said the server closed the connection.
contentLength: i64 The declared body length, or -1 when the response has none or is chunked or read until close.
error: i32 Why next() said HTTP1_ERROR: one of the HTTP1_E_ codes.
errorText: string What was wrong, once next() said HTTP1_ERROR; for a log.
keepStrings: boolean Whether names, values and reason are filled; fieldsAsSpans() clears it.
maxBody: i64 The most body bytes a response may carry, de-chunked; past it the parser refuses.
minor: i32 The minor version: 0 for HTTP/1.0, 1 for HTTP/1.1 and anything above it.
names: string[] The field names, in lowercase, in the order they arrived; values beside them, OWS trimmed. These and reason are filled only while the parser keeps strings, which it does until fieldsAsSpans(); fields(), fieldName, fieldValue, reasonText and header answer in either mode.
reason: string The reason phrase, which RFC 9112 §4 says a client ignores; for a log.
status: i32 The status code of the response last read.
values: string[] The field values, each beside its name in names.
wasConnect: boolean What expect said of the request this response answers: whether it was a CONNECT.
wasHead: boolean What expect said of the request this response answers: whether it was a HEAD.
wasUpgrade: boolean Whether expectUpgrade said the request asked to switch protocols.
buffered(): i32 How many bytes were fed and not consumed yet.
expect(methodWasHead: boolean, wasConnect: boolean): void Says what the request this response answers was: a response to HEAD has no body, and a 2xx to CONNECT is a tunnel. It takes effect only while the parser waits for a response's first status line, and the parser forgets it after that response's HTTP1_END; anywhere else it does nothing, so a pipelining caller says it once per response, after the last one's end.
expectUpgrade(): void Says the request this response answers asked to switch protocols (it was written by http1UpgradeRequestHead), so a 101 is that switch: the parser answers HTTP1_HEAD with its status and fields readable, HTTP1_END, and then HTTP1_UPGRADE, and tunnelBytes() answers what was fed past the head. Any other status is read as it would be without it. It is said after expect, which clears it, and like expect it takes effect only while the parser waits for a response's first status line. Without it a 101 is refused with HTTP1_E_SWITCH: a server may not switch a connection whose client never asked.
feed(data: u8[], off: i32, len: i32): void Appends data[off .. off + len) to the input. A window outside data panics. After finish(), or once the parser has closed or refused, the input is dropped; in a tunnel it is kept for tunnelBytes().
fieldName(i: i32): string Field i's name, in lowercase, as a fresh string.
fields(): i32 How many fields the response last read has; trailers are not kept.
fieldsAsSpans(): void Stops filling names, values and reason, which then stay empty: a response's fields are read through fields(), fieldName, fieldValue, reasonText and header, which make a string only when asked. A parser kept for many responses allocates nothing per response this way.
fieldValue(i: i32): string Field i's value, OWS trimmed, as a fresh string.
finish(): void Says the server closed the connection: a body read until close ends there, a parser waiting for a response with nothing fed is closed, and anything else is HTTP1_E_TRUNCATED once the bytes fed before it are read.
header(name: string): string | null The value of the first field named name (any case).
keepAlive(): boolean Whether the connection may carry the next request once this response has ended (RFC 9112 §9.3): a 1.1 response unless Connection lists close, a 1.0 response only when it lists keep-alive, and never one read until close, a tunnel or an error.
next(): i32 The next event the input holds; see the module comment.
reasonText(): string The reason phrase, as a fresh string.
reset(): void Makes the parser as it was new, for the next connection, keeping its limits, its mode and its buffers: what was fed and not read is dropped, and an error or a close is forgotten.
tunnelBytes(): u8[] In a tunnel: every byte fed past the response head, as a fresh array.

Limits

4,096

HTTP1_MAX_CHUNK_LINE: i32

The longest chunk-size line, extension included, the parser reads.

32

HTTP1_MAX_INFORMATIONAL: i32

The most interim 1xx responses the parser reads before a final one.

Constants

3

HTTP1_E_BARE_LF: i32

error: a line ended in a LF without its CR (RFC 9112 §2.2).

14

HTTP1_E_BODY_LIMIT: i32

error: a declared or a received body longer than maxBody.

9

HTTP1_E_CHUNK: i32

error: a malformed chunk size, chunk extension or chunk terminator (RFC 9112 §7.1).

16

HTTP1_E_CHUNK_LINE_LIMIT: i32

error: a chunk-size line longer than HTTP1_MAX_CHUNK_LINE.

7

HTTP1_E_CODING: i32

error: a transfer coding list that is not one chunked, or one in an HTTP/1.0 response.

13

HTTP1_E_COUNT_LIMIT: i32

error: more fields, trailers included, than maxHeaders.

4

HTTP1_E_FIELD: i32

error: a malformed field line: an obs-fold, a bad name, whitespace before the colon, a control byte.

6

HTTP1_E_FRAMING: i32

error: both Transfer-Encoding and Content-Length (RFC 9112 §6.1).

12

HTTP1_E_HEADER_LIMIT: i32

error: a header section longer than maxHeaderBytes.

17

HTTP1_E_INFORMATIONAL_LIMIT: i32

error: more than HTTP1_MAX_INFORMATIONAL interim responses.

8

HTTP1_E_LENGTH: i32

error: a Content-Length that is not a run of digits, or values that differ (a value past 2^53 differs from any).

0

HTTP1_E_NONE: i32

error: nothing is wrong.

1

HTTP1_E_STATUS_LINE: i32

error: the status line is not HTTP/1.x, a three-digit status from 100 to 599 and an optional reason.

11

HTTP1_E_STATUS_LINE_LIMIT: i32

error: a status line longer than maxStatusLine.

5

HTTP1_E_SWITCH: i32

error: a 101 to a request that did not ask to switch protocols (see expectUpgrade).

15

HTTP1_E_TRAILER_LIMIT: i32

error: a trailer section longer than maxTrailerBytes.

10

HTTP1_E_TRUNCATED: i32

error: the connection closed before the response ended.

2

HTTP1_E_VERSION: i32

error: the response's HTTP major version is not 1.

7

HTTP1_INFORMATIONAL: i32

next(): an interim 1xx response was read; status and the fields are its.