nish/net/postgres-carrier
PgCarrier, a PgPool's queries on an HttpHandler's loop, so a route handler queries PostgreSQL without blocking its worker.
import {
PG_CARRIER_BUILD,
PG_CARRIER_COMPLETE,
PG_CARRIER_DONE,
PG_CARRIER_ERROR,
PG_CARRIER_FAILED,
PG_CARRIER_FULL,
PG_CARRIER_MISSING,
PG_CARRIER_NONE,
PG_CARRIER_ROW,
PgCarrier,
PgCarrierLimits,
} from "nish/net/postgres-carrier";
import { HttpRequest } from "nish/net/handler"
import { PgConnection, pgTarget } from "nish/net/postgres"
import {
PG_CARRIER_BUILD, PG_CARRIER_DONE, PG_CARRIER_FAILED, PG_CARRIER_FULL,
PG_CARRIER_MISSING, PG_CARRIER_NONE, PG_CARRIER_ROW, PgCarrier, PgCarrierLimits,
} from "nish/net/postgres-carrier"
// A sketch: `pool` registered "article"; `handler` serves the routes;
// `ARTICLE` is the route's tag, `running` the server's loop condition,
// and `text` and `none` the program's content-type and empty body.
const carrier = new PgCarrier(new PgCarrierLimits(), pool, pgTarget("127.0.0.1", 5432), handler)
const answered: boolean[] = new Array<boolean>(256) // one per query number
while (running) {
handler.wait(50) // a timeout, so the connect timeout is checked
while (handler.ready()) {
const req: HttpRequest = handler.take()
if (carrier.run(req, ARTICLE) === PG_CARRIER_FULL) {
handler.send(req, 503, text, none, 0, 0)
}
}
let event: i32 = carrier.next()
while (event !== PG_CARRIER_NONE) {
if (event === PG_CARRIER_BUILD) {
const conn: PgConnection = carrier.connection()
conn.bindStart("article")
conn.paramInt(toI64(carrier.request().paramDecimal(0)))
conn.bindEnd()
carrier.send()
} else if (event === PG_CARRIER_ROW && !answered[carrier.query]) {
const conn: PgConnection = carrier.connection()
handler.send(carrier.request(), 200, text, conn.inputBuffer(), conn.colStart(0), conn.colLength(0))
answered[carrier.query] = true
} else if (event === PG_CARRIER_DONE || event === PG_CARRIER_MISSING || event === PG_CARRIER_FAILED) {
if (!answered[carrier.query]) {
handler.send(carrier.request(), event === PG_CARRIER_FAILED ? 503 : 404, text, none, 0, 0)
}
answered[carrier.query] = false
}
event = carrier.next()
}
}
carrier.close()
pool.dispose()
PgCarrier
A PgPool on an HttpHandler's loop: the pool's sockets sit in one epoll set of the carrier's own, which the handler watches beside its connections, so that a route handler queries PostgreSQL without blocking its worker.
PgCarrierLimits
The caps a PgCarrier is built with, all set before it starts: nothing grows after.
Why a module of its own
nish/net/postgres reads no socket, clock or entropy source; this module is
the one that does, and it is kept apart so that one stays sans-IO. The
carrier's sockets sit in an epoll set the handler watches (HttpHandler.watch), a
route handler hands a request to run (or to runInt and runText for an
attached PgBatcher), and next() answers PG_CARRIER_BUILD to bind a
plain query on connection(), its rows, and one final event —
PG_CARRIER_DONE, PG_CARRIER_MISSING or PG_CARRIER_FAILED — with
request() the HttpRequest to answer. It connects its pool's slots
itself, gives up on one that has not answered in connectTimeoutMs, and
sizes every table from PgCarrierLimits at construction.
Lifetime
A query number from run, runInt or runText, with its
tag and request(), holds until the next() after its final event,
which frees the number for reuse; a row's bytes on connection() hold
only until the next call. The carrier never answers a request itself: the
program answers each one once, on its first row or on its final event.
Limits. Every cap is fixed at construction (PgCarrierLimits), so a
full carrier answers PG_CARRIER_FULL rather than grow, and once the
connections are up nothing is allocated per query. There is no deadline on
a query once its connection is up; bound a slow statement with the
server's statement_timeout.
Errors
A connection that is refused, reset, closed or past its connect
timeout fails every query on it with PG_CARRIER_FAILED, failure the
connection's PG_ERR_* and socketError the socket's negative errno
(-110 for the timeout, 0 when the server closed cleanly; under TLS, see
below). A plain query
the server refuses answers PG_CARRIER_ERROR, then PG_CARRIER_FAILED
with failure PG_ERR_NONE; a batched query the server refuses answers
only PG_CARRIER_FAILED with failure PG_ERR_NONE.
Programming errors (a cap out of range, an unknown batcher id, send()
with no build open) panic.
TLS
With PgConfig.tls set to PG_TLS_REQUIRE or PG_TLS_VERIFY_FULL,
each connection opens with an SSLRequest and reads exactly the one byte of
its answer, so nothing the server sends in the clear reaches the protocol:
after N nothing more is read, and after S the next bytes are the
handshake's, so anything else fails it. S goes on to a TLS 1.3 handshake
(nish/net/tls/record-client) driven on the same loop without blocking, the
StartupMessage held until it is done; any other answer fails the connection
with PG_ERR_TLS. Under PG_TLS_VERIFY_FULL the server's chain is checked
with nish/net/tls/verify against PgConfig.tlsRootsPath (or the system's
store) and its name against tlsServerName (or the target's host);
PG_TLS_REQUIRE encrypts without checking who answered. A refused
certificate or failed handshake is PG_ERR_TLS with socketError the TLS
alert (positive); a refused SSLRequest has socketError 0. TLS needs a TCP
target.
tests/link/std_postgres_carrier runs one against a loopback backend,
tests/link/std_postgres_tls one over TLS; tests/postgres/carrier.ts
against a live server. Caps and findings: docs/security/postgres.md.
Written for this module; not ported from another implementation.
Classes
PgCarrier
A PgPool on an HttpHandler's loop: the pool's sockets sit in one
epoll set of the carrier's own, which the handler watches beside its
connections, so that a route handler queries PostgreSQL without blocking
its worker. Each turn the program waits on the handler, hands the requests
it takes to run (or runInt and runText, for an attached PgBatcher),
and reads next() until PG_CARRIER_NONE: every query gets its rows and
then exactly one final event (PG_CARRIER_DONE, PG_CARRIER_MISSING or
PG_CARRIER_FAILED), with request() the HttpRequest to answer and
tag the program's own number for what was asked.
The carrier connects the slots the pool wants to target, which must be a
Unix socket or a numeric address (it resolves no names), reads and writes
their bytes, fails a connection that has not authenticated within
connectTimeoutMs, and closes a failed one's socket. A connection that
fails takes the queries on it with it, each PG_CARRIER_FAILED. Every
table is sized at construction from PgCarrierLimits: run answers
PG_CARRIER_FULL rather than grow one, and once its connections are up
nothing is allocated per query. A final event's query is freed on the
following next() call, so query, tag and request() hold until then.
class PgCarrier
| Member | |
|---|---|
new PgCarrier(limits: PgCarrierLimits, pool: PgPool, target: PgTarget, handler: HttpHandler) |
A carrier for pool's connections to target, its descriptor watched on handler as limits.watch. Panics on a cap or watch id out of range, a target that failed or is not a numeric address, TLS asked for on a Unix socket, PG_TLS_VERIFY_FULL with no root certificate to read from tlsRootsPath (or the system's store), or a poll set the system will not create. It allocates every table here and opens its poll set, but connects nothing until the first next(). |
failure: i32 |
Behind a PG_CARRIER_FAILED: the connection's PG_ERR_* failure when it was lost, else PG_ERR_NONE (the server's error, or a bind with no room). |
query: i32 |
The query the last event is about, as run answered it; -1 for none. |
socketError: i32 |
Behind a PG_CARRIER_FAILED for a lost connection: the socket error that ended it (-110 for a connect timeout), or 0. |
tag: i32 |
The tag query was run with. |
attach(batcher: PgBatcher): i32 |
Registers batcher's statement on the pool and answers its id for runInt and runText. Panics past 8 batchers, and wherever PgPool.register refuses: once the pool has connected, on a NUL in the batcher's name or SQL, or past 256 statements. Attach every batcher before the first next(), which starts connecting. |
close(): void |
Tells every connection the session is over, writes what it can of that, and closes the sockets and the carrier's poll set, which closing takes off the handler's watch. Queries still in flight get no further event: their requests are the program's to answer. After it, run, runInt and runText answer PG_CARRIER_FULL and next() PG_CARRIER_NONE; a second close() does nothing. The pool is the program's to dispose(). |
connection(): PgConnection |
The connection the last PG_CARRIER_BUILD or row is on. Panics before any. |
next(): i32 |
The next event, resuming where the last call stopped: the events the pool's connections read, then the queries handed to connections (PG_CARRIER_BUILD), then PG_CARRIER_NONE once their bytes are written and nothing more can happen until the handler's next turn. The connect timeout is checked here, so the program's wait timeout bounds how late it fires. Panics while a PG_CARRIER_BUILD is unsent. PG_CARRIER_NONE once the carrier is closed, which connects nothing. The call first frees the query whose final event it answered last. |
request(): HttpRequest |
The request query was run for. Panics when it was run with none, or no event named a query. |
run(req: HttpRequest | null, tag: i32): i32 |
Queues a plain query for req (null for one of the program's own) and answers its number; PG_CARRIER_BUILD comes for it once a connection has room. PG_CARRIER_FULL when queries are waiting, the pool's queue is full or the carrier is closed. Returns at once. |
runInt(req: HttpRequest | null, batcher: i32, key: i64, tag: i32): i32 |
Queues req's int8 key on batcher batcher, as run does a plain query: the carrier sends it with the other keys waiting on that batcher in one execution, and answers its rows to this query alone. Panics on an id attach never gave, or a batcher whose key is text. |
runText(req: HttpRequest | null, batcher: i32, bytes: u8[], start: i32, length: i32, tag: i32): i32 |
Queues req's text key bytes[start, start + length) on batcher batcher, as runInt does an int8 key. The key is copied into the batcher's own store, so bytes may change once this returns. Panics on an id attach never gave, a batcher whose key is int8, or a window outside bytes. |
send(): boolean |
Ends the query PG_CARRIER_BUILD handed out, as PgPool.send does, and answers whether it went. A bind left open is dropped, and the query fails; one with nothing written fails at once, on the next next(). Panics when no build is open. |
PgCarrierLimits
The caps a PgCarrier is built with, all set before it starts: nothing
grows after.
class PgCarrierLimits
| Member | |
|---|---|
new PgCarrierLimits() |
|
connectTimeoutMs: i32 |
How long a connection has to connect and authenticate before it fails, in milliseconds (1 to 3600000). |
executions: i32 |
The most batched executions waiting or in flight at once (1 to 65536). |
queries: i32 |
The most queries waiting at once, plain and batched (1 to 65536). |
watch: i32 |
The id the carrier's descriptor is watched under on the handler (HttpHandler.watch), 0 to HANDLER_MAX_WATCHED - 1. |
Constants
1
PG_CARRIER_BUILD: i32
A plain query has its connection: bind it on connection(), then call send().
3
PG_CARRIER_COMPLETE: i32
A plain query's statement finished: connection().tagRows() reads its tag.
5
PG_CARRIER_DONE: i32
query is over and it worked: answer its request. A final event.
4
PG_CARRIER_ERROR: i32
The server refused a plain query's statement: connection().lastError says why, and PG_CARRIER_FAILED follows.
7
PG_CARRIER_FAILED: i32
query is over and it failed (failure and socketError say how). A final event.
-2
PG_CARRIER_FULL: i32
run, runInt or runText: no room for another query now.
6
PG_CARRIER_MISSING: i32
A batched query's key has no row. A final event.
0
PG_CARRIER_NONE: i32
PgCarrier.next(): nothing more this turn; wait on the handler again.
2
PG_CARRIER_ROW: i32
A row for query, to read on connection() before the next call.

