Security record: the PostgreSQL client
The record of nish/net/postgres, the sans-IO client side of PostgreSQL's
frontend/backend protocol 3.0 (wp39-http-server.md,
E6). It says what the client refuses from a server, and with what code, what a
server can make it hold or compute, where the password lives and when it is
wiped, and the test that pins each property. It is written with the code
rather than after an audit, so that the audit of the database layer starts
from evidence.
Anchors are function names in std/net/postgres.ts at the commit that adds
connection targets (pgTarget, pgTargetUrl), the last to change it, and
in std/net/postgres-carrier.ts at the commit that adds PgCarrier, which
also adds PgBatcher.shed; those in std/net/postgres-async.ts are at the
commit that pipelines the async pool's statements across tasks and bounds its
connect, the last to change it (PG-39, PG-40, PG-42).
Result. Every message is framed once, by its length, and a length under 4
or over maxMessage fails the connection before its body is read; every
field inside a message is bounded by that message's end, so no backend bytes,
however malformed, make the client read past them or panic. A message the
protocol does not allow at that point, COPY among them, fails the connection.
A server can make the client hold about 3 × maxMessage bytes (the message
in its input buffer, and the copies kept as lastError and lastNotice) and
one row of maxColumns columns, and compute at most maxIterations
SCRAM iterations; a program can queue at most maxOutput bytes unsent. Every
recorded exchange cut at every byte offset, and 4000 seeded mutants of them,
end on a defined failure code without a panic. The server's SCRAM signature is compared in constant time
before an AuthenticationOk is believed, and an AuthenticationOk that skips
it is refused.
The password is held as a Secret<u8[]> copy and wiped once
the proof is computed, or when the connection fails or is disposed, and every
SCRAM key is wiped, the hasher that makes StoredKey included. No finding is
open: PG-12 (ClientKey left in a hasher) and PG-13 (an unbounded send buffer)
were fixed in review, and PG-15 to PG-17 (a spent batch run that panicked,
a text key that failed its whole batch, and collidable keys) in the
batcher's.
Two properties a reader might assume are not provided, by
design, and are recorded as accepted: no way to require SCRAM
of a server (PG-10), and MD5 and cleartext refused (PG-11). TLS (PG-9) is
fixed in the carrier and off by default; see "TLS in the carrier". Two more are
accepted for a Unix socket target: nothing checks who owns the socket
(PG-21), and a host naming the socket file itself is taken as a directory,
as libpq takes it (PG-22).
PgPool reaches a server only through its connections: each pooled
connection is a PgConnection under the same caps, so a server can make the
pool hold at most size times what it can make one connection hold — about
3 × maxMessage bytes and one row of maxColumns columns per slot, at most
1024 slots.
The pool's own state is bounded by the program (at most 1024
connections, a queue of at most 2^20 requests, 1024 groups in flight per
connection, 256 registered statements), with a PG_POOL_FULL answer in place
of growth. A server does reach the queue in one way: a connection that fails
before it is ready, while no other is, takes the oldest waiting request with
it as PG_LOST, so a server that refuses or drops every connection drains
the queue one request per connect, which the program must be ready to see.
A connection that fails reports each request it took as PG_LOST and is made
again on its next use. The pool holds one copy of the password until
dispose, and each connection its own until its proof is computed. A fresh
nonce for each connect and the sockets' lifetimes are the program's (see
Limits). It landed with this record extended, as the rule of #893 asks of
attacker-facing code.
PgBatcher reaches a server only through the connection a program hands it,
and adds one thing a server says that it acts on: the ordinal in each row's
last column, which picks the requests the row is routed to. It refuses an
ordinal that is NULL, not an integer, outside the keys it bound, or (when
sorted) before the key the last row was for, and fails the run rather than route the row,
so a server cannot hand one request's rows to another, nor make the helper
read past its arrays (PG-14).
Its state is bounded by its config — at most queueSize requests and
queueBytes of text keys waiting, runs executions of maxRequests requests
each. A run holds two i32 arrays of maxRequests entries: at the most
maxRequests, 65,536, that is 512 KiB a run, and at the most runs, 1024,
512 MiB in all. With sorted false, a run holds a boolean array of
maxRequests more, at most 64 KiB.
The queue is allocated in its constructor; the dedup table and each run's
arrays grow, by doubling, only to the largest take the program's own add
calls queued, never past maxRequests, so no server answer makes it grow. A
key refused for room, by maxBytes or by the connection's maxOutput, is not
a panic: the take stops there, cancels the bind when nothing was bound, and the
request waits.
pgTarget and pgTargetUrl turn a host and port, or a postgres:// URL,
into the socket path or TCP host a program connects to; they open nothing.
The URL may come from a configuration file or an environment variable, so
its bytes are untrusted: every escape is checked, %00 and a bare NUL are
refused, so a decoded path cannot end early at a NUL (PG-19), the user
information ends where libpq ends it or the URL is refused (PG-20), and a socket
path too long for sun_path is refused rather than cut to a shorter path
that names another socket (PG-18). Neither panics on any input (the seeded
URLs under "Properties verified").
Scope
| File | Functions |
|---|---|
std/net/postgres.ts |
PgConfig; PgError; PgPassword; PgConnection (the constructor, received, receive, sent, closed, dispose, next, compact, fail, passwordWiped, dispatch, reserveColumns, readRow, readDescription, readError, authenticate, scramStart, scramContinue, scramFinal, the builders prepare, closeStatement, bindStart, the param* calls with paramIntArray, paramInt4Array and paramInt8Array, the array builder arrayStart, arrayElement, arrayInt, arrayNull and arrayEnd, the binary array builder binaryArrayStart, binaryArrayInt and binaryArrayNull, the parameter format codes written at bindEnd, bindEnd, bindCancel, sync, terminate, the output reservation and its compaction, and the row readers); PgText; PgArrayWalker; PgPool (the constructor, register, request, waiting, inFlight, dispatch, send, wanted, connect, connection, next, anyReady, passwordWiped, dispose) and PgPoolSlot; PgBatchConfig; PgBatcher (the constructor, waiting, addInt, addText, takeDue, take, row, end, fail, next, and the private helpers they call) and PgBatchRun; PgBatchClockConfig, PgBatchClock and PgBatchTurns, which take only the program's own times and counts, never a server's bytes; the helpers pgCopyBytes, pgBytesOf, pgDecimalLength, pgPutDecimal, pgHasNul, pgFind, pgBytesAre, pgSaltedPassword, pgSha256Wiped, pgScramKeys; the connection targets PgTarget, pgTarget and pgTargetUrl, with their helpers pgTargetFailed, pgDecoded (which calls urlDecode from std/net/url.ts, audited in url.md) and pgPortValue; PgBatcher.shed; PgConfig.tls, tlsRootsPath and tlsServerName, pgSslRequest |
std/net/postgres-async.ts |
PgAsyncPool (the constructor, live, close); PgAsyncParams; PgAsyncResult (ok, text, isNull, int, cell); pgAsyncConnect, pgAsyncStartup, pgAsyncClaim, pgAsyncRelease, pgAsyncQuery, pgAsyncPrepare, pgAsyncSeat, pgAsyncSend, pgAsyncTurn, pgAsyncAnswer, pgAsyncDone, and the helpers they call: pgAsyncForget, pgAsyncSlot, pgAsyncWaitFailure, pgAsyncWaitUntil, pgAsyncSince, pgAsyncSignal, pgAsyncDrain, pgAsyncPark, pgAsyncSeatOf, pgAsyncAhead, pgAsyncLostSeat, pgAsyncReady, pgAsyncDoom, pgAsyncFree, pgAsyncWakePool, pgAsyncFreeSeats, pgAsyncUnqueue, pgAsyncAbandon, pgAsyncSettle, pgAsyncSkippable, pgAsyncSkip, pgAsyncPlain, pgAsyncSkipAny, pgAsyncLabel, pgAsyncScan, pgAsyncLost, pgAsyncCheckCaps, pgAsyncEncode, pgAsyncBytes (which also refuses a NUL), pgAsyncRead, pgAsyncRow, pgAsyncError, pgAsyncDecimal |
std/net/postgres-carrier.ts |
PgCarrierLimits; the TLS helpers pgCarrierTrust and pgCarrierTlsStart, the SSLRequest and its one-byte answer, and the TLS record pass in front of each slot's reads and writes; PgCarrier (the constructor, attach, run, runInt, runText, request, connection, send, next, close, and the private helpers they call: the poll pass, connect and its timeout, the read and write of each slot, the routing of pool events to queries and batched executions, the ticket map, the free stacks and the shedding of a lost execution's requests) |
Out of scope: the nish:net calls themselves, which the carrier and the
async pool make and the client never does, the executor behind readable,
writable and sleep (docs/security/async.md), and HttpHandler, whose watch and fired the
carrier uses (handler.md); the TLS 1.3 client, its records and
its certificate checks, which the carrier drives and tls.md audits; the HMAC,
SHA-256, base64 and constant-time primitives the SCRAM exchange calls, which
are crypto-k1.md's; and what a program does with a column's
bytes.
Threat model
The attacker is the server, or anyone on the network between it and the
client: without TLS (and always for PgPool driven by hand and the async
pool), every backend byte is theirs, of any length, split anywhere. A win is a panic, a read or write outside the input or output
buffer, memory or time a server can make the client spend beyond the caps, a
connection that believes a server knows the password when it does not, a
password or key the client leaves in memory after it is done with it, or a
statement's parameters interpreted as SQL.
The program's own clients are attackers too, through the keys a program hands
PgBatcher: a key reaches the server only as an element of the bound array
parameter, never in SQL text, and a win for them is a panic, one client's key
failing or reading another's requests, or time a chosen key makes a take spend
beyond its caps.
So is whoever wrote the URL pgTargetUrl reads: a win for them is a panic,
or a target other than the one the URL spells — a socket path cut short, or
one a decoded byte ended early. A local user who can create the socket a
target names is out of reach of a parser, and is PG-21.
The program is trusted: its
PgConfig, its SQL and its names. A NUL in a name or in SQL is refused rather
than sent; the program's mistakes that would leave the connection with no
defined state are panics, listed under "Panic sites".
The password's bytes are used as they are, without SASLprep (RFC 4013): an ASCII password works, and a non-ASCII one only when it is already in the form SASLprep would give. That costs a connection, never a secret, and is not a finding; the guide says so.
Findings
Rows PG-1 to PG-8 are the shapes a hostile server's bytes can take, each checked and found to hold; they are recorded so that the audit can see what was tried. PG-10 and PG-11 are accepted; PG-9 was accepted and is fixed in the carrier by WP39 E6. PG-12 and PG-13 were found by the first review of this module and fixed before it merged; PG-15 to PG-17 by the batcher's first review, and fixed before it merged.
PG-18 to PG-20 are the shapes a URL can take, checked as the parser was
written; PG-21 and PG-22 are accepted. PG-23 to PG-27 are the carrier's
(std/net/postgres-carrier.ts): PG-23 and PG-24 were found as it was written
and fixed before it merged, PG-25 holds, and PG-26 and PG-27 are accepted.
PG-28 to PG-43 are the async pool's (std/net/postgres-async.ts): PG-28 to
PG-31 hold, PG-32 was found as it was written and fixed before it merged,
PG-33 and PG-34 are accepted, PG-35 and PG-36 were found by its first
review and fixed before it merged, PG-37, from the cancellation stage of
WP49, is accepted, and PG-38, found when a serve handler first ran the pool
against a live server, is fixed. PG-39 to PG-41 come with the pool's
pipelining (WP49 stage 6): PG-39 holds, PG-40, PG-42 and PG-43 are fixed and
PG-41 is accepted.
| Id | Severity | Where | Description | Disposition |
|---|---|---|---|---|
| PG-1 | High if it failed | next, received, receive, compact |
A message's length must not make the client read past its buffer or hold more than maxMessage. |
Holds. A length under 4 is PG_ERR_FRAME, one over maxMessage is PG_ERR_TOO_BIG before its body is read, and the input buffer grows by doubling to one message at most. receive panics on a window outside its source, as received does on a count past inputRoom(). A message split into 1-byte pieces frames the same as one whole. |
| PG-2 | High if it failed | dispatch, readRow, readDescription, readError |
A field inside a message must not be read past the message's end: a column longer than its row, a length of -2, a string without its NUL, a count short of its fields. |
Holds. Each is PG_ERR_FRAME; a row with bytes after its last column is too. |
| PG-3 | Medium if it failed | readRow, readDescription, reserveColumns |
A row or description must not make the client allocate without bound. | Holds. Over maxColumns (1664, PostgreSQL's own) is PG_ERR_TOO_BIG before the column arrays grow. |
| PG-4 | Medium if it failed | dispatch, authenticate |
A message at the wrong point — a row before authentication, BackendKeyData once ready, ReadyForQuery or an ErrorResponse for no group, COPY, an unknown type — must not be acted on. |
Holds. Each is PG_ERR_UNEXPECTED. A NotificationResponse is read and dropped. |
| PG-5 | High if it failed | scramContinue |
The server's SCRAM nonce must extend the client's, so an old exchange cannot be replayed into a new one. | Holds. A nonce that does not start with ours, adds nothing, or holds a byte outside !..~ is PG_ERR_SCRAM. |
| PG-6 | High if it failed | scramFinal, authenticate |
The client must not believe a server that does not know the password. | Holds. The v= signature must be 32 bytes and equal the one computed with the proof, by timingSafeEqual; a mismatch is PG_ERR_SERVER_SIGNATURE. An AuthenticationOk before the server's final message is PG_ERR_SCRAM. |
| PG-7 | Medium if it failed | scramContinue |
The iteration count must not make the client spin, nor overflow while it is read. | Holds. Outside minIterations..maxIterations (4096 to 100000) is PG_ERR_SCRAM, and the digits stop being read once the count passes maxIterations. |
| PG-8 | Medium if it failed | dispatch (ParameterStatus) |
A client_encoding other than UTF8 would make columnText decode bytes as what they are not; a DateStyle other than ISO or a TimeZone other than UTC would make PgText.timestamp read a time the server did not mean. |
Holds. The first is PG_ERR_ENCODING, the other two PG_ERR_SETTING, at startup or after a SET. |
| PG-9 | Medium | the whole module | No TLS: an eavesdropper reads every query and row, and a man in the middle can answer as the server. SCRAM keeps the password itself from them, and PG-6 keeps a server that does not know it from being believed, but a MITM can relay the exchange and then read and change what follows. | Fixed in the carrier by WP39 E6 (see "TLS in the carrier"): PgConfig.tls set to PG_TLS_VERIFY_FULL sends an SSLRequest and runs the TLS 1.3 client with chain and host-name checks, and a certificate under another root or for another name fails the connection with PG_ERR_TLS. Before: with the carrier's verdict forced to 0, the six refusal lines of tests/link/std_postgres_tls fail; after: they pass, and the live tls check sees alerts 48 and 42. The default stays PG_TLS_DISABLE. Every other path refuses a tls it cannot honour rather than run in the clear: the PgConnection constructor (and so pgAsyncConnect, which makes one) and PgPool.connect on a pool no carrier drives panic (Panic sites); before, at the change's first head, both took the config and connected in the clear. |
| PG-10 | Low | authenticate |
A server may answer AuthenticationOk at once (trust), and the client takes it: a program cannot insist on SCRAM, so an attacker that can answer as the server skips authentication. |
Accepted. Without TLS the attacker of PG-9 wins anyway. Under PG_TLS_VERIFY_FULL the network attacker is gone, and only a server holding a certificate the program trusts can skip authentication; a requireAuth setting is still not written. |
| PG-11 | Low | authenticate |
MD5 and cleartext passwords are refused with PG_ERR_AUTH_UNSUPPORTED, so a role whose password is MD5-hashed cannot connect. |
Accepted, under TLS too: the refusal does not look at the transport. Without TLS cleartext sends the password as it is, and an MD5 response is a hash anyone who sees it can replay; inside TLS either would hand the password, or a replayable hash, to any server the program trusts, where SCRAM proves it without sending it; SCRAM-SHA-256 is PostgreSQL's default since 14. nish/crypto/md5 is therefore not written. |
| PG-12 | Low | pgScramKeys, pgSha256Wiped |
sha256(clientKey), the one-shot that made StoredKey, left a hasher whose state and block had absorbed ClientKey, which is password-equivalent for that server. |
Fixed. StoredKey is computed by pgSha256Wiped, a hasher of its own whose state, block and schedule are wiped before it returns, as HmacSha256.digest does for its two. Not observable from outside (the hasher is a local); the wipes are read in the source. |
| PG-13 | Low | the builders, sent, the output reservation |
The output buffer doubled without a cap and was compacted only once fully drained, so a program that queued faster than it sent grew it without bound, toward an i32 offset's end. The program is trusted, so this is Low. |
Fixed. maxOutput (1 MiB by default) caps the bytes queued unsent: a builder call that would pass it answers false and writes nothing, the room for Sync and Terminate is kept free under it, and the sent prefix is moved off the front once more than half the buffer is gone, or before any message, bindEnd's, Sync and Terminate included, that would otherwise grow the buffer past its end; the buffer doubles only while twice its length fits in an i32. "a prepare that would pass maxOutput is refused", "a parameter that would pass maxOutput is refused", "unsent bytes count against maxOutput", "a send past half the buffer moves the rest to its start", "rather than double the buffer". |
| PG-14 | High if it failed | PgBatcher.row |
A row is routed to requests by the ordinal in its last column, which the server sends. An ordinal past the keys, of 0 or less, NULL, not an integer, or past an int8 would index outside the run's arrays or give one request another's rows; one that goes back would answer a request after its last event. | Holds. row reads the ordinal with the connection's integer decoder (int for a text batch, binaryInt for a binary one, with binary.failed saved and restored around it) and refuses each of these before it routes anything: it answers false, counts the row in refused, and fails the run, so every request still waiting on it gets PG_BATCH_FAILED. A run's rows are routed only while it is open, and take resets a reused run's keys, so a later batch cannot read an earlier one's. "an ordinal past the keys", "ordinal 0", "a negative ordinal", "a NULL ordinal", "an ordinal that is not an integer", "an ordinal past an int8", "a row of no columns", "an ordinal that goes back", "a later batch cannot read an earlier one's ordinals", "and a bad one does not set it" (the program's text.failed); for a binary batch, "a binary take answers a run", "its Bind asks for one result format, binary", "its ordinals are read as binary int8s, each row to its request", "binary.failed stays as the caller set it", "a text ordinal is refused" and "a NULL ordinal is refused" (link/std_postgres_batch). With sorted false the rows may come in any order, so an ordinal that goes back or repeats is accepted, but every other refusal holds, and no request has a last event before end, so no row can answer one after it: "an ordinal past the keys", "ordinal 0", "a negative ordinal", "a NULL ordinal", "an ordinal that is not an integer", "an ordinal past an int8", "a row of no columns", "a bad ordinal after rows fails those keys' requests too", "an ordinal that goes back is accepted", "a key's ordinal may come again", "a bad ordinal does not set the program's text.failed", "the reused run sees none of the old run's rows" (link/std_postgres_batch_unsorted). |
| PG-15 | Medium | PgBatcher.row, end, fail, next |
A run was named by its index, reused once answered, and an event on a run that was over panicked. So a server that sent CommandComplete twice, a DataRow or CommandComplete after an ErrorResponse, or a connection lost after its answer, ended the program; and had the panic been lifted, a late event would have reached the run's next batch, another set of requests. |
Fixed. take answers a handle, the run's index plus its uses times runs (the uses wrap after 2^20, so a handle stays under 2^30), and a run's next take answers a new one. A spent handle — its run answered or taken again — makes row, end and fail do nothing and next answer PG_BATCH_NONE, and counts the call in stray. Only the program's own mistakes still panic (see Panic sites). "a second CommandComplete is stray", "a row after CommandComplete is stray", "an error after CommandComplete, as a commit at Sync can fail, is stray", "a connection lost after CommandComplete, before the group is done", "a row after an error is stray", "a CommandComplete after an error is stray", each through a PgPool from the server's bytes; "the one run is taken again under a new handle", "a row on the spent handle answers false", "and next on it answers nothing", "each counted stray", "the new execution is untouched", "the run is free again, under its next handle: a use of 16 runs on". |
| PG-16 | Medium | PgBatcher.addText |
A text key that is not UTF-8, or holds a NUL, is refused by the server for the whole array, so one client's bad key failed every request batched with it. | Fixed. addText answers false for a key with a NUL or that is not RFC 3629 UTF-8 (an overlong form, a surrogate, past U+10FFFF, a stray or missing continuation byte), before it is queued. "a key with a NUL is refused", "a lone continuation byte is refused", "an overlong NUL is refused", "an overlong three-byte form is refused", "a surrogate is refused", "past U+10FFFF is refused", "a character cut by the window is refused", "a bad second continuation byte is refused", "only the good ones wait". |
| PG-17 | Low | PgBatcher.take |
Each key is bound once, found by an open-addressed table hashed with a 31-bit polynomial a client could collide, and maxRequests reached 65,536: keys chosen to collide made one take probe on the order of 2^31 slots. |
Fixed. Each request probes at most PG_BATCH_PROBES (1024) slots of the table: past that it stops looking and binds its key again, an extra element and ordinal the server answers in turn, so a take of n requests probes at most 1024·n slots whatever the hash, with maxRequests at most 65,536. Up to 1024 requests a take never reaches the bound, so it binds what it did when the cap was 1024. It compares a key's bytes only on an equal 31-bit hash, so the bytes compared are at most that bound times the key length, which maxBytes caps. FNV-1a, which replaced the polynomial, makes chance collisions rarer but does not set that bound: it is unseeded, and bwfgaa and zidhaa share a 31-bit hash. An int8 key's hash mixes its two 32-bit halves through MurmurHash3's 32-bit finalizer: a hash near the identity put two dense id ranges a power of two apart into one cluster longer than the bound, so their keys were bound again rather than found. "text keys quoted and escaped, each once", where only their bytes tell those two apart; link/std_postgres_batch_scale (1024 keys of one hash each found once; past the bound a key bound again, each copy routed by its own ordinal; two dense ranges 131072 apart and eight pairs at seeded offsets, each key bound once; 2048 int keys of one hash, each requested twice, each key past the bound bound again and every request routed its own row); tests/postgres/live batchscale (a colliding take of 4096 against PostgreSQL 18, each row checked against its request's key); link/std_postgres_batch_config_requests_panic (65537, after 65536 is taken); bench/postgres/batch-scale.mjs (time per request with every key colliding, bounded from 4096 to 65,536). |
| PG-18 | High if it failed | pgTarget |
sun_path holds 108 bytes on Linux, its NUL included; a longer path cut to fit would connect to a different socket, perhaps one another user made. |
Holds. A / path over 107 bytes, or an @ name over 107 after its @, is PG_TARGET_TOO_LONG, with path empty and unix false. macOS's 104-byte sun_path is left to unixConnect, which answers -36 there rather than cut. "a 107-byte socket path is taken", "a 108-byte socket path is refused", "not cut short", "nor unix", "an @ name of 107 bytes after the @ is taken", "one of 108 is refused", "a longer port counts", "multi-byte UTF-8 counts in bytes: 107 is taken", "and 108 refused"; through a URL, "a URL's @ name of 107 bytes after the @ is taken", its "one of 108 is refused", and "a long encoded directory is refused" (link/std_postgres_unix_target). |
| PG-19 | High if it failed | pgDecoded, pgTargetUrl, pgTarget |
A percent escape must decode to the byte it spells and nothing else: a % at the end, with one digit or a non-hex one would read past the text or decode garbage, and %00, or a NUL byte as given, would end the path early in the C call. |
Holds. Each is refused, PG_TARGET_BAD_ESCAPE for an escape and PG_TARGET_BAD_HOST for a raw NUL byte, in every part of the URL, including those pgTargetUrl does not otherwise read: pgTargetUrl scans the whole URL for a raw NUL before it splits it, and pgTarget refuses one in a host. Decoding is urlDecode's, which also refuses escapes that decode to bytes that are not UTF-8. "an escape that is not UTF-8 is refused", "escaped UTF-8 decodes"; "a bare % ends the host", "a % with one digit", "%G1 is no hex", "%1G is no hex", "%00 is refused", "in the user too", "in the database too", "in the port", "in a query key", "in a query value", "in an ignored value"; for a raw NUL, "a raw NUL in the user is refused" and its password, host, port, database, query key, query value and ignored value lines, and "a NUL in the host is refused" (pgTarget); "lower-case escapes" and "an encoded port" decode, and "%252F stays the TCP host %2F", decoded once. |
| PG-20 | Medium if it failed | pgTargetUrl, pgPortValue |
The host is the authority after the user information and before its port, or a query host; a password with an @, an encoded , (libpq's list of hosts) or a port past 65535 must not become a different host or wrap to another port. |
Fixed in review: the user information first ended at the authority's last @, where libpq ends it at the first, so postgres://u:p@ss@db/app named db here and ss@db to libpq. Now a raw @ after the first in the authority, or a first @ after a ? (libpq reads that ? as the user's), is PG_TARGET_BAD_HOST rather than read either way; an @ in a password is written %40. A , is PG_TARGET_BAD_HOST, since one target names one host; a port is 1 to 5 digits naming 1 to 65535, else PG_TARGET_BAD_PORT, so it cannot overflow. "a second @ is refused, not read as the host", "an encoded @ in the password is the user's", "an @ after a ? is refused", "an @ in the path is the database's", "an encoded , is two hosts", "two hosts are refused", "an empty port is refused", "a port of letters is refused", "a six-digit port is refused", "port 65536 is refused", "URL port 65536 is refused", "?port=0 is refused", "?host= overrides the authority, ?port= the port". An empty port, ?port= or ?host= is refused, where libpq takes it as unset: "an empty port is refused", "an empty ?port= is refused", "an empty ?host= is refused". |
| PG-21 | Low | PgConnection, over a socket |
A local user who can create the socket a target names, in a directory others may write such as /tmp, can answer as the server; under peer or trust the server asks for no password, so nothing proves it is the server. |
Accepted. As PG-10: there is no requirepeer to check the socket's owner, which nish:net cannot report. Put the socket in a directory only the server's user may write, as /var/run/postgresql is; the guide says peer and trust send no password. |
| PG-22 | Low | pgTarget |
A host that is a path to the socket file itself, not its directory, is taken as a directory, as libpq takes it, and names <file>/.s.PGSQL.<port>, which does not exist. |
Accepted. The connect fails (ENOTDIR); it reaches no other socket. Kept for libpq's meaning of the same string; the guide says so. "a full socket path is taken as a directory, as libpq does", "a URL's socket file is a directory too". |
| PG-23 | Medium | PgCarrier.next, its connect pass |
A server, or anything at its address, that accepts the connection and never sends a byte holds every query queued on that slot for ever; a route's clients wait on requests that are never answered. | Fixed as written: each slot's connect has a deadline, connectTimeoutMs (default 5000, at most an hour) from the connect, checked on every next() until the connection is ready; past it the connection fails with socket error -110 (ETIMEDOUT), its socket is closed and each query on it is PG_CARRIER_FAILED with failure 9 (PG_ERR_CLOSED). The program's wait timeout bounds how late the check runs. A refused connect fails at once rather than at the deadline. A connect that fails while no connection is ready takes the oldest waiting query with it (PgPool.next), so the queries waiting behind an unreachable server fail one per slot per turn, a turn being bounded by the program's wait timeout; a program that waits with no timeout holds them until the handler next wakes. A deadline found past on a connection the server already failed keeps the server's failure. "a server that never answers fails the query", "at the connect timeout, as ETIMEDOUT", "not before it", "a refused connect fails the query", "without waiting for the timeout" (link/std_postgres_carrier); live, the refused check. |
| PG-24 | Medium | PgCarrier routing, PgBatcher.shed |
A batched execution whose connection is lost before it is taken leaves its requests in the batcher, where they wait on a server the carrier cannot reach and their clients are never answered. | Fixed as written: the carrier fails one execution's worth (maxRequests) of the batcher's oldest waiting requests, each PG_CARRIER_FAILED with the connection's failure, through PgBatcher.shed, which drops the oldest and answers its tag; the next request queues a new execution on a new connection. "a batch whose connection ends fails its request", "as lost with the connection", "and the next batch", "after two connections ended, with a new one accepted", "a lost execution of three keys fails all three requests", "the three were one execution, answered by none". |
| PG-25 | Medium if it failed | PgCarrier.next, its read of a slot |
A socket that reports an error or hang-up stays readable; a carrier that read nothing from it and went on would be woken by it on every turn, spinning a worker. | Holds. A read that answers an error, or 0 bytes, ends the connection there: the error is kept as socketError, the socket closed and removed from the poll set, and the queries on it fail. "a query whose connection ends is the program's 503", "with the connection closed behind it" (link/std_postgres_carrier); live, the reset check (ECONNRESET, -104). |
| PG-26 | Low | PgCarrier |
Once a connection is up there is no deadline on a query: a server that authenticates and then never answers a statement holds that query, and the ones behind it on its connection, until the connection ends. | Accepted. A statement's time is the server's to bound (statement_timeout); the carrier holds request() so the program can answer with its own deadline, and a backend ended by the server fails its queries as lost. |
| PG-27 | Low | PgCarrier.next, its connect pass |
While the server is down, each next() pass makes one connect attempt for each slot the pool wants, with no backoff, and each attempt makes a new PgConnection and its buffers, which the arena keeps. |
Accepted. The attempts are bounded by the queries waiting, not by time: a slot is wanted only while more queries wait than the connections starting can take, and a connect that fails while no connection is ready fails the oldest waiting query with it (PG-23), so the attempts end when the queue is empty, and there is at most one for each slot in each pass, which the program's wait turns pace. The memory is a connection's for each query that failed for want of a server, which the Limits below record. A backoff would hold the clients of the queries waiting behind a server already down for longer; a program that would rather not pay it can answer 503 itself after failures instead of queueing more. "a refused connect fails the query", "without waiting for the timeout" (link/std_postgres_carrier). |
| PG-28 | High if it failed | pgAsyncRead, pgAsyncRow, pgAsyncError |
The async pool reads a server's answer with a framer of its own, not PgConnection's, so PG-1 to PG-3 must hold for it too: a length under 4 or past maxMessage, a field past its message's end, a row past maxColumns or rows without end. |
Holds. A length under 4, and a DataRow or ErrorResponse whose fields do not add up to its length, are PG_ERR_FRAME; a length past maxMessage is PG_ERR_TOO_BIG before its body is read, and the read buffer grows by doubling to maxMessage + 1 at most; a row past maxColumns is PG_ERR_TOO_BIG before a cell is kept; rows past maxRows are read and dropped, and the query fails PG_ERR_TOO_BIG at its ReadyForQuery, its connection still in step. A message type no extended query answers (a W, COPY among them) is PG_ERR_UNEXPECTED. A cell length past its row's end, a negative length other than -1 (NULL), a row whose cells end before its length, an ErrorResponse field with no NUL or no closing 0, and a row whose column count differs from the first row's are each PG_ERR_FRAME; a socket that ends mid-message is PG_ERR_CLOSED. Each failure but the caps on rows and on a result's bytes closes the connection. A message split across reads is reassembled, and the buffer is compacted and grown for a cell longer than its first read. "short: ok=false failure=1", "big: ok=false failure=2", "wide: ok=false failure=2", "odd: ok=false failure=3", "many: ok=false failure=2 socket=0 rows=3 tag=SELECT 5 reusable=true", "cellpast: ok=false failure=1", "negative: ok=false failure=1", "slack: ok=false failure=1", "unended: ok=false failure=1", "noend: ok=false failure=1", "reshape: ok=false failure=1 socket=0 rows=1", "midframe: ok=false failure=9", "large: ok=true failure=0 socket=0 rows=2 tag=SELECT 2 reusable=true first=s", "large cell: 20000 bytes", "split: ok=true failure=0 socket=0 rows=1 tag=SELECT 1 reusable=true first=p" (link/std_postgres_async). Not tested: a wait that answers false (pgAsyncWaitFailure), which needs a descriptor closed under the wait, and a write that answers EAGAIN, which is reachable (a parameter of several MiB to a listener that does not read) but has no test yet; both are read in the source, and each fails the query PG_ERR_CLOSED and closes the connection. split divides a message only inside its 5-byte header; large reassembles a DataRow body across reads. |
| PG-29 | Medium if it failed | pgAsyncRead, pgAsyncRelease |
A connection one task gives back must not carry its session to the next: an open transaction would put the next task's statements inside it, and a SET its settings (a client_encoding would change how rows decode, PG-8). |
Holds. A query is reusable only when it read to a ReadyForQuery of I (idle) with no ParameterStatus on the way; a socket error, a frame failure or a cap closes it too. pgAsyncRelease closes a connection whose query is not, and counts it in dropped. "begin: ok=true failure=0 socket=0 rows=0 tag=BEGIN reusable=false", "live after begin: 1", "set: ok=true failure=0 socket=0 rows=0 tag=SET reusable=false", "live after set: 0" (link/std_postgres_async). |
| PG-30 | High if it failed | pgAsyncClaim, pgAsyncSlot, pgAsyncQuery, pgAsyncRelease |
Two tasks on one connection would interleave their messages and read each other's rows; a connection given back twice would be claimed by two tasks at once. | Holds. A claim marks its slot and answers a ticket of the slot and its claim count; pgAsyncQuery and pgAsyncRelease take only a ticket that is the slot's current claim, a release spends it, and close() spends every one, so a second release, or one after close(), panics rather than give back a connection another task holds. The count wraps after 2^24 claims of one slot. Three tasks on a pool of two each get their own row, the third after one is given back: "B: ... first=q", "C: ... first=q", "A: ... first=held" (link/std_postgres_async); link/std_postgres_async_release_panic (a ticket never claimed), link/std_postgres_async_double_release_panic ("released 64 once, live 1", then the panic), link/std_postgres_async_release_closed_panic ("claimed 64, closed, live 0", then the panic). |
| PG-31 | Medium if it failed | pgAsyncClaim, pgAsyncSeat, pgAsyncPark |
A task that waits for a connection or a seat must not spin the executor while every one is held, nor wait without end. | Holds. It waits on the pool's wake, which pgAsyncRelease and pgAsyncDone ring, and answers PG_CARRIER_FULL (-2) once claimWaitMs (5000 by default) has passed; with no slot connected it answers -1 (PG_POOL_NONE) at once. Before this change it slept 1 ms, then twice as long up to 16 ms, so a connection given back was found up to 16 ms late ("-2 after 6 sleeps"). "a claim slept while both were held: true", "a claim with every connection held: -2 after 1 wait", "claim with none live: -1" (link/std_postgres_async); "waits for the held seat: 1", "a seat with the one seat held: -2 after 1 wait" (link/std_postgres_async_pipeline). |
| PG-32 | Low | pgAsyncEncode |
A NUL in a text parameter was sent, though the SQL was checked for one and pgAsyncQuery's documentation promised PG_ERR_ENCODING for both; the server refuses such a value, so it cost a round trip and a server error, not an injection. |
Fixed as written: each text parameter is checked before anything is written, the query fails PG_ERR_ENCODING and the connection stays reusable. Before, "nul: ok=true failure=0 socket=0 rows=1 tag=SELECT 1 reusable=true first=a b" (the stand-in echoed it); after, "nul: ok=false failure=8 socket=0 rows=0 tag= reusable=true" (link/std_postgres_async). |
| PG-33 | Low | pgAsyncQuery, pgAsyncTurn, pgAsyncAnswer |
A query's wait has no deadline: a server that accepts and never answers holds a query's task, and every task seated behind it on that connection, until the connection ends. The connect's wait is bounded since PG-40. | Accepted. As PG-26: a statement's time is the server's to bound (statement_timeout), and a task waiting on its socket costs the executor nothing. Since the cancellation stage of WP49 a program bounds the wait from its side: cancelAfter(taskId(), ms) in the task, or cancel from another, fails the wait, and pgAsyncWaitFailure answers -3 (cancelled) for it; a claim cancelled while it waits answers -3 too (link/std_postgres_async_cancel, PG-37). std/net/serve's deadline does this for a handler. |
| PG-37 | Low | pgAsyncQuery, pgAsyncWaitFailure, pgAsyncClaim |
A task cancelled while its query is on the wire must not give its connection back in step, since the answer would be read by the next task; and the server is not told the query was abandoned. | Accepted. A query whose wait is cancelled fails PG_ERR_CLOSED with socketError -3 and reusable false, so its connection is closed and the pool dials another; one cancelled before it sends anything fails the same way with reusable true; a claim cancelled while it waits answers -3 and holds nothing. No CancelRequest is sent: the backend runs the statement on until it next writes to the closed socket, so bound a statement with statement_timeout, as for PG-33. "hold: ok=false failure=9 socket=-3 reusable=false cell=- cancelled=true", "dropped: 1", "greedy: first true, second -3, cancelled true", "quick: claim -3, cancelled true", and a query after each runs on a fresh connection: "quick: ok=true failure=0 socket=0 reusable=true cell=q cancelled=false" (link/std_postgres_async_cancel). |
| PG-34 | Low | pgAsyncQuery |
A query awaited outside a spawned task, through a function that is handed the pool, keeps the buffers, rows and strings it built until an enclosing scope ends: about 9 KB a query in main, so a loop of queries in main grows the arena without bound. |
Accepted. The compiler gives such a function no scope of its own; a spawned task's region is freed when the task ends, so the module and the guide say to run each query in a task, which link/std_postgres_async_flat does: "Arena.used() flat over 2000 queries: true". |
| PG-35 | Medium | pgAsyncRead |
A ReadyForQuery is the end of a query's answer, so a connection was given back as reusable however many bytes followed it in the same read; those bytes were dropped, and the next task's answer would be read out of step, or a message the server sent after it read as the next query's. | Fixed in review: a query is reusable only when its ReadyForQuery is the last byte read, so a connection with bytes after it is closed and counted in dropped. Before, "trail: ok=true failure=0 socket=0 rows=1 tag=SELECT 1 reusable=true first=t"; after, "trail: ok=true failure=0 socket=0 rows=1 tag=SELECT 1 reusable=false first=t", and the next query on the pool runs on a fresh connection: "quick: ok=true ... reusable=true first=q" (link/std_postgres_async). The check sees only bytes that arrived in the same read as the ReadyForQuery: a byte the server sends in a later segment stays in the socket until the next query on that connection reads it first. A whole notice or notification there is skipped; a partial message or a type no query expects fails that query (PG_ERR_FRAME or PG_ERR_UNEXPECTED) and closes the connection; but a well-formed answer sent late, a whole C or Z among them, is read as part of the next query's answer. Recorded open: only a server that already speaks out of turn can do it, and it can then answer any query wrongly anyway. |
| PG-36 | Medium | pgAsyncRead, pgAsyncQuery, pgAsyncCheckCaps |
A result was bounded only by maxRows × maxColumns × maxMessage, about 2^60 bytes at the defaults, so a server could make one query keep the task's memory without bound; and a cap set out of range (a maxColumns of 0, a negative maxMessage) went unchecked into the framer's arithmetic. |
Fixed in review: maxResultBytes (16 MiB by default) bounds a result's kept bytes, counting each row's bytes and 16 for each cell, and each CommandComplete's and ErrorResponse's bytes, since each keeps a fresh string; rows, tags and errors past it are read and dropped as rows past maxRows are, the query failing PG_ERR_TOO_BIG with its connection still in step. (A first fix counted only rows, so a server could send C or E messages without end; the second review found it.) pgAsyncQuery first checks every cap is in range (maxMessage and maxResultBytes 1024 to 2^30-1, maxColumns 1 to 65535, maxRows 1 to 2^24) and panics if not, so row * columns cannot overflow: the cells kept number fewer than maxResultBytes / 16. "fat: ok=false failure=2 socket=0 rows=3 tag=SELECT 5 reusable=true" (five rows of 300 bytes under a maxResultBytes of 1024; link/std_postgres_async); "manyc: ok=false failure=2 socket=0 rows=0 tag=UPDATE 1 reusable=true" (150 tags of 9 bytes) and "manye: ok=false failure=2 socket=0 rows=0 tag= reusable=true" (100 errors of 14 bytes; uncounted, it would fail 0 as a refused query); link/std_postgres_async_caps_panic ("claimed 64, maxMessage 1023", then the panic: the ticket is claimed on a pool with a connection, so without the check the query would fail on the socket and print "not reached"). |
| PG-38 | Low | pgAsyncStartup, pgAsyncConnect |
A connect must not wait on a server that is itself waiting for the client: against a server that asks for SCRAM-SHA-256, the startup queued the SASLInitialResponse (and, a step later, the SASLResponse) and then read before writing it, so client and server each waited for the other and pgAsyncConnect never returned. A denial of service on the client's own pool, not a disclosure: nothing was sent that should not be. The stand-in backend of link/std_postgres_async logs in by trust, so only a live server showed it. |
Fixed in the WP49 stage 5, part 8 change: a startup step that queued an answer writes it before it reads, so the exchange runs in the order the protocol sets. Before, tests/postgres/live servepool printed no "listening" line in 30 s, its connect still waiting on the SCRAM exchange; after, "listening\nserve 0: answered 3, served 1, refused 1" with the answers "200 42", "400 refused: 22P02" and "404" (tests/postgres/live servepool, against PostgreSQL 18 with scram-sha-256 authentication). |
| PG-39 | High if it failed | pgAsyncSeat, pgAsyncSend, pgAsyncTurn, pgAsyncAnswer, pgAsyncDone, pgAsyncScan, pgAsyncDoom |
Statements from many tasks share a connection: each answer must reach the task that sent its statement, in order, however the batch fails, and a connection whose order is lost must not be read on. | Holds. A seat fixes a statement's place in its connection's order and no other seat is taken until it is sent; answers are read in that order. A statement the server refuses fails alone; one with a NUL is never sent and hands its place back; a statement whose task is cancelled before its answer is read fails alone, and the next task reads past its answer (PG-43); a connection lost, sent a frame past a cap, or left out of step by an abandoned answer that was not a plain one fails every statement in flight on it with PG_ERR_CLOSED and is closed. "sent before the gate was answered: 6" with "echo 0 … first=e0" to "echo 4 … first=e4"; "badparse first: … error=42601", "badparse behind: … error=26000", "echo nul: ok=false failure=8", "echo behind: ok=true … first=z", "live: 1, dropped: 0"; "cancelled: ok=false failure=9 socket=-3", "behind the cancelled: ok=true … first=q", "live: 1, dropped: 0"; "begin: ok=false failure=9 socket=-3", "behind the begin: ok=false failure=9 socket=0 … reusable=false", "live: 0, dropped: 1"; "die" and "behind the die" both failure=9; "big: ok=false failure=2", "behind the big: ok=false failure=9" (link/std_postgres_async_pipeline). |
| PG-40 | Low | pgAsyncConnect, pgAsyncStartup, pgAsyncWaitUntil |
A server that accepts a connection and never answers its startup held pgAsyncConnect, and so its caller, for good: a program that redials the pool while it serves stopped redialling. |
Fixed in the WP49 stage 6, part 5 change: each connection has connectWaitMs (10000 by default) to connect and finish its startup, and is closed past it, its slot left empty. Before, pgAsyncStartup awaited the socket with no deadline (read in the parent commit; the test needs connectWaitMs, so it cannot run there), and pgAsyncConnect against a listener that never answers did not return; after, "a server that never answers its startup: 0 live, 0 made" in 50 ms (link/std_postgres_async). |
| PG-41 | Low | bench/techempower/async-db-main.ts, tfbKeepPool |
The async database entry keeps the role's password, as a Secret, for the life of the process, so that it can dial again a connection the pool closed. |
Accepted. A pool that redials needs the password; it stays in one Secret, is passed to pgAsyncConnect by reference, and main wipes it once serving ends, or at once when the first connect fails. A program that would rather not hold it connects once and does not call tfbKeepPool. Read in the source; link/tfb_async_db runs the redial ("tfbKeepPool dials it again"), not the wipe. |
| PG-42 | Medium | pgAsyncSeat, pgAsyncSignal, pgAsyncPark, pgAsyncDrain |
Under load the pool turned tasks away that a free seat could have served. A task done with a seat could take the next one before the tasks waiting were woken, so a waiter could lose every race until claimWaitMs ran out; and the pool's wake was one byte that the first task woken read back, while the runtime reports at most 64 ready sockets a turn, so with more than 64 tasks waiting the rest never saw it and waited out claimWaitMs. Any client able to hold a few hundred requests open could have every one answered 503 after 5 s (p99 about 5 s at 256 connections in the TechEmpower /db test). |
Fixed in the WP49 stage 6, part 5 change: tasks waiting for a seat queue in the order they began to wait, up to 16384, and a task is seated only in its turn; each waits on one of 128 lane wakes its queue number picks, which rings when it reaches the queue's head; and a wake's byte is read back only once every task waiting when it was written has woken. Before, "holder again: … first=q" was answered before "parked: … first=q", and with the wake fix alone taken out, "a crowd of 150 behind the held seat: 1 answered, 149 failed"; after, "parked" is answered before "holder again", and "a crowd of 150 behind the held seat: 150 answered, 0 failed" (link/std_postgres_async_pipeline). |
| PG-43 | Medium | pgAsyncTurn, pgAsyncAbandon, pgAsyncSkip, pgAsyncPlain, pgAsyncDone, pgAsyncDoom |
A pipelined statement whose task was cancelled before its answer was read (a client that hung up, a request deadline) left the connection out of step, so the pool closed it and failed every statement seated behind it. Under load, where a serve handler's deadline cancels tasks often, this closed connections faster than they were dialled: the pool fell to no live connection and turned requests away with PG_POOL_NONE. |
Fixed in the WP49 stage 6, part 5 change: the cancelled task fails alone (socketError -3) and leaves its seat abandoned. The task behind it, or the next task to ask for a seat when none is behind it, reads the abandoned answer and drops it, and the connection stays, but only when that answer is a plain one: frames this module expects, ending in a ReadyForQuery that says "idle", with no ParameterStatus. An abandoned answer that left a transaction open, changed a setting, or overflows the input buffer still fails the statements behind it and closes the connection, as before. Before: "behind the cancelled: ok=false failure=9 socket=0 rows=0 reusable=false", "live: 0, dropped: 1". After: "behind the cancelled: ok=true failure=0 socket=0 rows=1 reusable=true first=q", "live: 1, dropped: 0"; a task cancelled while it waits its turn, "behind the waiter: ok=true … first=q"; with depth 1 and nobody behind, "after the lone: ok=true … first=q"; and an abandoned BEGIN, "behind the begin: ok=false failure=9 socket=0 … reusable=false", "live: 0, dropped: 1" (link/std_postgres_async_pipeline). An abandoned answer is refused two ways: pgAsyncPlain refuses a frame type it does not expect, and pgAsyncSkip refuses an answer larger than the input buffer, which never lies whole in it. The first is tested: an abandoned SET, whose answer carries a ParameterStatus, gives "behind the set: ok=false failure=9 … reusable=false", "live: 0, dropped: 4", and with ParameterStatus let through it gave "behind the set: ok=true … first=q", "live: 1, dropped: 3" (link/std_postgres_async_pipeline). The second is read in the source and untested: the stand-in backend cannot send it in an answer a cancelled task leaves. A frame length under 4 is not one of these defences: pgAsyncScan refuses it, for every answer, before pgAsyncPlain could see it, so pgAsyncPlain's own check of it is unreachable, kept so the walk does not rest on the scan. |
| PG-44 | Low | pgAsyncTurn, pgAsyncPark, pgAsyncDrain |
Found by the review of the PG-43 change. Of the tasks a wake's byte woke, the last to wake reads it back. In pgAsyncTurn, only a task that left its loop read it, so a task that woke, found its turn not yet come and waited again could wait on a byte nobody would read: it would wake at once, every time, spinning the executor until the answer ahead of it arrived. |
Fixed in the WP49 stage 6, part 5 change: pgAsyncTurn reads the byte back after each wait once every task it was written for has woken, as the claim and seat loops already did. Reading the byte there first lost wakes: a wake rung again while its byte was unread wrote nothing, so a task that had waited again after the first ring could see the byte read back before the runtime polled it, and pgAsyncTurn waits without a limit. Under the TechEmpower load at 64 connections /db fell to 34 req/s with every request timed out. pgAsyncSignal now counts again, at every ring, the tasks the byte must stay for, so it is read back only once each task waiting at the last ring has woken; /db at 64 connections went back to 31220 req/s. Section 12 of link/std_postgres_async_pipeline reproduces both: a seat whose turn has come but whose answer is held, with three seats behind it that wait before it and three that wait after, so it is neither the first nor the last woken. After, "waits while the turn's answer was held: at most 64" (turnWaits over the seven). With the drain after each wait taken out, the seats behind it spun: "waits while the turn's answer was held: 36587" (28462 to 36587 over four runs). With the recount at every ring taken out, a wake was lost: "4 of 8 still waiting after 3 s". The previous bound, "waits while the gate's answer was held: at most 3", with two seats behind a held answer, passes without either fix: under the executor's order the task whose turn came wakes last and reads the byte. |
keys, the 64 bytes pgScramKeys answers, the proof taken from it and the
server signature kept for the final message are none of them secret (the
proof is sent on the wire), but each is wiped once used, and on fail(), so
that nothing of the exchange outlives it. The wipe of these three is read in
the source; no test observes it.
Panic sites
A panic is for the program's own mistakes, never for a server's bytes:
| Where | When | Test |
|---|---|---|
| constructor | a nonce shorter than 18 bytes | link/std_postgres_nonce_panic (17 bytes, after 18 are taken) |
| constructor | an empty user |
link/std_postgres_user_panic (after a one-letter user is taken) |
| constructor | a startup parameter with a NUL | link/std_postgres_nul_panic (a NUL in database, after one without is taken) |
| constructor | a cap out of range: maxMessage or maxOutput under 1024 or over 2^30 − 1, maxColumns outside 1..65535, maxInFlight under 1, minIterations under 1 or over maxIterations, maxIterations over 214,748,363 |
link/std_postgres_config_panic covers maxOutput of 1023, after 1024 is taken; the other bounds share its condition and are not tested one by one |
received |
a count past inputRoom() |
— |
receive |
a window outside its source | link/std_postgres_receive_panic |
sent |
a count past the bytes queued | — |
pgPutDecimal |
a window outside the output buffer ("a decimal outside its room"); its one caller, putDecimal, reserves pgDecimalLength(value) bytes first, so no input reaches it. That reservation was a flat 20 bytes before; the buffer may now grow one call later, and paramInt's maxOutput check still counts 20, and the bytes sent are unchanged |
link/std_postgres_decimal, which ends a number at the buffer's last byte and never reaches it |
pgFind |
a range outside the buffer; every caller passes one inside a framed message, so no input reaches it | the fuzz below, which never reaches it |
pgScramKeys, scramFinal |
an HMAC shorter than 32 bytes, keys not 64 bytes; hmacSha256 always answers 32 |
— |
PgPool constructor |
a size outside 1..1024 ("a PgPool size out of range") | link/std_postgres_pool_size_panic (1025 slots, after 1024 are taken) |
PgPool constructor |
a queue outside 1..2^20 ("a PgPool queueSize out of range") | link/std_postgres_pool_queue_panic (a queue of 0, after 2^20 is taken) |
PgPool constructor |
maxInFlight over 1024, which sizes each slot's ring |
link/std_postgres_pool_inflight_panic (1025, after 1024 is taken) |
PgPool.connect |
a slot that wants no connection, or one outside 0..size − 1 | link/std_postgres_pool_connect_panic (the slot just connected, after the wanted one is taken) and link/std_postgres_pool_connect_range_panic (slot 2 of 2) |
PgPool.connect |
after dispose |
link/std_postgres_pool_connect_dispose_panic (a wanted slot, after dispose) |
PgPool.connection |
a slot outside 0..size − 1 or with no connection made | link/std_postgres_pool_connection_panic (slot 1, after slot 0's is read) and link/std_postgres_pool_connection_range_panic (slot 2 of 2) |
PgPool.next |
a slot outside 0..size − 1, by connection, which it calls first |
link/std_postgres_pool_next_panic (slot 2 of 2, after slot 0 answers) |
arrayElement, arrayNull |
no array open ("an array element with no arrayStart()") | link/std_postgres_array_element_panic (arrayNull, after a parameter is taken with no array) |
arrayEnd |
no array open ("arrayEnd() with no arrayStart()") | link/std_postgres_array_end_panic (a second arrayEnd, after the first) |
arrayStart |
an array already open ("arrayStart() while an array is open") | link/std_postgres_array_start_panic (a second arrayStart, after the first opens) |
param* |
an array open ("a parameter while an array is open"), which would write a parameter inside it | link/std_postgres_array_param_panic (paramInt inside an open array) |
bindEnd |
an array open ("bindEnd() while an array is open") | link/std_postgres_array_bindend_panic (after the array opens) |
arrayElement |
a window outside its source ("arrayElement() outside its source") | link/std_postgres_array_window_panic (one byte past the end, after an in-bounds window) |
paramIntArray |
a window outside its array ("paramIntArray() outside its array") | link/std_postgres_int_array_range_panic (one value past the end, after an in-bounds window) |
arrayInt |
no array open ("an array element with no arrayStart()") | link/std_postgres_array_int_panic (after a parameter is taken with no array) |
paramInt4Array, paramInt8Array |
a window outside its array ("an int array parameter outside its array") | link/std_postgres_binary_window_panic (an int8[] window one value past the end, after a text parameter is taken); link/std_postgres_binary_window_start_panic (an int4[] window starting at −1); link/std_postgres_binary_window_count_panic (an int8[] count of −1) |
paramInt4Array |
an element outside int4 ("paramInt4Array() with an element outside int4"), checked before a byte is written | link/std_postgres_binary_int4_range_panic (2^31, after a text parameter is taken) |
binaryArrayStart |
an element OID other than 23 or 20 ("binaryArrayStart() with an element OID other than int4 (23) or int8 (20)") | link/std_postgres_binary_array_oid_panic (OID 21, int2) |
binaryArrayStart |
an array already open ("binaryArrayStart() while an array is open") | link/std_postgres_binary_array_start_panic (a second start, after the first opens) |
binaryArrayInt |
an element outside int4 in an int4[] ("binaryArrayInt() with an element outside int4") |
link/std_postgres_binary_array_int_panic (−2^31 − 1) |
binaryArrayInt, binaryArrayNull |
no binary array open ("a binary array element with no binaryArrayStart()") | link/std_postgres_binary_array_element_panic (binaryArrayInt in a text array); link/std_postgres_binary_array_int_none_panic and link/std_postgres_binary_array_null_none_panic (each with no array open at all) |
arrayElement, arrayInt |
a binary array open ("a text array element in a binary array") | link/std_postgres_binary_array_text_element_panic (arrayInt in an int8[]) |
paramBytes, paramNull, paramTextArray, paramIntArray |
an array open ("a parameter while an array is open") | link/std_postgres_array_param_bytes_panic, _null_panic, _text_array_panic, _int_array_panic (each inside an open array, after the array opens) |
PgBatcher constructor |
a keyType other than PG_BATCH_INT or PG_BATCH_TEXT ("a PgBatchConfig keyType out of range") |
link/std_postgres_batch_config_key_panic (after a text batcher is made) |
PgBatcher constructor |
an empty name or select, or a NUL in them or in orderBy ("a PgBatchConfig name or query empty or with a NUL") |
link/std_postgres_batch_config_query_panic (after an ordered batcher is made); link/std_postgres_batch_config_empty_panic (an empty select); link/std_postgres_batch_config_name_empty_panic (an empty name); link/std_postgres_batch_config_name_nul_panic and link/std_postgres_batch_config_select_nul_panic (a NUL in each) |
PgBatcher constructor |
a cap out of range: maxRequests outside 1..65536, maxBytes under 22 (the widest int8 key in its braces) or over 2^30 − 1, queueSize outside 1..2^20, queueBytes outside 0..2^30 − 1, runs outside 1..1024 ("a PgBatchConfig cap out of range") |
link/std_postgres_batch_config_cap_panic covers maxBytes of 21, after 22 is taken, and link/std_postgres_batch_config_requests_panic maxRequests of 65537, after 65536; link/std_postgres_batch_config_queue_size_panic queueSize of 0, after 1; link/std_postgres_batch_config_queue_bytes_panic queueBytes of −1, after 0; link/std_postgres_batch_config_runs_panic runs of 1025, after 1024; the other bounds share their condition |
PgBatcher.addInt |
a text batcher | link/std_postgres_batch_add_int_panic (after addText on it is taken) |
PgBatcher.addText |
an int8 batcher | link/std_postgres_batch_add_text_panic (after addInt on it is taken) |
PgBatcher.addText |
a window outside its source | link/std_postgres_batch_add_text_window_panic (one byte past the end, after the whole source) |
PgBatcher.row, end, fail, next |
a handle no take could answer: negative, or runs · 2^20 or more ("a PgBatcher run out of range"); a spent one is not a panic (PG-15) |
link/std_postgres_batch_run_range_panic (next of 2 · 2^20 with 2 runs, after the last handle); link/std_postgres_batch_run_negative_panic (fail(-1), after a never-taken handle is counted stray) |
PgBatcher.row, end, fail |
events still owed ("before next(run) answered PG_BATCH_NONE") | link/std_postgres_batch_pending_panic (a second fail while the second request's event is owed); link/std_postgres_batch_row_pending_panic (a row after a fail and one next) |
PgBatcher.takeDue |
a clock whose waiting count is not the batcher's, or whose maxCount is below maxRequests ("PgBatcher.takeDue with a clock that does not mirror the batcher") |
link/std_postgres_batch_clock_take_due_panic (a request added without arrived, after a mirrored take) |
PgBatchClock constructor |
a setting out of range: maxCount outside 1..2^20, minCount outside 1..maxCount, factor outside 0..64 or NaN, weight outside (0, 1] or NaN, maxWaitUs or lightWindowUs outside 0..60,000,000, lightCount outside 0..1024, queueSize outside 1..2^20 ("a PgBatchClockConfig setting out of range") |
link/std_postgres_batch_clock_config_panic (factor 64.5, after 32 is taken); the other bounds share its condition |
PgBatchClock.started |
a size under 1, over the items waiting, or over maxCount ("PgBatchClock.started with a size out of range") |
link/std_postgres_batch_clock_started_panic (a second start with none waiting, after one) |
PgBatchClock.startsFor |
a reason outside PG_CLOCK_WAIT..PG_CLOCK_IDLE |
— |
PgBatchTurns constructor |
a size outside 1..1024 | — |
PgBatchTurns.started, release |
a slot out of range or holding a batch ("PgBatchTurns slot out of range or holding a batch") | — |
PgBatchTurns.finished |
a slot holding no batch ("PgBatchTurns.finished on a slot holding no batch") | link/std_postgres_batch_clock_turns_panic (slot 0 finished twice, after once) |
PgCarrier constructor |
a cap out of range: queries or executions outside 1..65536, connectTimeoutMs outside 1..3,600,000, or a watch id outside 0..15 (HANDLER_MAX_WATCHED − 1), checked before the poll set is made so that none leaks ("a PgCarrierLimits cap out of range") |
link/std_postgres_carrier_limits_panic (queries of 0, after 1 is taken); the other bounds share its condition |
PgCarrier constructor |
a target that failed, a TCP host that is not a numeric address (the carrier resolves no names), or a poll set the kernel refuses | — |
PgConnection constructor (and pgAsyncConnect, which makes one) |
PgConfig.tls other than PG_TLS_DISABLE: only PgCarrier speaks TLS, and a connection driven by hand would run in the clear ("PgConfig.tls asks for TLS, which only PgCarrier speaks") |
link/std_postgres_tls_connection_panic (PG_TLS_VERIFY_FULL, after PG_TLS_DISABLE) |
PgPool.connect |
PgConfig.tls other than PG_TLS_DISABLE on a pool no PgCarrier drives (the carrier marks its pool with carryTls, and its connections are made with a cleartext copy of the config under the carrier's TLS) |
link/std_postgres_tls_pool_panic (PG_TLS_REQUIRE, after PG_TLS_DISABLE) |
PgCarrier constructor |
PgConfig.tls other than PG_TLS_DISABLE on a Unix socket target ("PgConfig.tls asks for TLS on a Unix socket") |
link/std_postgres_tls_unix_panic (PG_TLS_REQUIRE, after PG_TLS_DISABLE) |
PgCarrier constructor, through pgCarrierTrust |
PG_TLS_VERIFY_FULL with no root certificate read from tlsRootsPath, or from the system's store when it is empty ("PG_TLS_VERIFY_FULL with no root certificate to read") |
link/std_postgres_tls_roots_panic (a roots file that is not there, after PG_TLS_REQUIRE, which reads none) |
PgCarrier.attach |
past 8 batchers, or where PgPool.register refuses: after the pool connected, a NUL in the name or SQL, past 256 statements |
link/std_postgres_carrier_attach_panic (a ninth, after eight) |
PgCarrier.runInt, runText |
a batcher id attach never gave ("PgCarrier batcher id out of range") |
link/std_postgres_carrier_batcher_panic (id 1 with only id 0 attached) |
PgCarrier.request |
no event has named a query yet, or the event's query was run with no request | link/std_postgres_carrier_request_panic (a query run, no event read yet) |
PgCarrier.connection |
before any event named a connection | link/std_postgres_carrier_connection_panic (a query run, no event yet) |
PgCarrier.send |
no PG_CARRIER_BUILD open |
link/std_postgres_carrier_send_panic (with no event read yet) |
PgCarrier.next |
a PG_CARRIER_BUILD not yet ended by send() |
link/std_postgres_carrier_next_panic (with a build open) |
PgAsyncPool constructor |
a size outside 1..64 ("a PgAsyncPool size out of range"); a ticket packs its slot in 6 bits | link/std_postgres_async_size_panic (65, after 64 is taken) |
PgAsyncResult.text, isNull, int, through cell |
a row or column outside the result ("a cell out of range") | — |
pgAsyncQuery, pgAsyncRelease |
a ticket that is not a slot's current claim: never claimed, already released, or claimed before close() ("a ticket that is not a current claim") |
link/std_postgres_async_release_panic (a release of ticket 0 on a pool that never connected, after its size is read); link/std_postgres_async_double_release_panic (a second release of ticket 64); link/std_postgres_async_release_closed_panic (a release after close()) |
pgAsyncSeat |
a statement pgAsyncPrepare did not register ("a statement pgAsyncPrepare did not register"); a depth outside 1..16 ("a PgAsyncPool depth out of range") |
link/std_postgres_async_seat_statement_panic (statement 1 after one was prepared); link/std_postgres_async_depth_panic (depth 17) |
pgAsyncSend, pgAsyncTurn, pgAsyncAnswer, pgAsyncDone, through pgAsyncSeatOf |
a ticket that is not a current seat: never seated, already done, or seated before close() ("a ticket that is not a current seat") |
link/std_postgres_async_seat_ticket_panic (pgAsyncDone of ticket 0 on a pool that never connected) |
pgAsyncAnswer |
a turn not yet reached: an answer read without the turn pgAsyncTurn gave ("an answer read before its turn") |
— |
pgAsyncQuery |
a cap out of range: maxMessage or maxResultBytes outside 1024..2^30-1, maxColumns outside 1..65535, maxRows outside 1..2^24 ("a PgAsyncPool cap out of range", PG-36) |
link/std_postgres_async_caps_panic (maxMessage 1023) |
pgAsyncConnect |
a target that failed, or a TCP host that is not a numeric address (the pool resolves no names) | — |
Apart from the array misuse above, the builder calls do not panic: param*,
arrayStart and bindEnd with no bind open answer false and write nothing
("paramText without a bind is refused" and the lines after it, "without a
bind, arrayStart is refused"), as does paramBytes on a window outside its
array. An open array is the program's state, not the server's, so misusing it
is a panic rather than a refusal a caller could miss and send a Bind whose
array never closed.
PgCarrier.close() is not a panic site either. Queries in flight when it is
called get no further event: their requests are the program's to answer,
before or after it. Once closed, run, runInt and runText answer
PG_CARRIER_FULL and next() answers PG_CARRIER_NONE, so a carrier used
after close() makes no connection on its closed poll set, and a second
close() does nothing ("after close(), a second close() does nothing and
the carrier refuses queries", "and connects nothing").
TLS in the carrier
WP39 E6 closes PG-9 for PgCarrier. PgConfig.tls picks one of three modes:
-
PG_TLS_DISABLE(0, the default): no SSLRequest; the connection is in the clear, as before. -
PG_TLS_REQUIRE(1): TLS or no connection, but the certificate is not checked. It defeats a passive eavesdropper only: a man in the middle answers with a certificate of their own and is taken. -
PG_TLS_VERIFY_FULL(2): TLS, with the chain checked to a root read fromtlsRootsPath(the system's store when it is empty) and the leaf checked fortlsServerName, or the target's host when that is empty. This is the mode that defeats the PG-9 attacker.
Each connection writes the 8-byte SSLRequest (pgSslRequest, code
80877103), and only once all eight bytes are out reads exactly one byte of
answer, so bytes a server or man in the middle sends in the clear after it
never reach the backend parser (CVE-2021-23222's shape). After an N
nothing more is read; the test cannot tell a trailer read from one left
unread, since either way the result is PG_ERR_TLS with socketError 0,
so that rests on the code. After an S the next bytes go to the record layer as
the handshake's first record, which must be the server's ServerHello, so
anything else fails the handshake: a fake AuthenticationOk and
ReadyForQuery there fail the connection with alert 10
(unexpected_message). An answer other than S fails the connection with
PG_ERR_TLS and socketError 0. The TLS 1.3 client offers ALPN postgresql, as libpq
does, and key shares for x25519 and secp256r1: PostgreSQL before 18 allows
only ssl_ecdh_curve, prime256v1 unless set, and asks for it with a
HelloRetryRequest. A failed handshake is PG_ERR_TLS with socketError
the alert the client sent: 48 (unknown_ca) for a chain to no trusted root,
42 (bad_certificate) for a leaf naming another host. The startup message,
the SCRAM exchange and every query then pass through the TLS records; the
password and the SCRAM keys are handled as before (PG-12), and the TLS key
schedule is the client's, wiped when its connection is dropped, as
tls.md records.
The default stays PG_TLS_DISABLE. A program written before this change
keeps the bytes it sent, so no deployment that connects in the clear today
(a local server, a sidecar, a network that is trusted by construction) stops
connecting on upgrade; a Unix socket, where TLS buys nothing, keeps working
unchanged. PG_TLS_REQUIRE as a default would look safe and stop only an
eavesdropper, and PG_TLS_VERIFY_FULL as a default would refuse every
server without a certificate the host trusts. The choice is therefore the
program's, made once in its PgConfig, and the guide tells a program that
crosses an untrusted network to set PG_TLS_VERIFY_FULL.
TLS on a Unix socket target is a panic (Panic sites), not a silent
downgrade, and so is PG_TLS_VERIFY_FULL with no root to read. So is a
tls other than PG_TLS_DISABLE anywhere the carrier does not drive the
connection: a PgConnection made by hand, a pgAsyncConnect connection,
and PgPool.connect on a pool no PgCarrier took. A program cannot ask for
TLS and get the clear. No test reaches pgAsyncConnect with a TLS config:
its refusal is pinned through the PgConnection constructor it calls
(link/std_postgres_tls_connection_panic). The one way around is
PgPool.carryTls(), which the carrier calls: it is @internal, and a
program that calls it by hand has chosen the clear, as if it set tls to
PG_TLS_DISABLE.
Out of scope: TLS for the async pool (std/net/postgres-async.ts) and for
PgPool driven by hand, whose bytes the program moves itself: both have
no TLS, and both refuse a PgConfig that asks for it (above);
client certificates (sslcert), which TLS-7 records the client does not
offer; and sslnegotiation=direct, PostgreSQL 17's TLS without the
SSLRequest.
Tests: tests/link/std_postgres_tls serves a stand-in server on loopback
that answers the SSLRequest and runs the TLS 1.3 server with a chain made
for the test. It checks the SSLRequest's bytes; an N answer refused
with no StartupMessage sent; a query over PG_TLS_VERIFY_FULL, by address
and by tlsServerName, with the StartupMessage inside TLS; and
PG_TLS_REQUIRE taking a chain no root vouches for. Its six refusal lines
check a root the roots file does not hold, a leaf for another name and a
good leaf under another tlsServerName, each refused with no StartupMessage
sent. Forced to accept every chain (the verdict replaced by 0), those six
fail.
tests/postgres/live's tls check, which CI's postgres-live job runs
against a PostgreSQL service given a leaf from a root made in the job, reads
pg_stat_ssl under each mode (ssl true TLSv1.3 under require and
verify-full, ssl false under disable) and sees the untrusted root and
the wrong name refused with alerts 48 and 42.
Limits
Edges of the module a caller owns, recorded so that no one assumes otherwise:
-
The carrier's state is fixed at construction.
PgCarrierLimitssizes its query and execution tables (each 1 to 65,536, 256 and 64 by default) and its ticket map; at most 8 batchers attach, before the pool connects.run,runIntandrunTextanswerPG_CARRIER_FULLrather than grow a table, so a flood of a route's clients costs refusals, not memory: "past the queries cap the carrier is full".Once its connections are up nothing is allocated per query ("and the arena stays flat", over thousands of pipelined queries). That window is the connected steady state only: a statement the server refuses and a reconnect are outside it. Making a connection allocates — the pool's
PgConnectionand its buffers — so a server that drops connections in a loop costs one connection's memory each time it is made again, as it does forPgPoolwithout the carrier. -
The carrier does not drive
PgBatchClock. A batcher it carries is taken as soon as a connection can take it, never held for a batching window;PgBatchClockstays the tool of a program that drivesPgPoolitself. -
A server that skips authentication is believed. An
AuthenticationOkin place of a SASL request is taken; there is norequire_authequivalent (PG-10). -
maxInFlighthas no upper bound on a lone connection. It is the program's own cap on groups sent and unanswered; the bytes they hold are bounded bymaxOutput. A pool refuses one over 1024, since each slot holds a ring of that many tickets. -
A fresh nonce for each connect is the program's.
PgPool.connecttakes the nonce it is given; the pool checks its length (18 bytes or more), not its freshness. A program that reuses one weakens SCRAM's replay protection;crypto.getRandomValuesfor each call is the rule. -
A
PgPool's sockets are the program's. The pool never opens or closes one; under aPgCarrier, the carrier is that program. The program closes a slot's socket onPG_FAILED, callsclosed()when a read answers 0 or an error other thanAGAIN, or a write fails, and at the end sendsterminate()and closes each socket itself:disposewipes the passwords but neither sendsTerminatenor closes a socket. The carrier'sresetcheck pins the read error. -
A pool neither times out nor retries. A request waits in the queue until a connection takes it; one lost with its connection (
PG_LOST) is the program's to run again, since the pool cannot know whether the server committed it. The carrier owns every clock. -
Named statements are not tracked. A second
prepare("x")before the first is closed is sent, and the server answers the group with42P05; the caller keeps names unique. -
Timestamps are checked for shape, not range.
PgText.timestampchecks the ISO digits and separators, the year (0001–9999) and the offset, but not the month, day or time:2024-99-99passes into its RFC 3339 output. The values are the server's, and trusted as such.PgBinary'stimestampandisocompute the date from a microsecond count, so every output is a real date; a year outside 0001–9999,infinityamong them, is refused. -
A binary column is read as the program says it is. Rows carry no format of their own: a program that reads a text row with
PgBinary(or a binary one withPgText) gets the wrong value orfailed, never a read outside the column. Each binary decoder checks the column's window against its buffer and its length against the type's (2, 4 or 8 for an integer, 1 for abool, 8 for afloat8or a time) before it reads a byte.An 8-byte column's bytes are then read by
nish/bytes'sgetI64BEorgetF64BE, which sincebench/pg-decodecheck all eight indices once rather than one at a time: inside the window this decoder has already checked, the read is the same value; outside it, the same panic (docs/security/bytes.md).PgBinaryArraywalks a one-dimensionalint2[],int4[]orint8[]the same way: its header'sndim(0 or 1),hasnulland element OID (20, 21 or 23) are checked, and every element's length word must equal its type's size (a NULL needshasnull1), each before the bytes it names are read. The walk ends withPG_ARRAY_ENDonly when it has consumed the column exactly, so a count the header gives and bytes left over, or a short column, both fail. Its only allocation is its constructor's, and no element is read past the column's window. -
The output buffer's doubling is bounded by
maxOutput. Every message first makes room by moving the unsent bytes to the front (PG-13), so the buffer grows past its length only when the unsent bytes need it, and never past twicemaxOutput(at most 2^31 − 2, inside ani32). -
maxOutputis the program's to size againstmaxBytes. A take stops at the first key the connection refuses for room, and a key longer than a connection could ever hold stays at the head of its queue;addTextrefuses only a key that could never fitmaxBytes. Such a key blocks every request behind it: each take answers -1 and none is ever sent, which is why the sizing rule below matters.A take whose array fills
maxBytesfits a connection with nothing else unsent whenmaxBytes + name.length + 32 <= maxOutput(the Bind's head of 11 and the name, the array's length word of 4, andPG_TAIL's 17): "a full take fits maxOutput at the bound", "and a byte past it is refused". The guide states that bound; the batcher does not read the connection's caps when it is made, since one batcher serves every connection of a pool. -
The async pool's query waits have no deadline of their own (PG-33, PG-37). A connect has
connectWaitMs(10 s) to connect and finish its startup (PG-40). A server that accepts a query and never answers holds the task that awaits it, and every task seated behind it on that connection, until the task is cancelled (cancelAfter, orstd/net/serve'sdeadline); a cancelled query's connection is closed, and the server is not sent a CancelRequest, so setstatement_timeouton the server, or the role, to bound a statement. -
A claim or a seat waits at most
claimWaitMs(PG-31). When every connection is held (or every seat,deptha connection),pgAsyncClaimandpgAsyncSeatwait on the pool's wake, which a release or a done rings, and answer -2 onceclaimWaitMs(5000 by default) has passed. Tasks waiting for a seat are seated in the order they began to wait (PG-42): the first as many as there are seats free may each take one, so seats freed together go in one loop turn, and a task that has not waited takes one only while none waits: "eight seats freed together: 8 seated, over at most 5 loop turns", where one a turn took nine (link/std_postgres_async_pipeline). A wake counts only the seats a task could take at once, so a connection mid-write wakes no one until its write ends. -
A task waiting for its answer yields before it parks.
pgAsyncTurnyields to the loop (sleep(0)) up toPG_ASYNC_TURN_YIELDS(4) times while the answer ahead of its own is unread, then parks on its seat's turn lane. A done, skip or abandon that leaves a turn come rings the lanes of the seats waiting on that connection in the order their answers come. Under Linux epoll, which hands back distinct descriptors in the order they became ready, seats freed together are each woken in the one loop turn; the runtime promises only the kernel's order, and kqueue specifies none, so there a seat woken early parks again and costs a turn. The yields are bounded, so a stalled server costs four loop turns and then the same park,claimWaitMsand cancel as before. -
pgAsyncReadgrows its buffer only as an answer needs it. The buffer starts the size of the input already read for the answer, then doubles (fromPG_ASYNC_INPUT_SIZEat the least) whenever it is full, clamped tomaxMessage+ 1 once doubling would pass it, so a message overmaxMessagestill failsPG_ERR_TOO_BIGbefore its body is read. -
A pool holds up to 257 socket pairs for its wakes. The pool has a wake, and so does each of 128 queue lanes and 128 turn lanes. A seat waiting for its turn waits on the turn lane its seat index picks, so a pool of
sizeconnections uses at most 16 ×sizeof them, and slots eight apart share theirs. A pair is opened the first time a task waits on its wake and kept untilclose(). A pool nobody waits on holds none, and one whose waits stay within a few lanes holds a few. Under a long queue, with every seat waiting its turn, it holds every pair its seats can pick: 514 descriptors for a pool of 8 or more atdepth16, 290 for a pool of 1. The ceiling rose with the turn lanes, from 2 × (size+ 129): a pool of 64 went from 386 to 514, one of 8 from 274 to 514, one of 1 from 260 to 290. When no pair can be opened (out of descriptors), a waiter sleeps 1 ms and looks again instead of failing. -
A pipelined connection is only as fast as its slowest statement (PG-39). Statements seated on one connection are answered in the order they were sent, so a slow one delays up to
depth- 1 behind it, and a connection lost or out of step fails every statement in flight on it withPG_ERR_CLOSED. -
An async query keeps its memory until its scope ends (PG-34). Run each query in a spawned task, whose region is freed when it ends; one awaited in
mainthrough a function handed the pool keeps about 9 KB untilmain's scope ends. -
An async result keeps at most
maxResultBytes(PG-36). 16 MiB by default: each row's bytes and 16 for each cell, and the bytes of each command tag and error kept. A query past it, or pastmaxRows(100000), failsPG_ERR_TOO_BIGwith the rows that fit; a message pastmaxMessage(1 MiB) fails before its body is read. -
The async pool reconnects only when told. A connection it closed stays closed;
pgAsyncConnectfills the slots that have none, and a claim with none live answers -1 at once.
Properties verified
Each property below is pinned by a test that fails if it stops holding. The
client's own checks are lines of tests/link/std_postgres/expected.out, which
link/std_postgres: exits with 0 and stdout matches expected.out compares
byte for byte. Exchanges recorded from PostgreSQL 18 by
tests/postgres/record.mjs are replayed whole, in 7-byte and in 1-byte
pieces, by link/std_postgres_replay: exits with 0 and stdout matches expected.out.
No backend bytes make the client panic (PG-1, PG-2):
the replay cuts every recording at every byte offset, feeds what is left and
calls closed(), and pins the failure code each cut ends on ("every
recording cut at every offset"); then 4000 mutants drawn from a fixed seed
(xorshift32, a bit flipped, a byte inserted or deleted, or the bytes cut)
are replayed, each must end PG_FAILED with a code from 1 to 10, and the
count of each code and a checksum over codes and logs are pinned ("fuzz: the
seed's mutants, by the code they ended with").
The end-state check cannot fail on its own, since closed() always leaves
PG_FAILED; what fails is a panic, which ends the run, or a moved count or
checksum. That a field is never read past its message's end, rather than merely
inside the buffer, rests on the targeted negatives under "Fields inside a
message" below, not on the fuzz.
Framing (PG-1): "a length under 4 is a framing failure", "a failed
connection stays failed", "a message over maxMessage is refused before it is
read", "a message larger than the buffer grows it", "and is read whole", "a
message a little over the buffer", "doubles it", "a byte at a time", every
recording "in 1-byte pieces", and link/std_postgres_receive_panic: exits with 1 and stdout matches expected.out for receive.
Fields inside a message (PG-2): "an error without its terminator", "a ParameterStatus without its value", "an authentication request under 4 bytes", "AuthenticationOk with data", "BackendKeyData of 4 bytes", "ReadyForQuery with a status other than I, T, E", "ParseComplete with a body", "a tag without its NUL", "EmptyQueryResponse with a body", "NoData with a body", "ParameterDescription short of its count", "a column longer than its row", "a row with bytes after its columns", "a column length of -2", "a RowDescription field cut short".
Caps (PG-3, PG-13): "a row over maxColumns", the maxOutput lines under
PG-13, and "maxInFlight groups sent: no
more", "prepare is refused", "bindStart is refused" for the client's own cap.
Order (PG-4): "a row before authentication", "authentication once started", "BackendKeyData once ready", "ReadyForQuery with no group in flight", "a row with no group in flight", "an ERROR with no group in flight", "an unknown message", "ReadyForQuery for a group not yet synced", "a notification is ignored".
SCRAM (PG-5 to PG-7): "a server nonce that is not ours", "a server nonce
that adds nothing", "a server nonce with a space", "no salt", "a salt that is
not base64", "iterations under minIterations", "iterations over
maxIterations", "iterations that would overflow", "iterations that are not
digits", "AuthenticationOk before the server proved the password"; "a server
signature the password does not make" (PG-6's PG_ERR_SERVER_SIGNATURE), "a
server signature of 31 bytes", "a server signature that is not base64", "a
server final without v=", "a server final of one byte"; "SASL without
SCRAM-SHA-256", "a SASL list without its end", "SASLContinue before SASL".
The recorded scram-ok exchange computes the proof PostgreSQL accepts and
checks the signature it sends, and scram-wrong-password ends with the
server's FATAL 28P01.
Refused methods (PG-11): "MD5 is refused", "cleartext is refused", and
the recorded cleartext exchange.
Encoding and session (PG-8): "a client_encoding other than UTF8", "DateStyle German, DMY", "DateStyle ISOX", "DateStyle empty", "TimeZone Europe/Paris", "TimeZone UTC0", "a SET TimeZone once ready fails the connection", against "DateStyle ISO", "DateStyle ISO, MDY" and "TimeZone UTC" accepted.
The password: read through expose, the connection's copy is the
password while SCRAM runs ("a new connection holds its copy of the
password", "still after SASL starts") and zeros after ("zeros once the proof
is computed", "zeros after fail()", "zeros after dispose()", "a failed
connection holds no password: its copy is zeros", "trust computes no proof:
zeros once AuthenticationOk arrives"); "dispose may be called twice".
Parameters never meet the SQL: every statement goes through Parse, Bind and Execute ("Parse and Describe", "Bind with every parameter kind, then Execute"), and "a NUL in a name is refused", "a NUL in SQL is refused", "a NUL in a closed name is refused", "a NUL in a bound name is refused", "and nothing is written".
Array parameters: an array goes as one text parameter in PostgreSQL's
array literal, and every element stays data. paramIntArray writes "every
value in decimal, the most negative too"; the builder writes "NULL unquoted,
every text element quoted, " and \ escaped, UTF-8 as it is", so an element
of NULL, ,, {, } or a quote cannot end the element or the array (the
live arrays scenario reads each back through unnest).
The size is exact before a byte is written: "an int8[] that would pass
maxOutput is refused" ("its exact size is asked for: one value less fits"), "an
element whose escapes pass maxOutput is refused" ("its escapes are counted: one
less fits"), "an element that would pass maxOutput is refused" "and the refused
one left nothing behind", and room for the closing } is held for each
element, so "the array still ends".
With an array open, "no sync while an array is open", "no prepare", "no closeStatement", "no bindStart"; a compaction under it leaves "the array's length patched where it moved to"; and "terminate drops an open array with its bind".
Binary integer arrays: paramInt4Array, paramInt8Array and the binary
builder write PostgreSQL's array wire form (array_send): ndim 1, the
has-nulls flag, the element OID (23 or 20), the length and lower bound 1, then
each element as its length word (4 or 8, or −1 for NULL) and its big-endian
bytes; an empty array is ndim 0 with no dimension, 12 bytes.
Every byte is the program's own integer, so nothing a client hands it can end
the array early or be read as another element: "an int4[] in binary: format
code 1, then ndim 1, no NULLs, OID 23, length 2, lower bound 1, each element 4
bytes", "an empty int8[] in a binary bind: ndim 0 and no dimension, 12 bytes",
"a NULL element is length -1", and PgBinaryArray reads each back, int4 and
int8 minimum and maximum included. An int4 element past int4 is a panic,
never a value cut to 32 bits that would match another row.
A Bind with a binary parameter carries one format code per parameter: the first
binary one holds room for a code per parameter so far and one more, each later
parameter holds two bytes, and bindEnd moves the parameters down and writes
the codes into room already counted against maxOutput ("a binary parameter
among text ones: a code for each, text 0 but its own 1", "an int4[] one element
too long is refused", "one that fits is taken", "and bindEnd and Sync stay
within maxOutput").
A bind with no binary parameter keeps its single zero word, so text binds are
byte for byte what they were ("a bind after a cancelled binary one owes no
codes", "a text int array after a binary one keeps code 0"). The element OID
must be the statement's: an int4[] sent to an int8[] parameter is the
server's ErrorResponse (SQLSTATE 42804), read like any other, and the live
intarrays scenario shows the next request on the connection succeeding.
Server errors: "an error during startup fails the connection", "FATAL
once ready fails it too", "an ERROR ends the statement, then the group", and
the recorded no-database exchange. The recorded pipeline-unique exchange
sends five groups in one write, the third a duplicate key: it ends with
23505 and the fourth and fifth are answered.
Pipelining: "sixteen groups, the default maxInFlight, go out unanswered", "a seventeenth may not start", "the first answer frees one place", "so the seventeenth goes out", "the answers come back in the order the groups were sent", "one answer more than was asked for".
Results from a real server: the recorded typed-row exchange decodes an
int8 past 2^53, an int4, a text, a timestamptz, a bool, a text[] and a
NULL; row-counts reads UPDATE 3, DELETE 2 and UPDATE 0 through
tagRows.
What it reaches: link/std_postgres_caps builds a connection under a
policy that denies every capability, so the module reaches none; the
std_postgres and replay programs compile under the same policy, the replay
with fs.read for its recordings.
std/net/postgres-carrier.ts is the module that reaches the network
(nish:net), the clock (monotonicNanos) and the entropy source (the SCRAM
nonce), and is its own module so that this one keeps to none.
The carrier on a handler's loop: link/std_postgres_carrier runs a
PgCarrier on an HttpHandler against a backend of its own in the same
program, which answers each statement from its bytes, and HTTP clients on
loopback. A plain query's row and a batched key's row each answer their own
request, and a key with no row is the program's 404 without failing the key
after it; three clients' keys go out in one execution, which the backend
counts, after the handler took all three in one turn, and each client
gets its own row or 404.
With the backend holding a slow query, another client is answered while it is held, and the held one only once the backend goes on. A connection that ends fails its plain query and its batch (PG-24) and closes its socket (PG-25), an execution of three keys lost with its connection fails all three requests, and the next queries are answered on new connections.
A statement the backend refuses with an ErrorResponse reaches its own request
as PG_CARRIER_ERROR, with the server's SQLSTATE, then PG_CARRIER_FAILED
with failure 0; a refused batch fails each of its requests, with failure 0 and
no PG_CARRIER_ERROR; and the connection that refused them, still open,
answers the queries after them. close() with a query held by the backend
sends Terminate and closes that connection, which the backend sees end, and
the handler goes on answering; the held query gets no further event, which
close()'s documentation states.
After it, a second close() does nothing, the
carrier answers PG_CARRIER_FULL to a query and PG_CARRIER_NONE to
next(), and the backend accepts no new connection.
After 100 warm-up rounds, a thousand rounds of four pipelined queries, half
of them batched, leave Arena.used() unchanged, with none refused for room; the queries cap answers
PG_CARRIER_FULL; a backend that never answers fails at the connect timeout
and not before it, and a refused connect fails at once (PG-23).
Decoders never write what they refuse: "year 0000" "leaves out as it
was", and a tag's count past an i64 is -1 and "leaves text.failed alone".
The binary decoders likewise: -infinity "leaves out as it was", and "one
byte", "three bytes", "no bytes", "a window past the buffer", "a window
before it", "NULL's length", "bool: 2", "float8: four bytes" and
"timestamp: four bytes" each answer failed without reading past the
column. "binary failures leave text.failed alone".
Binary results: "one result format code, 1: every column binary" pins
the Bind's bytes, and "the next bindStart is text again: no format codes"
that a binary bind does not leak into the next; "a binary bind refuses
that one" and "bindEnd is never refused for room" that its two extra bytes
are kept free under maxOutput from bindStartBinary on.
tests/postgres/live's binary scenario reads one statement's rows from
PostgreSQL 18 in binary and in text and gets the same line for each, and its
stamps_binary batch (a PgBatchConfig with binary set) reads the same
creation times as its stamps_text batch.
Binary arrays: link/std_postgres walks hand-built arrays. "ndim 0: the
empty array", "one int4", "1000 elements", "a NULL with hasnull 1" and "int2
elements" read in place and end with PG_ARRAY_END.
Each refusal sets binary.failed: "ndim 2", "another element type: text", "a
3-byte int4 element", "a 4-byte element in an int2 array", "a hasnull that is
not 0 or 1", "a NULL with hasnull 0", "a negative dimension", "more elements
promised than given", "bytes after an empty array", "truncated inside the
element", "truncated inside the length word", "truncated inside the header", "a
window past the buffer" and "a window before it". The live binary scenario
reads int4[] and int8[] columns, each empty, one element or 1000, in binary and
in text, and gets the same line for each.
Terminate: "Terminate drops an open bind", so a half-built Bind is never sent, and "terminate drops an open array with its bind".
The pool: link/std_postgres_pool drives a PgPool against scripted
servers. Its bounds: "a full queue refuses" "and holds what it had", "256
register; the 257th is refused", "a NUL in a name is refused", "a NUL in SQL
is refused". Its scheduling: "a full connection takes no more", "in FIFO
order", "the next to the one with fewer in flight".
Its failures: "a connection that fails loses each request it took, oldest
first", "a connection that fails starting, none ready, takes the oldest request
with it", "a request being built when its connection fails is lost", "it
answers nothing after PG_FAILED", "two slots fail starting, none ready: the
first takes the oldest". Its isolation: an error in the second of three groups
on one connection fails only that group ("the error is its own"). Its password:
"dispose wipes the pool's password" "and its connections'". It passes as
link/std_postgres_pool: exits with 0 and stdout matches expected.out.
The batcher: a key reaches the server only as an element of the one
bound array parameter, never in SQL text: sql is fixed when the batcher is
made, and keys are written by arrayElement and arrayInt ("text keys
quoted and escaped, each once" "and routed"). A row's ordinal is refused
when it is NULL, not an integer, outside the keys or going back (PG-14's
tests).
A spent run handle routes nothing, so no late event reaches a later
batch (PG-15's tests, and "a later batch cannot read an earlier one's
ordinals"). With binary set, a batch's rows come in binary as well: take
opens its bind with bindStartBinary and row reads the ordinal with
binaryInt, so a binary batch's refusals are PG-14's, read back in binary
("a binary take answers a run", "binary.failed stays as the caller set it").
Its state is bounded by its config, with each cap's bound a
panic test under Panic sites, and its refusals answered rather than grown:
"only the ones that fit wait", "a full take fits maxOutput at the bound",
"and a byte past it is refused". It passes as link/std_postgres_batch: exits with 0 and stdout matches expected.out, and the live batch
scenario below drives three batchers through a pool against PostgreSQL 18.
Connection targets (PG-18 to PG-20): the checks quoted in those rows are
lines of tests/link/std_postgres_unix_target/expected.out, compared byte
for byte by link/std_postgres_unix_target: exits with 0 and stdout matches expected.out.
That pgTargetUrl and pgTarget never panic is pinned by
3000 URLs drawn from a fixed seed (xorshift32) out of the bytes and escapes
the parser splits on — /, @, :, %, ?, &, =, brackets, ,,
%2F, %40, %00, a raw NUL, é and %C3%A9, ?host= and &port=
among them, cut anywhere: each must answer a failure code from
PG_TARGET_OK to PG_TARGET_TOO_LONG ("3000 seeded URLs: each answers a
defined failure"), no target it accepts may hold a NUL in its host or path
("no accepted target holds a NUL", PG-19), each socket target it
gives must end /.s.PGSQL.<port> within 107 bytes, or 108 for an @ name
("and each socket target fits sun_path"), and some must be sockets ("some of them sockets").
A panic ends the run, so the line after it is missing and the comparison fails.
Against a live server: tests/postgres/live/run.mjs runs the pool and
tests/postgres/carrier.ts against PostgreSQL 18, in CI's postgres-live
job against a postgres:18 service.
It checks that a wrong password is refused with 28P01 — the client refuses
MD5 and cleartext, so the server got that far only through SCRAM; that 100
pipelined requests over four connections, every one of which carries groups,
four of which violate a unique key, fail only those four, whose connections
recover, and that the rows the other 96 inserted are committed while the failed
groups' are not (read back with a fresh connection); that a backend ended by
pg_terminate_backend is reported lost and made again; that a division by zero
fails only its own group in a pipeline; that text[], int8[] and int4[]
parameters come back through unnest element for element — NULLs, quotes,
backslashes, braces, commas, the text NULL, the empty string, multibyte
UTF-8, the empty array and the extreme int8 values — with the int4[] literal
coerced by the server; that the batch scenario's three batchers (users by
int8 id over two takes, articles by text slug, and tags by article, several
rows a key in name order), each with a duplicate key and a key with no row,
answer every request its own rows and no other's through a pooled connection;
and that the conduit article list decodes, escapes and truncates as its golden
says. The carrier serves HTTP while a query sleeps. Its refused check points
it at a port with nothing listening: the query fails at once, failure 9, socket -111 (PG_ERR_CLOSED, ECONNREFUSED), rather than wait for the connect
timeout's -110. Its reset check points it at a server that resets the
connection on the startup message: the query fails failure 9, socket -104
(ECONNRESET), and the carrier ends the connection there rather than leave the
readable socket to spin. Its unix scenario connects through a postgres://
URL whose host is the percent-encoded socket directory: the server asks for no
password (peer for a cluster run.mjs starts, trust in CI's service), so a
password the server would refuse still logs in, and its copy is wiped; and a
multi-row read comes back over the socket, where inet_server_addr() is NULL.
The async pool, over loopback. tests/link/std_postgres_async runs the
pool against a stand-in backend in the same process (server.ts), so it needs
no server. Three tasks on a pool of two finish B, C, A, though A was sent
first: A's answer waits until B and C are done, and C could start only on the
connection B gave back, so the tasks overlapped and none waited on another's
socket, with no clock read (PG-30). It then checks, in one task, parameters in
text with a NULL, a server error that leaves the connection reusable, rows past
maxRows (PG-28), a NUL in a parameter (PG-32), a transaction left open and a
setting changed, each of which closes its connection (PG-29), and a claim with
none live; a connection that ends under a query (PG_ERR_CLOSED); the hostile
frames of PG-28, among them a message split across two writes, a 20000-byte
cell that grows the buffer, and a socket that ends mid-message; bytes after a
ReadyForQuery (PG-35); a result past maxResultBytes in rows, tags or errors (PG-36); and a claim that
gives up after claimWaitMs (PG-31), and a connect that gives up after
connectWaitMs (PG-40). tests/link/std_postgres_async_pipeline puts six
statements from six tasks in flight on one connection and checks each answer
reaches its task in order, through a refused Parse, a NUL, a row past the
input buffer, a cancelled task, a lost connection and a frame past
maxMessage (PG-39), that a waiting seat is woken, not polled (PG-31),
that tasks waiting for a seat are seated in their order, 150 of them at once
(PG-42), and that a cancelled task's answer is read past, whether it was
cancelled reading or waiting, and closes the connection only when it left a
transaction open (PG-43).
Three panic tests give a ticket back
twice, give one back after close() (PG-30), and set a cap out of range
(PG-36).
tests/link/std_postgres_async_flat runs 2000 queries, each in its own
spawned task, and reads Arena.used() before and after: "Arena.used() flat
over 2000 queries: true" (PG-34). tests/postgres/live's servepool runs the
pool against a live server: a serve handler awaits a query on a pool its
spawner connected, logging in by SCRAM (PG-38).
The async pool's tests:
tests/link/std_postgres_async(PG-28 to PG-32, PG-35, PG-36, PG-40)tests/link/std_postgres_async_flat(PG-34)tests/link/std_postgres_async_cancel(PG-33, PG-37)tests/link/std_postgres_async_pipeline(PG-31, PG-39, PG-42, PG-43)tests/link/tfb_async_db(PG-41)tests/postgres/liveservepool(PG-38)-
tests/link/std_postgres_async_double_release_panic,tests/link/std_postgres_async_release_closed_panic(PG-30) tests/link/std_postgres_async_caps_panic(PG-36)-
tests/link/std_postgres_async_seat_statement_panic,tests/link/std_postgres_async_depth_panic,tests/link/std_postgres_async_seat_ticket_panic(Panic sites) -
tests/link/std_postgres_async_release_panic,tests/link/std_postgres_async_size_panic(Panic sites)
The carrier's TLS tests (PG-9, "TLS in the carrier"):
tests/link/std_postgres_tls(PG-9)tests/postgres/livetls(PG-9)-
tests/link/std_postgres_tls_unix_panic,tests/link/std_postgres_tls_roots_panic,tests/link/std_postgres_tls_connection_panic,tests/link/std_postgres_tls_pool_panic(Panic sites)

