Security record: the SQLite amalgamation
The record of the SQLite that nish build links into a program that calls it
(runtime/sqlite/README.md). It says what is
vendored and from where, which inputs SQLite trusts and which are an
attacker's, the options it is compiled with, and how it is bumped for a CVE.
It is written when the C lands, not after an audit, so that an audit starts
from evidence.
sqlite3.c and sqlite3.h are as vendored at 46559bf; the compile options
are named by the "SQLite's compile-time options" section of
scripts/build.sh.
Result. SQLite 3.54.1, byte for byte as upstream publishes it; no line of it is patched. The review of the adapter found two Low findings: one is fixed, one is open (Findings).
Scope
| File | What |
|---|---|
runtime/sqlite/sqlite3.c |
The amalgamation, upstream 3.54.1, unchanged |
runtime/sqlite/sqlite3.h |
Its public header, upstream 3.54.1, unchanged |
runtime/sqlite/nish-sqlite.c |
The nishsqlite_* adapters nish/sqlite calls: handles, and the bounds of a buffer SQLite reads or writes; the B-tree cursors (nishsqlite_cursor_*, nishsqlite_key_*, nishsqlite_begin_read, nishsqlite_end_read) and the authorizer that invalidates them; the #include "sqlite3.c" that makes it SQLite's one unit |
std/sqlite.ts |
nish/sqlite, the module over the adapters, Cursor and Database.beginRead, endRead, openCursor among them |
Out of scope: the program's own SQL.
The adapter
nish/sqlite never holds a C pointer. A connection or a statement is an i32
handle, (generation << 16) | index into a thread-local table, and every
adapter looks it up before it touches SQLite: an index past the table, an empty
slot or a stale generation is SQLITE_MISUSE, so a handle used after close
or finalize, or forged, reaches no freed or foreign object. Slot 0 is never
handed out, so the zero handle names nothing.
A freed slot's generation moves on, and a slot whose generation reaches 32,767,
the field's width, is retired rather than restarted, so no handle is ever
handed out twice and a stale one can never come to name a live object. A table
whose 65,535 slots are all live or retired answers SQLITE_NOMEM. Tests:
tests/link/std_sqlite (a handle after close and after finalize, a forged
handle).
The tables are per thread: a handle passed to another thread names nothing
there and answers SQLITE_MISUSE. A thread's tables are not freed when it
exits, so connections a thread leaves open stay allocated with their files
open; that is a leak, not a use after free, since no other thread can name
them.
Errors carry the connection's message only when SQLite set it. A code the
adapter answers without SQLite — SQLITE_MISUSE, SQLITE_RANGE, and
SQLITE_NOMEM from a full table or a value SQLite could not convert to text
in columnInto — takes sqlite3_errstr's message for the code instead of the
connection's, which is still that of some earlier call.
A read of a column that is not there, or of any column when no row is current
(before the first step, or after the last), is answered before SQLite and
leaves the connection's message as it was. close frees the handle only when
sqlite3_close_v2 succeeds, so a close that fails leaves the connection and
its message readable; its statements are already finalized. columnText reads
a value SQLite fails to convert under memory pressure as null, the same as
SQL NULL (SQL-2).
SQL text, file names and bindText's string reach C as const char *, where
a NUL byte would end them early. The FFI checks each one before the call and
stops the program with the ffi-nul panic rather than passing a shortened
string (tests/cases/panics_ffi_nul); bindTextBytes and bindBlob take a
length and pass NUL bytes through.
The adapter bounds every buffer it is given with the buffer's length: a bind of
buf[off, off + len) outside 0..n is SQLITE_MISUSE and binds nothing, and
columnInto copies only when the whole column fits after at, otherwise
answering the length it needs and writing nothing. Text and blobs are bound
SQLITE_TRANSIENT, so SQLite copies them and no Nish array is read after the
call returns. Tests: tests/link/std_sqlite (stale and forged handles, windows
out of range, a buffer too small).
Cursors
A Cursor reads one table or index of main through SQLite's own B-tree
cursor and record decoder, called from the same unit
(docs/sqlite-cursor.md). The adapter has no reader of
its own for the file format or the record format, and writes no field of
BtCursor, Btree, BtShared, Mem or sqlite3.
A cursor is an i32 handle in its own slot table, with the generation scheme
above, and the slot records the connection that owns it. A closed, forged or
invalidated handle is SQLITE_MISUSE. nishsqlite_close frees a
connection's cursors before its statements and its sqlite3, so no handle
outlives the B-tree it reads.
A raw B-tree cursor is outside SQLite's statement bookkeeping, so the transaction could end, or pages move, under it. The adapter frees every cursor of a connection before that can happen:
- at
endRead, before itsCOMMIT; - in
nishsqlite_exec, before it runs anything; -
in
nishsqlite_step, before it steps a statement that is not a plain read. The authorizernishsqlite_openinstalls classifies each statement at prepare: a plain read authorizedSQLITE_SELECTand nothing butSQLITE_SELECT,SQLITE_READ,SQLITE_FUNCTIONandSQLITE_RECURSIVE; -
in the authorizer itself, when an action off that list is authorized
outside a prepare it classifies (a re-prepare inside
sqlite3_step); -
in every cursor call (
cursor_of), whensqlite3_get_autocommitis set or themainB-tree's transaction state isSQLITE_TXN_NONE: a rollback SQLite made by itself.
The authorizer never refuses an action; it only watches. A statement it cannot classify as a plain read is treated as one that invalidates.
openCursor takes the root page from sqlite_schema on the same
connection, never from the caller. It refuses with SQLITE_SCHEMA an object
it cannot read as a plain rowid B-tree of known shape: a view, a trigger, a
virtual table and its shadow tables, a WITHOUT ROWID table or an index on
one, a generated column, an expression or non-BINARY index column, a
declared column list that differs from the schema's, and a database whose
encoding is not UTF-8.
Column reads are bounded twice. A column number outside the declared list is
SQLITE_RANGE, and one past the fields the row's record holds is
SQLITE_SCHEMA. columnInto and keyTextBytes bound the caller's buffer as
columnInto and bindTextBytes do: an at past the buffer, or a window
outside it, is SQLITE_MISUSE, and a value is copied only when it fits.
A record's length is read with sqlite3BtreePayloadSize. Over
SQLITE_MAX_LENGTH it is SQLITE_CORRUPT, which keeps the decoder's u32
offsets from wrapping. sqlite3VdbeRecordUnpack checks each field's start
against the record's length and sets the field that runs past the end to
NULL, so no value a read answers reaches past the record. On a corrupt
record it can still read a header varint and an 8-byte value past the end.
Where the decoded bytes lie:
-
A record wholly on its page is decoded where it lies. That is the read
SQLite's own VDBE and
sqlite3BtreeIndexMovetomake, into the overrun area the page cache keeps after each page buffer. -
A record that spills onto overflow pages is copied whole with
sqlite3BtreePayloadinto the cursor's buffer. 18 zeroed bytes are kept after it: the paddingsqlite3BtreeIndexMovetogives the copy it decodes.
A seek key is built only with sqlite3VdbeMemSetInt64 and
sqlite3VdbeMemSetStr (text copied, SQLITE_TRANSIENT), and no more key
parts than the index has key columns: one more is SQLITE_RANGE.
Tests:
-
tests/link/std_sqlite_cursor: the moves, the seeks and the column reads, checked against SQL, with 142 records spilling onto overflow pages; a closed cursor, a column out of range, anatpast the buffer, akeyTextByteswindow outside it, one key part too many, and a buffer too small. -
tests/link/std_sqlite_cursor_stale: a cursor open acrossCOMMIT,ROLLBACK,commit(),rollback(), a write throughexecand through a prepared statement, a preparedROLLBACK,endRead, aSELECTon apragma_*function andcloseanswersSQLITE_MISUSE; so does one open inside a read across a preparedUPDATE, anexecDELETE, DDL throughexecand a prepared statement,SAVEPOINTthenRELEASEand aPRAGMA x = y, and a closed cursor's handle once a new cursor takes its slot. It runs again with-DSQLITE_DEBUG, so SQLite's assertions check the same sequences. -
tests/link/std_sqlite_cursor_sql: every walk of a table and an index in both directions, and each seek, checked row by row against theSELECTthat answers the same question, with index keys and records on overflow pages andNULLkeys. -
tests/run.js(sqlite-link): the adapter built with-DSQLITE_DEBUGandTHREADSAFE=1under a C helper that calls every exported cursor and key function, so SQLite's assertions check that each holds the connection's mutex; and, with memory statistics on, two passes of index seeks and a backward walk over overflow rows that leave SQLite's memory in use and its high-water mark where the first pass left them. -
tests/link/std_sqlite_cursor_schema: each refused object, and a column whose name holds a comma or ends inDESC. It runs again with-DSQLITE_ENABLE_RTREE, for an rtree table and its shadow table. -
tests/link/std_sqlite_cursor_feed: a feed read through four cursors, equal byte for byte to SQL's.
Untested: a read that answers SQLITE_ABORT because the row moved under the
cursor (sqlite3BtreeCursorHasMoved; the cursor stays open), SQLITE_CORRUPT
for an oversized record, and the SQLITE_NOMEM paths, each of which needs a
corrupt file or a failing allocator the suite has no harness for; the
invalidation by a re-prepare whose statement now authorizes an action off the
allowlist, which needs another connection to change the schema between a
prepare and its step; and the invalidation cursor_of makes when SQLite has
rolled the transaction back by itself, which needs an I/O error mid-read.
Findings
| ID | Severity | Where | Finding | Status |
|---|---|---|---|---|
| SQL-1 | Low | runtime/sqlite/nish-sqlite.c (slot_give) |
A freed slot's 15-bit generation wrapped back to 1, so after 32,767 reuses of one slot a stale handle named the live object then in it: a statement or connection the holder never opened. | Fixed. A slot whose generation reaches 32,767 is retired, never handed out again. |
| SQL-2 | Low | std/sqlite.ts (columnText) |
sqlite3_column_text answers NULL both for SQL NULL and when converting the value to text runs out of memory, and columnText reads either as null. A program can take a value it failed to read for an empty column. columnInto tells the two apart and answers SQLITE_NOMEM. |
Open. Telling them apart in columnText needs a Result answer, a change to the API. |
Trust model
SQL text is the program's and is trusted: SQLite runs whatever statement it is given, so a program that splices an attacker's string into SQL has handed them the database. Values reach a statement through bound parameters, never through its text.
A database file is not. A file from an untrusted source — a download, an
upload, anything another party could have written — is attacker input to
SQLite's parser, and SQLite's own guidance for it applies
(sqlite.org/security.html): open it with
SQLITE_DBCONFIG_DEFENSIVE set, which refuses the writes that can corrupt a
file (PRAGMA writable_schema, writes to shadow tables, direct page writes),
and run PRAGMA quick_check or integrity_check before trusting its content.
Loadable extensions are compiled out (below), so a database or a statement
cannot load code.
Compile options
Every unit in runtime/sqlite/ is compiled with:
| Option | Why |
|---|---|
SQLITE_DQS=0 |
A double-quoted string is an identifier only, never a string literal |
SQLITE_DEFAULT_MEMSTATUS=0 |
No memory statistics, and no mutex to keep them |
SQLITE_DEFAULT_WAL_SYNCHRONOUS=1 |
synchronous=NORMAL in WAL mode |
SQLITE_LIKE_DOESNT_MATCH_BLOBS |
LIKE and GLOB do not match a blob |
SQLITE_MAX_EXPR_DEPTH=0 |
No limit on expression depth: the parser does not count it |
SQLITE_OMIT_DEPRECATED |
No deprecated interface |
SQLITE_OMIT_PROGRESS_CALLBACK |
No progress callback |
SQLITE_OMIT_SHARED_CACHE |
No shared cache. The cursors rely on it: with a shared cache another connection could free pages under one, so nish-sqlite.c refuses to build without it (#error) |
SQLITE_USE_ALLOCA |
Scratch space on the stack where SQLite allows it |
SQLITE_OMIT_LOAD_EXTENSION |
No load_extension: nothing can load code |
SQLITE_THREADSAFE=0, or =2 with --threads |
Single-thread mode, or multi-thread mode: no mutex guards a connection, so a program must not share one across threads (serialized mode is 1); always 0 on wasi |
NISH_SQLITE_CFLAGS is appended, and a -D or -U in it for one of these
names replaces the default. A program built with it is compiled with what its
builder chose, and this record covers only the defaults.
SQLITE_MAX_EXPR_DEPTH=0 removes the limit, so a deeply enough nested
expression recurses until the stack overflows and the process dies. Only SQL
text reaches the parser, and it is trusted, so this is recorded here rather
than as a finding. A program that runs SQL it did not write rebuilds with
NISH_SQLITE_CFLAGS=-DSQLITE_MAX_EXPR_DEPTH=1000 (SQLite's default); that is
the only mitigation. sqlite3_limit(db, SQLITE_LIMIT_EXPR_DEPTH, n) has no
effect in the default build: the height check and every nHeight update sit
under #if SQLITE_MAX_EXPR_DEPTH>0 (sqlite3ExprCheckHeight at
sqlite3.c:23070 is an empty macro otherwise), so the runtime limit is never
read.
HAVE_USLEEP is not defined, and need not be. In 3.54.1 the default busy
handler (sqliteDefaultBusyCallback, sqlite3.c:190106) and unixSleep
(sqlite3.c:47805) key on HAVE_NANOSLEEP, which is on unless a build defines
it to 0. The default build therefore backs off in 1 to 100 ms steps, and
PRAGMA busy_timeout honours a timeout under a second. The whole-second
handler returns SQLITE_BUSY at once for any such timeout. A SQLite bump or a
changed default that brings it back fails "link/sqlite_busy_backoff"
(tests/link/sqlite_busy_backoff): a second connection that waits 300 ms on an
exclusive lock must give up after at least 250 ms.
Bumping for a CVE
Follow "Updating" in runtime/sqlite/README.md:
it lists every file the version lives in and the sections to run. Then update
the version and the commit in this record, and the version in
docs/guide/using-nish/cli.md. The changed sources change the SQLite cache
key, so every program relinks against the new objects.
Not yet covered by the scope check
scripts/security-scope.mjs reads top-level runtime/*.c and does not descend
into runtime/sqlite/, so it does not yet require these files to be named
here. Widening its surface is a scripts/ change for a later pull request.
Tests
Besides the cursor tests under Cursors:
tests/link/sqlite_busy_backoff pins the lock-wait backoff above
("link/sqlite_busy_backoff: exits with 0 and stdout matches expected.out").
In tests/run.js, section sqlite-link:
- "sqlite-link: runtime/sqlite/README.md records the SHA3-256 of sqlite3.c and sqlite3.h as they are" — the vendored files are the ones the README's hashes name, so an edit to either fails it.
- "sqlite-link: NISH_SQLITE_CFLAGS=-DSQLITE_THREADSAFE=1 compiles runtime/sqlite/ alone again, into a new entry, and replaces the default" — a builder's option replaces the default rather than joining it.
- "sqlite-link: a program that does not call SQLite compiles no C, adds no entry and links no SQLite symbol" — a program that never names SQLite carries none of its code.
-
"sqlite-link: a string constant that names
@sqlite3_opendoes not link SQLite" and "sqlite-link: a string constant that names@nishsqlite_opendoes not link SQLite" — only a declaration triggers the link, not text in the program. -
"sqlite-link: the napi addon exports no
sqlite3_symbol" — an addon keeps SQLite's API out of the Node process's namespace.

