Security record: nish/bytes — byte-class runs and copyBytes
The record of the byte-class runs of nish/bytes (#862): skipBlank,
skipDigits, skipJsonText and skipJsonPlain (#892). Each of these lowers
to a runtime kernel in runtime/runtime-bytes.c with scalar, SSE2, NEON and
wasm simd128 paths, and AVX2 paths for the first three. The record also covers copyBytes, which lowers to llvm.memmove. It
says what each function refuses and with what, where each bound is checked,
and the test that pins each property. It is written against main at
804120e and cites functions rather than lines.
Result. Given an array of at most 2^31-1 bytes, no window the attacker
chooses makes a run read outside the array or answer outside its clamped
window. No range makes copyBytes touch memory outside either array. An
out-of-range copy panics, and that is the program's own mistake. Overlapping
copies are correct in both directions. Every run is linear and allocates
nothing. Two Low findings are recorded, both fixed:
-
BYTES-1: an array longer than 2^31-1 handed in by a host reads with a
negative
.length. The runs then answer outside the window andcopyBytespanics falsely. Memory safety still holds. The N-API shim now refuses such an array, andnish.hstates the bound for C hosts. -
BYTES-2: nothing pinned that
std/cryptokeeps out of the variable-time runs. A check intests/run.jsnow does.
Scope
| File | Functions |
|---|---|
runtime/runtime-bytes.c |
nish_bytes_skip_blank, nish_bytes_skip_digits, nish_bytes_skip_json_text, nish_bytes_skip_json_plain; the helpers nish_run_in, nish_run_utf8, nish_run_wide, nish_run_scalar, nish_run_resume, nish_run_bad_sse2, nish_run_cut, nish_run_sse2, nish_run_avx2_utf8, nish_plain_scalar, nish_plain_base, the SSE2/AVX2/NEON/wasm block walkers, nish_cpu_has_avx2, nish_run_choose, nish_run_settle and the resolver |
std/bytes.ts |
bytesRunEnd, bytesRunStart, skipBlank, skipDigits, skipJsonText, skipJsonPlain, copyBytes, and the walkers they lower from |
std/bytes.ts |
getU64BE and getU64BEByByte: the single range check of an eight-byte big-endian read, which getI64BE and getF64BE call |
src/kernels.ts, src/emit-arrays.ts, src/std-modules.ts |
bytesWalkOf, bytesKernelOf, emitBytesKernel, bytePointer, isBytesModule |
src/runtime.ts, runtime/nish.h |
the kernel declarations and their contract |
src/secret.ts |
isPureRuntime's admission of the four kernels |
Out of scope:
-
std/json-tokenizer.ts, which calls the runs; its record isdocs/security/json.md. -
The host interop boundary as a whole (
docs/security/codegen.md,docs/security/runtime.md), apart from where it reaches this module (BYTES-1). - The NEON path was checked by inspection and by the binfmt agreement run, not under a sanitizer.
Threat model
The attacker writes the bytes and chooses from, to, at and count:
any i32, including negative values, INT32_MIN and INT32_MAX. A win is
any of:
- a byte read or written outside an array;
- a run answer outside
[from, to]after the documented clamp; - a copy that overlaps wrongly or passes its bounds;
- a panic on a valid call;
- time not linear in the window.
The program is trusted. A host that builds arrays itself is trusted to keep the runtime's length invariant, which BYTES-1 asks to have stated and enforced.
Findings
| Id | Severity | Where | Description | Disposition |
|---|---|---|---|---|
| BYTES-1 | Low | src/interop-napi.ts (nish_napi_array_arg), runtime/nish.h; seen through std/bytes.ts (bytesRunEnd, bytesRunStart, copyBytes) |
The runtime caps arrays at 2^31-1, but a host can hand in a longer one: a C-ABI nish_array with len past INT32_MAX, or under --emit-napi a borrowed Uint8Array(2**31+5), which the shim takes without a cap. In i32 mode toI32(b.length) is then negative. skipBlank(b, 0, 10) answers -2147483643. The copyBytes guard's sub nsw overflows: poison in the IR, a wrap in practice. A valid small copy then panics and exits the host process. No out-of-bounds access was shown. |
Fixed. nish_napi_array_arg answers -1 for len > INT32_MAX, and the shim throws TypeError: <fn>: argument N (<name>) has more than 2147483647 elements; nish.h states the bound for C hosts, which the runtime cannot check ("N-API: a typed array longer than 2^31-1 is a TypeError, not an array whose length reads negative"). |
| BYTES-2 | Low | src/secret.ts (isPureRuntime) |
isPureRuntime admits the run kernels (three at the audit, four since #892), which are variable-time, so code inside expose may call them. No test pins the rule that std/crypto does not, though no std/crypto module imports nish/bytes today. nish_str_index_of_any has the same pattern. |
Fixed. tests/run.js compiles every std/crypto/*.ts and refuses IR that names nish_bytes_skip_* or nish_str_index_of_any; tests/link/std_bytes_runs is its control, which must name them ("std/crypto: no module's IR calls a variable-time scan kernel (14 modules; std_bytes_runs, the control, names 8)"). |
Properties verified
No load passes to, on any path (simd: byte-runs native ${process.arch}: ...
and the wasm and aarch64 labels, tests/simd/byte-runs.c). Each vector path
loads a 16- or 32-byte block only while at least that many bytes remain
before to, and finishes in the scalar loop. A text run that stops on a lead
byte resumes through nish_run_wide, which reads no further than to. The
audit also ran these paths under ASan+UBSan and against guard-page windows;
CI pins agreement between paths, not the sanitizer run.
Every path answers as the scalar one does (simd: byte-runs native x64: 7682442 cases agree across scalar base avx2 (plain: scalar base); the resolver picks every pin). This covers every class, the plain class's scalar and
16-byte paths, every alignment and every pin of NISH_SIMD.
The SSE2 block check stops a text run where the scalar UTF-8 step does
(the same label; since #1584). A text block holding a high byte is checked
whole: a byte is flagged when its continuation count disagrees with the leads
before it, when it is a lead no RFC 3629 form starts with (C0, C1, F5 and up),
or when it is the second byte of an overlong, surrogate or past-U+10FFFF form.
The block is taken only when nothing is flagged up to its first ASCII stop, or
in it at all; a sequence the block's end cuts is left to the next block, and
anything else goes to the scalar step, which finds the exact stop. On AVX2 a
block with no stop at all hands the run back to the 32-byte loop at its start,
a block boundary no sequence crosses, so neither side resumes mid-sequence.
Before the change the step took every sequence. tests/simd/byte-runs.c puts each valid
edge form, each overlong, surrogate and out-of-range form, each form cut by
ASCII, " or a new lead, and each stray or extra continuation at every offset
of a 72-byte stretch in dense multi-byte text, with to on every byte past it,
then runs 60,000 dense random trials; every path must give the scalar answer.
The JSON fixtures that read invalid UTF-8 (link/std_json_tokenizer,
link/std_json_conformance) answer JSON_TOK_ERR_UTF8 at the same offsets.
A run answers inside its clamped window (link/std_bytes_runs: exits with 0 and stdout matches expected.out). bytesRunEnd clamps to into
[0, b.length] and bytesRunStart clamps from into [0, end] before the
kernel is called. The audit tried windows with bounds at INT32_MIN, -1,
0, INT32_MAX and either side of the length, and found no mismatch. The
fixture covers only in-range windows; the extreme ones are not yet pinned.
copyBytes checks the whole range before it copies
(link/std_bytes_copy_outside: exits with 1 and stdout matches expected.out).
The guard refuses a negative count, at or from, and refuses
at > dst.length - count and from > src.length - count. With lengths of at
most 2^31-1 and count >= 0, neither subtraction overflows, so the copy is
inside both arrays. Overlap is a memmove. The fixture pins only a source
range past the end; the other edges and the overlap order were checked in the
audit and are not yet pinned.
An eight-byte big-endian read checks its range once and panics as before
(link/std_bytes_read_be_end, link/std_bytes_read_be_short,
link/std_bytes_read_be_empty, link/std_bytes_read_be_cap,
link/std_bytes_read_max_offset, link/std_bytes_read_be_negative).
getU64BE reads the eight bytes without a per-byte check only when off >= 0,
off < 2147483640 and off + 8 <= toI32(b.length). The cap keeps off + 8
from overflowing, and in a program that calls it the checker proves each of
b[off] to b[off + 7] from the guard; where nothing calls it
(link/std_dns_caps), the checks on b[off + 1] to b[off + 7] stay, behind
a guard they cannot fail under. That is why the capability report now gives
getI64BE's and getF64BE's call to getU64BE "via": "index" where it gave
"overflow". via names the first site, in run-time order, that the callee
can panic at. Before, that was the overflow check on the loop's off + i; now
it is the index check on the fast path's b[off + 1]. The set of inputs that
panic is the same.
Any other offset goes to getU64BEByByte, the earlier loop, which checks each
index and panics index on the first byte not in the array, so the panic text
is unchanged. An array longer than 2^31-1 (BYTES-1) makes toI32(b.length)
negative, or smaller than the real length, so the guard can only refuse an
in-range read, never pass one that is out of range. The fixtures pin a read
ending on the array's last byte, one byte short, an empty array, the first
capped offset, INT32_MAX and -1.
Only the CLI's nish/bytes lowers to the kernels (none yet). isBytesModule
matches the module by package as well as path. A std/bytes.ts compiled as a
root calls its own walkers.
No std/crypto module calls a variable-time run (BYTES-2's check,
above). The rule is the ct-verification record's: secret-dependent input
never reaches a kernel that returns early.
A host cannot hand in an array whose length reads negative (BYTES-1's
N-API check, above). A C host is bound by nish.h's contract instead.
The freestanding wasm link adds the kernels once
(simd: the freestanding --profile wasm link of nish/bytes's runs compiled ${kind} adds runtime-bytes.c once, and answers as Node does).
Tests:
simd: byte-runs native ${process.arch}: ... — the template
simd: ${label}: ${said}, with labels byte-runs native ${process.arch},
byte-runs wasm32-wasi, the scalar base path,
byte-runs wasm32-wasi -msimd128 and
byte-runs aarch64 through binfmt, the NEON path;
link/std_bytes_runs: exits with 0 and stdout matches expected.out,
link/std_bytes_copy_outside: exits with 1 and stdout matches expected.out,
link/std_bytes_read_be_end: exits with 0 and stdout matches expected.out,
link/std_bytes_read_be_short: exits with 1 and stdout matches expected.out,
link/std_bytes_read_be_empty: exits with 1 and stdout matches expected.out,
link/std_bytes_read_be_cap: exits with 1 and stdout matches expected.out,
link/std_bytes_read_max_offset: exits with 1 and stdout matches expected.out,
link/std_bytes_read_be_negative: exits with 1 and stdout matches expected.out —
each the template link/${name}: exits with ${wantCode} and stdout matches expected.out;
link/std_bytes_copy_outside: every one of 50 runs at 2 and 8 threads prints expected.stderr once and exits 1 — the template link/${name}: every one of ${PANIC_ONCE_RUNS} runs at 2 and 8 threads prints expected.stderr once and exits ${wantCode};
simd: the freestanding --profile wasm link of nish/bytes's runs compiled ${kind} adds runtime-bytes.c once, and answers as Node does;
simd: the freestanding --profile wasm link adds runtime-bytes.c to no module that calls no run, and not again when the inputs name it;
N-API: a typed array longer than 2^31-1 is a TypeError, not an array whose length reads negative (BYTES-1);
std/crypto: no module's IR calls a variable-time scan kernel (14 modules; std_bytes_runs, the control, names 8) — the template ending
(${cryptoFiles.length} modules; std_bytes_runs, the control, names ${control.length}) (BYTES-2).
Divergences from Node
None found in the answers: under Node, the runs and copyBytes are the
TypeScript walkers in std/bytes.ts, which the kernels match on every input
the audit tried. How Node's toI32 reads a Uint8Array longer than 2^31-1
was not tested, so BYTES-1 is recorded for the native build only.

