nish/bytes
Fixed-width integers and doubles read from, and written into, a byte array at an offset, in either byte order.
import {
copyBytes,
getF64BE,
getF64LE,
getI32BE,
getI32LE,
getI64BE,
getI64LE,
getU16BE,
getU16LE,
getU32BE,
getU32LE,
getU64BE,
getU64LE,
setF64BE,
setF64LE,
setI32BE,
setI32LE,
setI64BE,
setI64LE,
setU16BE,
setU16LE,
setU32BE,
setU32LE,
setU64BE,
setU64LE,
skipBlank,
skipDigits,
skipJsonPlain,
skipJsonText,
} from "nish/bytes";
import { getU32BE, setU16LE } from "nish/bytes";
export const main = (): i32 => {
const header: u8[] = [0, 0, 1, 0, 0, 0];
const length = getU32BE(header, 0); // 256
setU16LE(header, 4, toU16(513)); // header[4] = 1, header[5] = 2
return toI32(length);
};
copyBytes
count bytes of src from from written over dst from at, as one copy: where dst and src are one array and the two ranges overlap, the bytes land as if every one were read before any was written, as copyWithin promises.
getF64BE
The f64 whose IEEE 754 bits are getU64BE's.
getF64LE
The f64 whose IEEE 754 bits are getU64LE's.
getI32BE
getU32BE's bits as a two's-complement i32.
getI32LE
getU32LE's bits as a two's-complement i32.
getI64BE
getU64BE's bits as a two's-complement i64.
getI64LE
getU64LE's bits as a two's-complement i64.
getU16BE
The u16 whose high byte is b[off] and low byte b[off + 1].
getU16LE
The u16 whose low byte is b[off] and high byte b[off + 1].
getU32BE
The u32 whose bytes, most significant first, are b[off] to b[off + 3].
getU32LE
The u32 whose bytes, least significant first, are b[off] to b[off + 3].
getU64BE
The u64 whose bytes, most significant first, are b[off] to b[off + 7].
getU64LE
The u64 whose bytes, least significant first, are b[off] to b[off + 7].
setF64BE
Writes v's IEEE 754 bits as setU64BE does.
setF64LE
Writes v's IEEE 754 bits as setU64LE does.
setI32BE
Writes v's two's-complement bits as setU32BE does.
setI32LE
Writes v's two's-complement bits as setU32LE does.
setI64BE
Writes v's two's-complement bits as setU64BE does.
setI64LE
Writes v's two's-complement bits as setU64LE does.
setU16BE
Writes v into b[off] and b[off + 1], high byte first.
setU16LE
Writes v into b[off] and b[off + 1], low byte first.
setU32BE
Writes v into b[off] to b[off + 3], most significant byte first.
setU32LE
Writes v into b[off] to b[off + 3], least significant byte first.
setU64BE
Writes v into b[off] to b[off + 7], most significant byte first.
setU64LE
Writes v into b[off] to b[off + 7], least significant byte first.
skipBlank
The first index in [from, to) whose byte is not one of JSON's blanks — space, tab, newline, carriage return — or to when every byte there is one.
skipDigits
skipBlank for the ASCII digits 0 to 9: the same clamps, the same kernel family, nish_bytes_skip_digits.
skipJsonPlain
The first index in [from, to) whose byte a JSON writer cannot copy as itself — a control byte below 0x20, a quote or a backslash — or to, with the range clamped as skipBlank's is.
skipJsonText
skipBlank for the bytes a JSON string holds as themselves: printable ASCII other than " and \, and every well-formed UTF-8 sequence (RFC 3629 §4) that ends before to.
A wire format, a file header or a hash's message schedule is a u8[] with
numbers laid out in it, and every reader of one writes the same four-line
shift-and-or. This module is that loop written once:
Every function takes the array and the offset of its first byte; the LE
name puts the least significant byte there, the BE name the most
significant. The widths are u16, u32 and u64, the signed i32 and
i64, which are the same bytes read as two's complement, and f64, whose
bytes are its IEEE 754 bits (f64ToBits). There is no f32: the language
has no bit cast for one.
Each access is an ordinary index
Each access is an ordinary index, so an offset whose bytes do not all lie
in the array panics index, naming the first byte that does not.
A guard in the caller does not yet remove these checks: in a loop guarded by
off + 3 < b.length, the checker proves an index it can see but not one
inside a call, and after getU32LE inlines, LLVM keeps the checks on off + 1 to off + 3, because the guard compares the length truncated to i32 and
the checks compare it unsigned at 64 bits. A loop that needs them gone
indexes the bytes itself.
getU64BE, and with it getI64BE and getF64BE, is the exception: it
checks the whole range once and reads the eight bytes unchecked, which lowers
to one load and a byte swap. An offset outside the range still goes byte by
byte, so it panics as the others do.
The in-range answers are byte for byte those of the copies std/ keeps for
itself: sha256LoadWord (std/crypto/sha256.ts) for getU32BE,
chacha20Word and poly1305Le64 (std/crypto/chacha20poly1305.ts) for
getU32LE and getU64LE, loadWord and storeWord (std/crypto/sha512.ts)
for getU64BE and setU64BE, http2FramePut32 (std/net/http2-frame.ts)
for setU32BE, and tlsPutU16 (std/net/tls/codec.ts) for setU16BE. None
of them is replaced by this module. Out of range they differ: the AES and
TLS word helpers read zero past the end and drop bytes written there, which
their callers rely on, where this module panics.
Under Node (docs/RUN_UNDER_NODE.md) the u16 and u32 functions are
expected to answer as they do natively, because the top byte of a u32 is
multiplied in rather than shifted and each byte written is masked before
its toU8: there << turns bit 31 into a sign and the prelude's toU8
and toU32 saturate, and natively the multiply and the mask fold into the
shift and the truncation. The others do not yet answer there: the i32
functions cross between signed and unsigned with toI32 and toU32, which
the prelude saturates too (setI32LE(b, 0, -2) writes zeros), and the
64-bit functions and f64 meet its BigInt u64 and i64, which toU8
refuses.
The module also has the runs a parser walks and the copy it makes of
them.
skipBlank, skipDigits, skipJsonText and skipJsonPlain each answer
the first index in a range whose byte is not in their class — JSON's blanks,
the ASCII digits, the bytes a JSON string holds as themselves, the bytes a
JSON writer copies without an escape — and copyBytes copies a
window of one array into another, or into itself, as one move:
import { skipBlank, skipDigits } from "nish/bytes";
export const main = (): i32 => {
const text: u8[] = [32, 32, 52, 50, 44]; // " 42,"
const start = skipBlank(text, 0, 5); // 2
const end = skipDigits(text, start, 5); // 4
return end - start;
};
The four runs clamp their range and never panic; copyBytes panics on a
range outside its array. Natively each is a runtime kernel
(docs/LANGUAGE.md, "Byte runs"), and the runs are variable-time: they are
for text, never for secret data.
Functions
copyBytes
count bytes of src from from written over dst from at, as one
copy: where dst and src are one array and the two ranges overlap, the
bytes land as if every one were read before any was written, as
copyWithin promises. Where dst.set(src, at) copies all of src, this
copies a window of it, and allocates nothing.
A negative count, or a range that does not lie inside its array —
0 <= from <= from + count <= src.length and the same for at in dst —
panics copyBytes before any byte is written. Natively the copy is one
llvm.memmove.
const copyBytes: (
dst: u8[],
at: i32,
src: readonly u8[],
from: i32,
count: i32,
) => void
dstu8[]requiredThe array written into; only dst[at..at+count) changes.
ati32requiredWhere in dst the first copied byte goes.
srcreadonly u8[]requiredThe array copied from, which may be dst itself.
fromi32requiredWhere in src the copied bytes start.
counti32requiredHow many bytes to copy; zero copies nothing.
getF64BE
The f64 whose IEEE 754 bits are getU64BE's.
const getF64BE: (b: u8[], off: i32) => f64
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
returnsf64The double whose bits the eight bytes are.
getF64LE
The f64 whose IEEE 754 bits are getU64LE's.
const getF64LE: (b: u8[], off: i32) => f64
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
returnsf64The double whose bits the eight bytes are.
getI32BE
getU32BE's bits as a two's-complement i32.
const getI32BE: (b: u8[], off: i32) => i32
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
returnsi32The number read, from -2^31 to 2^31 - 1.
getI32LE
getU32LE's bits as a two's-complement i32.
const getI32LE: (b: u8[], off: i32) => i32
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
returnsi32The number read, from -2^31 to 2^31 - 1.
getI64BE
getU64BE's bits as a two's-complement i64.
const getI64BE: (b: u8[], off: i32) => i64
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
returnsi64The number read, from -2^63 to 2^63 - 1.
getI64LE
getU64LE's bits as a two's-complement i64.
const getI64LE: (b: u8[], off: i32) => i64
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
returnsi64The number read, from -2^63 to 2^63 - 1.
getU16BE
The u16 whose high byte is b[off] and low byte b[off + 1].
const getU16BE: (b: u8[], off: i32) => u16
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 1], and a byte of those outside b panics index.
returnsu16The number read.
getU16LE
The u16 whose low byte is b[off] and high byte b[off + 1].
const getU16LE: (b: u8[], off: i32) => u16
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 1], and a byte of those outside b panics index.
returnsu16The number read.
getU32BE
The u32 whose bytes, most significant first, are b[off] to b[off + 3].
const getU32BE: (b: u8[], off: i32) => u32
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
returnsu32The number read.
getU32LE
The u32 whose bytes, least significant first, are b[off] to b[off + 3].
const getU32LE: (b: u8[], off: i32) => u32
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
returnsu32The number read.
getU64BE
The u64 whose bytes, most significant first, are b[off] to b[off + 7].
The fast path's ors are grouped in pairs, which LLVM folds into one load and
a bswap; a flat chain of eight ors stayed byte by byte. The byte loop is
its own function so its last load is not merged with the fast path's.
const getU64BE: (b: u8[], off: i32) => u64
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
returnsu64The number read.
getU64LE
The u64 whose bytes, least significant first, are b[off] to b[off + 7].
const getU64LE: (b: u8[], off: i32) => u64
bu8[]requiredThe bytes the number is read from; they are not changed.
offi32required
Where in b its first byte is; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
returnsu64The number read.
setF64BE
Writes v's IEEE 754 bits as setU64BE does.
const setF64BE: (b: u8[], off: i32, v: f64) => void
bu8[]requiredThe bytes written into; only the eight at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
vf64requiredThe double to store, as its IEEE 754 bits.
setF64LE
Writes v's IEEE 754 bits as setU64LE does.
const setF64LE: (b: u8[], off: i32, v: f64) => void
bu8[]requiredThe bytes written into; only the eight at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
vf64requiredThe double to store, as its IEEE 754 bits.
setI32BE
Writes v's two's-complement bits as setU32BE does.
const setI32BE: (b: u8[], off: i32, v: i32) => void
bu8[]requiredThe bytes written into; only the four at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
vi32requiredThe number to store; a negative one is stored as its two's complement.
setI32LE
Writes v's two's-complement bits as setU32LE does.
const setI32LE: (b: u8[], off: i32, v: i32) => void
bu8[]requiredThe bytes written into; only the four at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
vi32requiredThe number to store; a negative one is stored as its two's complement.
setI64BE
Writes v's two's-complement bits as setU64BE does.
const setI64BE: (b: u8[], off: i32, v: i64) => void
bu8[]requiredThe bytes written into; only the eight at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
vi64requiredThe number to store; a negative one is stored as its two's complement.
setI64LE
Writes v's two's-complement bits as setU64LE does.
const setI64LE: (b: u8[], off: i32, v: i64) => void
bu8[]requiredThe bytes written into; only the eight at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
vi64requiredThe number to store; a negative one is stored as its two's complement.
setU16BE
Writes v into b[off] and b[off + 1], high byte first.
const setU16BE: (b: u8[], off: i32, v: u16) => void
bu8[]requiredThe bytes written into; only the two at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 1], and a byte of those outside b panics index.
vu16requiredThe number to store.
setU16LE
Writes v into b[off] and b[off + 1], low byte first.
const setU16LE: (b: u8[], off: i32, v: u16) => void
bu8[]requiredThe bytes written into; only the two at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 1], and a byte of those outside b panics index.
vu16requiredThe number to store.
setU32BE
Writes v into b[off] to b[off + 3], most significant byte first.
const setU32BE: (b: u8[], off: i32, v: u32) => void
bu8[]requiredThe bytes written into; only the four at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
vu32requiredThe number to store.
setU32LE
Writes v into b[off] to b[off + 3], least significant byte first.
const setU32LE: (b: u8[], off: i32, v: u32) => void
bu8[]requiredThe bytes written into; only the four at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 3], and a byte of those outside b panics index.
vu32requiredThe number to store.
setU64BE
Writes v into b[off] to b[off + 7], most significant byte first.
const setU64BE: (b: u8[], off: i32, v: u64) => void
bu8[]requiredThe bytes written into; only the eight at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
vu64requiredThe number to store.
setU64LE
Writes v into b[off] to b[off + 7], least significant byte first.
const setU64LE: (b: u8[], off: i32, v: u64) => void
bu8[]requiredThe bytes written into; only the eight at off change.
offi32required
Where in b the first byte goes; it covers b[off] to
b[off + 7], and a byte of those outside b panics index.
vu64requiredThe number to store.
skipBlank
The first index in [from, to) whose byte is not one of JSON's blanks —
space, tab, newline, carriage return — or to when every byte there is
one. to is clamped to [0, b.length] and from to [0, to], as
indexOfAny clamps, so no range panics: one past the end answers the end.
Natively the scan is the runtime's nish_bytes_skip_blank, which reads 16
or 32 bytes at a time (docs/LANGUAGE.md, "Byte runs"). It is
variable-time: how long it takes says how long the run is, so it is
never for secret data.
const skipBlank: (b: u8[], from: i32, to: i32) => i32
bu8[]requiredThe bytes scanned; they are not changed.
fromi32requiredWhere the scan starts, clamped into [0, to].
toi32requiredWhere the scan stops, exclusive, clamped into [0, b.length].
returnsi32
The index of the first byte in the clamped range that is not a
blank, or
the clamped to when there is none.
skipDigits
skipBlank for the ASCII digits 0 to 9: the same clamps, the same
kernel family, nish_bytes_skip_digits.
const skipDigits: (b: u8[], from: i32, to: i32) => i32
bu8[]requiredThe bytes scanned; they are not changed.
fromi32requiredWhere the scan starts, clamped into [0, to].
toi32requiredWhere the scan stops, exclusive, clamped into [0, b.length].
returnsi32
The index of the first byte in the clamped range that is not a
digit, or
the clamped to when there is none.
skipJsonPlain
The first index in [from, to) whose byte a JSON writer cannot copy as
itself — a control byte below 0x20, a quote or a backslash — or to, with
the range clamped as skipBlank's is. Every byte from 0x20 up is in the
run, 0x7f and the bytes from 0x80 up included, and no UTF-8 is checked:
where skipJsonText is the reader's run, this is the writer's, which passes
the bytes of a string through as they are. Natively the scan is
nish_bytes_skip_json_plain, 16 bytes at a time. Variable-time, as
skipBlank is.
const skipJsonPlain: (b: u8[], from: i32, to: i32) => i32
bu8[]requiredThe bytes scanned; they are not changed.
fromi32requiredWhere the scan starts, clamped into [0, to].
toi32requiredWhere the scan stops, exclusive, clamped into [0, b.length].
returnsi32
The index of the first byte in the clamped range that a JSON
writer must escape, or
the clamped to when there is none.
skipJsonText
skipBlank for the bytes a JSON string holds as themselves: printable ASCII
other than " and \, and every well-formed UTF-8 sequence (RFC 3629 §4)
that ends before to. The run stops at the first quote, backslash or control
byte, and at the lead byte of a sequence that is malformed — an overlong
form, a surrogate, a code point past U+10FFFF, a stray continuation byte — or
cut short by to.
So a whole JSON string body with no escape is one call, and where it stops is
the byte a decoder has to look at. Natively the scan is
nish_bytes_skip_json_text: blocks of ASCII 16 or 32 bytes at a time, each
multi-byte sequence a byte at a time. Variable-time, as skipBlank is.
const skipJsonText: (b: u8[], from: i32, to: i32) => i32
bu8[]requiredThe bytes scanned; they are not changed.
fromi32requiredWhere the scan starts, clamped into [0, to].
toi32requiredWhere the scan stops, exclusive, clamped into [0, b.length].
returnsi32
The index of the first byte in the clamped range that is not in
the run, or
the clamped to when there is none.

