nish/json-value
Any JSON document as a tree a program can read, build and write back, with every number kept as the text it was written in.
import {
JSON_VALUE_ARRAY,
JSON_VALUE_BOOLEAN,
JSON_VALUE_DEFAULT_MAX_VALUES,
JSON_VALUE_DUPLICATE_LAST,
JSON_VALUE_DUPLICATE_REFUSE,
JSON_VALUE_ERR_ATTACHED,
JSON_VALUE_ERR_DUPLICATE,
JSON_VALUE_ERR_HANDLE,
JSON_VALUE_ERR_NUMBER,
JSON_VALUE_ERR_RANGE,
JSON_VALUE_ERR_VALUES,
JSON_VALUE_MAX_VALUES,
JSON_VALUE_NONE,
JSON_VALUE_NULL,
JSON_VALUE_NUMBER,
JSON_VALUE_OBJECT,
JSON_VALUE_OK,
JSON_VALUE_STRING,
JsonDocument,
JsonValue,
} from "nish/json-value";
import { JsonDocument, JSON_VALUE_NUMBER } from "nish/json-value"
import { JsonLimits } from "nish/json-tokenizer"
const doc = new JsonDocument(new JsonLimits())
const root = doc.parseText('{"id":18446744073709551615,"tags":["a","b"]}')
if (root < 0) {
// a JSON_TOK_ERR_* code, or JSON_VALUE_ERR_DUPLICATE; the byte is doc.errorAt
}
const id = doc.get(root, "id")
doc.kind(id) === JSON_VALUE_NUMBER // true
doc.isU64(id) // true; doc.isI64(id) is false
doc.numberText(id) // "18446744073709551615", as written
doc.stringify(root) // the same text back, byte for byte
nish/json-typed reads JSON into a class the program declares, which is the
fast and safe way when the shape is known. This module is for when it is
not: a configuration file, a proxy that rewrites one field and passes the
rest through, a tool that prints whatever it was given.
A value is a handle
JsonValue is an i32 naming one value of the
document that made it, and every reader and builder is a method of that
document. A handle stays valid until the document parses again or is
cleared, which forgets every value at once and keeps the memory: the
nodes, the member slots and the bytes of every string and number live in a
handful of arrays the document owns and reuses. A server keeps one document
and parses each request into it, and once the arrays have grown to the
largest request seen, parsing allocates nothing.
A handle carries the generation of the document's contents it was made in,
which moves on at every parse and clear, so a handle kept from an earlier
request names nothing in the next one, even when the next one has the same
shape: it is JSON_VALUE_NONE to kind, and every reader answers its empty
value for it rather than another document's.
The generation has 511 values and moves by one at a clear, one at a parse
and two at a parse that fails, so the check misses only a handle kept across
exactly a multiple of 511 of those steps; a program must not keep handles
across documents, and this catches the mistake rather than licensing it. A
negative handle, or one past the last value, is JSON_VALUE_NONE too.
The document's size is bounded
Each value costs about 46 bytes of the
arrays the document keeps for its life, 66 after a parseRanged, on top of
the decoded text of its
strings, keys and numbers, which JsonLimits.maxTotalBytes bounds through
the input (docs/security/json.md, JSON-10). maxValues caps the first
part by capping how many values one document holds:
JSON_VALUE_DEFAULT_MAX_VALUES unless the program sets it, at most
JSON_VALUE_MAX_VALUES. A parse that would make one more is
JSON_VALUE_ERR_VALUES at the byte of the value that does not fit, and a
builder answers the same code.
Numbers are lossless
A number keeps its source text, and is read as
whatever the caller asks for when it asks: i64 and u64 when the text is
an integer that fits (isI64, isU64), whether its value is whole at all
(isIntegral, so 1.0 and 1e2 are), the nearest f64 (f64, correctly
rounded, the tokenizer's numberF64), or the text itself (numberText).
Writing it back copies the text. So 18446744073709551616, wider than any
integer type, and 0.1000000000000000055511151231257827, more digits than an
f64 holds, come back out exactly as they went in, where JavaScript's
JSON.parse turns both into the nearest double.
Source ranges are opt-in
parseRanged and parseTextRanged parse as
parse and parseText do, and every value and every member key also keeps
the range of bytes it was read from: start(v) and end(v) bound a value,
keyStart(o, i) and keyEnd(o, i) the key of member i, its quotes
included. They are offsets into the bytes parseRanged was given, or
parseTextRanged's string, so a line index over that text (YamlLineIndex
is one) turns them into a line and a column.
Under Node, where parseTextRanged takes one byte per UTF-16 code unit
(below), its offsets count code units, so past the first non-ASCII character
they differ from a native build's, which count UTF-8 bytes. Ranges cost about
20 bytes per value more: two offsets for the value, two for its key and about
one on the parse's stacks, which grow with the nesting and the members
pending. They are entry points of their own rather than a flag parse tests,
so that parse keeps the loop it had before ranges existed.
A plain parse writes nothing for them: ranges carry the generation they
were written in, which clear moves anyway, so after a parse every reader
answers -1, and their arrays are made on first use, so a document that
never keeps ranges allocates none. A loader that builds a document from
another text clears, then gives each value it makes its range with
setRange and each key with setKeyRange; until then a built value or key
answers -1. set over an existing key keeps the member's key range and
takes the new value's own.
Under JSON_VALUE_DUPLICATE_LAST a repeated key's member keeps its first
key's range and holds the last value, with that value's.
Objects keep insertion order
Objects keep insertion order, the order the members were written in or
added. get finds a member by its key with a scan, so it is for a handful of
lookups per object; a program that looks up many keys of a large object
walks it once with keyAt and at.
Duplicate keys are refused by default. {"a":1,"a":2} is
JSON_VALUE_ERR_DUPLICATE, with errorAt at the first key that repeats an
earlier one, because two readers that pick different members of the same
text are a check bypassed (docs/security/json.md, JSON-3), and refusing
is what nish/json-typed does too. A program that has to accept what
JSON.parse accepts sets duplicates to JSON_VALUE_DUPLICATE_LAST, and
gets JSON.parse's answer: one member, at the place of the first, holding
the value of the last. The check sorts each object's keys, so an object of
n members costs n log n comparisons whatever the keys are.
Every limit and every parse error is the tokenizer's. parse runs a
JsonTokenizer with the JsonLimits the document was made with, and
answers its JSON_TOK_ERR_* code and offset unchanged. The depth limit is
capped at JSON_WRITER_MAX_DEPTH, 64, so that every document a parse
accepts can be written back; a deeper one is the tokenizer's
JSON_TOK_ERR_DEPTH. Nothing an input holds panics, and the parse keeps its
own stack of open containers rather than recursing.
Writing goes through `JsonWriter`
write(v, w) writes a value into a
writer the caller set up, so its buffer, its reuse and its sticky
JSON_WRITE_* codes are the writer's own, and stringify is a convenience
that sizes a buffer and answers a string. Strings come out as JsonWriter
escapes them, which is JSON.stringify's rule, numbers as their text, and
members in order. Only a built tree can be deeper than
JSON_WRITER_MAX_DEPTH, and writing one is the writer's
JSON_WRITE_TOO_DEEP.
Building a tree
Building a tree is a new* call per value, then push onto an array or
set on an object, each answering JSON_VALUE_OK or a negative code. A
value lives in at most one container, so a tree built through push and
set can never contain itself: adding a value that is already in a
container, or a container to itself or to anything inside it, is
JSON_VALUE_ERR_ATTACHED, and writing always ends.
Strings handed to a builder are taken as their bytes, as JsonWriter takes
them, and a string read back is made from the stored bytes in one allocation
(String.fromBytes). Under Node a string is UTF-16, and parseText and the
builders take one byte per code unit, so non-ASCII text is stored differently
there than natively, where a string is its UTF-8 bytes; parse over a u8[]
reads the same bytes both ways.
Public is not the whole API
The compiler ignores private, so every
member is reachable by a program; private and @internal mark what is
not API, and the guarantees above hold only through the members that are
not marked so. append with indices of the program's choosing, for
example, skips the check that keeps a tree finite.
Every name here is json-prefixed, because a std/ module is compiled into
the program that imports it and the symbol namespace is flat
(std/README.md). Every offset, count and code is an i32, so the module is
the same under --number-mode f64, and it runs under the Node prelude, where
tests/run.js holds every method to the native answers.
Classes
JsonDocument
A JSON tree, and the memory every value of it lives in. Make one, then
parse into it or build with the new* methods; the module header has the
rules. Its private and @internal members are reachable, as every
member in Nish is, because the compiler ignores private; use the others.
class JsonDocument
| Member | |
|---|---|
new JsonDocument(limits: JsonLimits) |
An empty document whose parses are held to limits, with maxDepth capped at JSON_WRITER_MAX_DEPTH so that every document a parse accepts can be written back. maxValues starts at JSON_VALUE_DEFAULT_MAX_VALUES. |
byteTop: i32 |
How many bytes of the decoded-text store are in use. |
duplicates: i32 |
JSON_VALUE_DUPLICATE_REFUSE or JSON_VALUE_DUPLICATE_LAST; read by parse. |
errorAt: i32 |
The byte offset in the parsed array the error was found at, and -1 until one. |
failed: i32 |
The last parse's code: 0, a JSON_TOK_ERR_* code or JSON_VALUE_ERR_DUPLICATE. |
generation: i32 |
The generation every handle of the current document carries; see JSON_VALUE_INDEX_BITS. |
maxValues: i32 |
The most values the document holds; more is JSON_VALUE_ERR_VALUES. At most JSON_VALUE_MAX_VALUES. |
rangeGeneration: i32 |
The generation the ranges were written in; 0, never a generation, when none were. |
ranges: JsonValueRanges | null |
The source ranges, null until the first parseRanged or setter, so that a document that never keeps ranges allocates nothing for them. They are current only while rangeGeneration is generation, and then only for the first rangeTop values and slotRangeTop slots: past them is an earlier document's, and reads as -1, so that a value or member a builder adds writes nothing here. Validity rides on the generation, which clear moves anyway, so that a plain parse writes nothing for ranges. |
slotTop: i32 |
How many member slots are in use, and how many bytes. |
top: i32 |
How many values are in use: the handles are 0 .. top - 1. |
at(v: JsonValue, i: i32): JsonValue |
Element i of an array, or the value of member i of an object, in order; -1 when there is none. |
boolean(v: JsonValue): boolean |
A boolean's value; false for any other value. |
clear(): void |
Forgets every value, keeping the memory for the next parse or build. Every handle is invalid afterwards. |
end(v: JsonValue): i32 |
One past the offset of v's last byte: v is src[start(v), end(v)). |
f64(v: JsonValue): f64 |
A number's nearest f64, ties to even, as JSON.parse reads it; 0 for any other value. |
get(v: JsonValue, key: string): JsonValue |
The value of the member of object v whose key is key's bytes; -1 when there is none. |
i64(v: JsonValue): i64 |
A number's value as an i64 when isI64(v) holds, and 0 otherwise. |
isI64(v: JsonValue): boolean |
Whether v is a number written without a fraction or an exponent whose value fits an i64: the spelling, so 1.0 is not one. isIntegral(v) asks whether the value is whole. |
isIntegral(v: JsonValue): boolean |
Whether v is a number whose value is whole, however it is written: JSON Schema 2020-12's "integer", the tokenizer's isIntegral. 1.0, 1e2 and 1e400 are, where isI64(v) and isU64(v) go by spelling and by range and answer false for all three. false for any other kind and for a handle from an earlier parse. |
isU64(v: JsonValue): boolean |
Whether v is a number written without a sign, a fraction or an exponent whose value fits a u64: the spelling, as isI64(v) is. |
keyAt(v: JsonValue, i: i32): string |
The key of member i of an object, decoded; "" when v is not an object or has no member i. |
keyEnd(o: JsonValue, i: i32): i32 |
One past the offset of the last byte of the key of member i of object o, its closing quote. |
keyStart(o: JsonValue, i: i32): i32 |
The offset of the first byte of the key of member i of object o, its opening quote. |
kind(v: JsonValue): i32 |
v's JSON_VALUE_* kind, or JSON_VALUE_NONE when v names no value of this document's current contents. |
length(v: JsonValue): i32 |
How many elements an array has, or members an object has; -1 for any other value. |
newArray(): JsonValue |
A new empty array, which push fills. |
newBoolean(b: boolean): JsonValue |
A new boolean. |
newF64(x: f64): JsonValue |
A number holding x as String(x) spells it, the shortest digits that read back as x, with -0 as 0: what JsonWriter.f64 writes. JSON_VALUE_ERR_NUMBER for NaN and both infinities, which JSON cannot spell. |
newI64(x: i64): JsonValue |
A number holding x's digits. |
newNull(): JsonValue |
A new null. |
newNumber(text: string): JsonValue |
A number holding text exactly, which must be one JSON number and nothing else (RFC 8259 §6); JSON_VALUE_ERR_NUMBER otherwise. This is how a program writes a number no integer or float type holds. |
newObject(): JsonValue |
A new empty object, which set fills. |
newString(s: string): JsonValue |
A string holding s's bytes. |
newU64(x: u64): JsonValue |
A number holding x's digits. |
numberText(v: JsonValue): string |
A number's text, exactly as it was written or built; "" for any other value. |
parse(src: u8[], off: i32, len: i32): JsonValue |
Clears the document and parses src[off, off + len) into it. Answers the root value, or a negative code — the tokenizer's JSON_TOK_ERR_* or JSON_VALUE_ERR_DUPLICATE — which is also in failed, with the byte offset in errorAt. Nothing is kept from a parse that fails. Keeps no source ranges; parseRanged does.Keep in step with parseRanged, the same loop plus the ranges. |
parseRanged(src: u8[], off: i32, len: i32): JsonValue |
parse, and every value and member key keeps the range of bytes it was read from, for start, end, keyStart and keyEnd. The same answer, code and errorAt as parse on the same bytes.A loop of its own rather than a flag parse tests, so that parse runs the loop it ran before ranges existed: a dispatch inside parse let this loop inline there and cost the plain loop more instructions. A program that links both can still see the plain loop's code laid out differently (docs/BENCHMARKS.md). Keep in step with parse. |
parseText(s: string): JsonValue |
parse over the bytes of s, copied into an array the document reuses; errorAt is a byte offset in s. Keep in step with parseTextRanged. |
parseTextRanged(s: string): JsonValue |
parseRanged over the bytes of s, as parseText is parse's; its offsets are byte offsets in s. Keep in step with parseText. |
push(a: JsonValue, v: JsonValue): i32 |
Appends v to array a. JSON_VALUE_OK, or JSON_VALUE_ERR_HANDLE or JSON_VALUE_ERR_ATTACHED. |
set(o: JsonValue, key: string, v: JsonValue): i32 |
Sets member key of object o to v: in place, keeping its position, when o has the key, and appended last when it does not, as assigning a property of a JavaScript object does. The value it replaces leaves the object and is in no container afterwards. JSON_VALUE_OK, or JSON_VALUE_ERR_HANDLE or JSON_VALUE_ERR_ATTACHED. |
setKeyRange(o: JsonValue, i: i32, start: i32, end: i32): i32 |
Gives the key of member i of object o the range [start, end), as setRange does a value's. JSON_VALUE_ERR_HANDLE when o is not an object or has no member i. |
setRange(v: JsonValue, start: i32, end: i32): i32 |
Gives v the range [start, end), for a loader that builds a document from another text with the new* methods and keeps where each value came from. JSON_VALUE_OK, JSON_VALUE_ERR_HANDLE, or JSON_VALUE_ERR_RANGE when 0 <= start <= end does not hold. |
start(v: JsonValue): i32 |
The offset of v's first byte in the bytes the document parsed. |
string(v: JsonValue): string |
A string's decoded bytes as a string; "" for any other value. |
stringEquals(v: JsonValue, s: string): boolean |
Whether v is a string whose decoded bytes are s's. Allocates nothing. |
stringify(v: JsonValue): string |
v as minified JSON text: written into a buffer that doubles until it fits, then made into one string. "" when v names nothing or the writer refuses it, which only a built tree deeper than JSON_WRITER_MAX_DEPTH makes it do; write gives the code. |
u64(v: JsonValue): u64 |
A number's value as a u64 when isU64(v) holds, and 0 otherwise. |
write(v: JsonValue, w: JsonWriter): void |
Writes v into w as one value, members in order and numbers as their text; the outcome is the writer's, in w.failed. A value that names nothing writes nothing and fails nothing. A parsed tree is never deeper than the writer nests; a built one deeper than JSON_WRITER_MAX_DEPTH ends at JSON_WRITE_TOO_DEEP, where the recursion stops. |
Types
JsonValue
A value in a JsonDocument: an index into it, valid until the document parses again or is cleared.
type JsonValue = i32
Limits
262,144
JSON_VALUE_DEFAULT_MAX_VALUES: i32
The most values one document holds by default: a parse that would make
more is JSON_VALUE_ERR_VALUES. Each value costs the document about 46
bytes of arrays it keeps, about 66 after a parseRanged (docs/security/json.md,
JSON-10), so this caps them near 12 MB, or 17 MB; a program raises maxValues up to
JSON_VALUE_MAX_VALUES for larger documents.
4,194,304
JSON_VALUE_MAX_VALUES: i32
The most maxValues can be: a handle keeps a value's index in its low 22 bits.
Constants
5
JSON_VALUE_ARRAY: i32
An array: length and at read its elements, in order.
2
JSON_VALUE_BOOLEAN: i32
The kind of true and false, which boolean reads.
1
JSON_VALUE_DUPLICATE_LAST: i32
JSON.parse's rule: one member, at the first key's place, with the last key's value.
0
JSON_VALUE_DUPLICATE_REFUSE: i32
The default: a repeated key is JSON_VALUE_ERR_DUPLICATE.
-32
JSON_VALUE_ERR_ATTACHED: i32
A value already in a container, or a container added to itself or to a value inside it.
-30
JSON_VALUE_ERR_DUPLICATE: i32
A key an object already has, refused while duplicates is JSON_VALUE_DUPLICATE_REFUSE.
-31
JSON_VALUE_ERR_HANDLE: i32
A handle that names no value, or a value of the wrong kind: push onto an object, set on an array.
-33
JSON_VALUE_ERR_NUMBER: i32
Text that is not one JSON number, for newNumber, or an f64 that is NaN or an infinity, for newF64.
-35
JSON_VALUE_ERR_RANGE: i32
setRange or setKeyRange with a range that is not 0 <= start <= end.
-34
JSON_VALUE_ERR_VALUES: i32
One value more than JsonDocument.maxValues, by a parse (with errorAt at its first byte) or a builder.
0
JSON_VALUE_NONE: i32
What kind answers for a handle that names no value.
1
JSON_VALUE_NULL: i32
The kind of null.
3
JSON_VALUE_NUMBER: i32
A number, kept as its text: numberText, isI64, i64, isU64, u64 and f64 read it.
6
JSON_VALUE_OBJECT: i32
An object: length, keyAt, at and get read its members, in order.
0
JSON_VALUE_OK: i32
What a builder answers when it did what it was asked.
4
JSON_VALUE_STRING: i32
A string, decoded: string and stringEquals read it.

