nish/json

The value of a field, or of several, of one flat JSON object; and JsonWriter, which writes any JSON value into a byte buffer.

import {
  JSON_WRITER_MAX_DEPTH,
  JSON_WRITE_BAD_WINDOW,
  JSON_WRITE_DONE,
  JSON_WRITE_FULL,
  JSON_WRITE_KEY_EXPECTED,
  JSON_WRITE_NON_FINITE,
  JSON_WRITE_OK,
  JSON_WRITE_TOO_DEEP,
  JSON_WRITE_UNBALANCED,
  JSON_WRITE_VALUE_EXPECTED,
  JsonWriter,
  jsonField,
  jsonFields,
  jsonStringifyString,
} from "nish/json";
import { jsonField } from "nish/json";

for (const line of splitLines(stdout)) {
  if (!line.startsWith("{")) {
    continue;
  }
  const code = jsonField(line, "code");
  if (code !== null && code === "NL0003") {
    panic("the compiler crashed");
  }
}

The reader exists because the compiler's own machine-readable surface is JSON: under --json every diagnostic is one flat object on a line of stdout, whose code is a stable rule identifier (AGENTS.md, and docs/wp12-release.md for the exit-code bands). A Nish program that reads that surface — a wrapper, an editor plug-in, or tests/nish/cli.ts, which pins the contract — needs the value of a named field and nothing else, and the language has no JSON.parse to give it.

So the reading half is a reader, not a parser, and the difference is worth stating:

  • It answers text. A string field comes back with its escapes decoded and without its quotes; a number, a true or a null comes back as the bytes it was written with, for parseInt or === at the call site. A field whose value is null and one whose value is the string "null" therefore answer the same thing, which is the one ambiguity a program that needs to tell them apart cannot live with — and the sign that it wants a real parser rather than this.
  • It answers null for a field that is not there, so "absent" and "empty" stay apart the way they do in getenv.
  • It does not validate. A nested object or array is stepped over so that the fields after it are still reachable, and a truncated object still answers the fields that precede the truncation. A program that has to know whether the whole line was well-formed is asking a question this does not answer.

The reader is two exported functions, and few on purpose. A std/ module is compiled into the program that imports it and the symbol namespace is flat (std/README.md), so every name here is a name its importer cannot use: every helper is json-prefixed or a method of JsonWriter. Splitting a stream into lines and picking the ones that start with { is two calls the caller already has — splitLines from std/text and startsWith — and duplicating them here would cost more names than it saves.

jsonField scans from the start of the line on every call, so a caller that wants three fields of each line reads each line three times. jsonFields reads it once: jsonFields(line, ["code", "line", "message"]) answers an array whose slot k is exactly what jsonField(line, names[k]) answers — absent, first-of-a-duplicate and malformed input included — because the two share the step that reads one member (jsonReadMember) and stop at the same one. Its array goes back to the caller with the values in it, so a loop that calls it keeps its per-pass arena release, as one calling jsonField does; jsonFields' own comment says when that holds.

The writer

The writer is for the other direction: a server's response body (WP34 N11, whose first caller is a TechEmpower /json entry writing {"message":"Hello, World!"} on every request). JsonWriter writes into a u8[] the caller owns and reuses, puts in every comma and colon itself, and allocates nothing per call except an f64's digits, which it gives back before the call returns. What it writes is byte for byte what JSON.stringify writes for the same value: the seven short escapes and \u00xx below 0x20, / and 0x7f as themselves, integers as their digits, an f64 as String(v) spells it with -0 as 0, and NaN and the infinities as null. jsonStringifyString is the same string rule answered as a string.

Two modes

Two modes, both off by default and both kept across reset, so a server sets them once. setIndent(n) and setIndentTab() write what JSON.stringify(v, null, n) and JSON.stringify(v, null, "\t") write: one member or element a line, ": " after a key, an empty {} or [] on one line, and n clamped to 10. The indent is one byte stored as many times as the depth asks, so the indented text allocates no more than the minified one, and a raw fragment is copied as it is, not re-indented.

setStrict(true) makes NaN and the infinities, which JSON cannot spell, the sticky JSON_WRITE_NON_FINITE instead of null: for a caller to whom a null where a number was due would be a silently corrupt record.

Misuse is a sticky code

Misuse is a sticky code, as in nish/net/hpack: never a panic, and no Result to carry on the hot path. The first call that cannot be honoured sets failed to a negative JSON_WRITE_* code — a value where a key is due, a key where a value is due, an unbalanced or mismatched end*, nesting past JSON_WRITER_MAX_DEPTH, a buffer with no room, a second top-level value, a bad raw window or start, a non-finite f64 in strict mode — and every later call does nothing.

A call that fails writes nothing past buf.length (a full buffer is checked before the bytes go in, never by the bounds check), so the caller writes the whole value, reads failed once at the end, and grows the buffer and retries or answers a 500. complete() says whether one whole value was written with every container closed.

A string is its bytes

A string is its bytes (WP33 class C, docs/wp33-round-trip.md §3.2, the first row). The writer escapes by byte and copies every byte at or above 0x80, so UTF-8 text comes out as the UTF-8 that JSON.stringify's answer encodes to, and a string that is not UTF-8 — a lone 0xFF — comes out raw, where no Node string could hold it in the first place. Under Node the same source reads UTF-16 units, which agree on ASCII and only there; tests/link/std_json_writer pins both edges natively, and its node.ts is the ASCII half that tests/run.js holds to JSON.stringify under Node.

Every offset here is a byte offset and every width is spelled, with each length and byte a builtin answers read through toI32 — .length and charCodeAt answer number, which is f64 under --number-mode f64, so the conversion is what makes this one module in both modes rather than two (docs/wp26-stdlib.md §4).

Functions

jsonField

The value of name in object, or null when the object does not have that field.

A string value comes back unquoted and unescaped; any other value comes back as the bytes it was written with, so parseInt reads a number and === "true" reads a boolean. The scan is left to right and answers the first field of that name, which is what a duplicated key means to every JSON reader that does not build a map.

A nested value is stepped over rather than searched, so name is a field of this object and not a path into it: that is the "flat" in the module header, and it is the shape the compiler's --json line has.

const jsonField: (
  object: string,
  name: string,
) => string | null
objectstringrequired

the text of one JSON object.

namestringrequired

the field to look for, matched against each key's decoded text.

returnsstring | null

the first such field's value, or null when there is none.

jsonFields

The value of every one of names in object, in the order of names: slot k is exactly what jsonField(object, names[k]) answers.

It is for a caller that wants several fields of one line — a diagnostic's code, line and message — and would otherwise scan the line once per field. This scans it once, left to right, and stops as soon as every name has answered: a speed path and nothing more, since a slot is filled only once and reading on could change no answer. Each slot keeps the first field of its name, as jsonField does; a name asked for twice answers in both slots; and a malformed object answers in each slot what jsonField answers for that name, because both stop at the same member.

The answer is a fresh array and the only allocation besides the values. Each value is stored into it by a plain values[k] = text statement, and the array is only stored into, tested and returned, so the values travel with the array: the call lets nothing else out (LANGUAGE.md, "Memory model", a value stored into an array that is returned with it). A loop calling this keeps its per-pass release on a line older than the pass, as jsonField's does, and a using a = arena() block may hold the call.

On a line built in the pass neither reader keeps the release (LANGUAGE.md, "Memory model"). Reading values back, passing it to a function, or storing it anywhere before the return would undo this; tests/link/std_json pins the flat arena and the using block.

const jsonFields: (
  object: string,
  names: string[],
) => (string | null)[]
objectstringrequired

the text of one JSON object.

namesstring[]required

the fields to look for.

returns(string | null)[]

one value per name, in the order of names, each null when that field is missing.

jsonStringifyString

s as JSON.stringify(s) writes it — quoted, with the escapes JsonWriter.string writes — as a string. A thin layer on the writer: it sizes a buffer to fit exactly, writes into it and reads it back. A string with nothing to escape skips the writer and is only quoted.

const jsonStringifyString: (s: string) => string
sstringrequired

the text to quote.

returnsstring

the quoted, escaped text.

Classes

JsonWriter

A JSON text written into a byte buffer the caller owns: buf[start ..), up to buf.length, with end the cursor. Every comma and colon is the writer's, so the caller says only what the value is:

const w = new JsonWriter(out, 0)
w.beginObject()
w.key("message")
w.string("Hello, World!")
w.endObject()
if (w.complete()) {
  // out[0 .. w.end) is {"message":"Hello, World!"}
}

No call allocates except f64 (whose digits are a string the call releases before it returns) and text: a writer reset onto the same buffer for each request keeps Arena.used() flat, which tests/link/std_json_writer measures over 10,000 writes.

Misuse is a sticky code, not a panic and not a Result (the module header has the policy). Once failed is negative every later call does nothing, so a caller writes the whole value and reads failed once at the end.

class JsonWriter
Member
new JsonWriter(buf: u8[], start: i32) A writer that starts its text at buf[start ..), as reset does.
buf: u8[] The caller's buffer the text is written into; the writer never grows it.
depth: i32 How many containers are open.
end: i32 The cursor: the bytes written are buf[start .. end).
failed: i32 JSON_WRITE_OK, or the first JSON_WRITE_* code a call hit.
indentWidth: i32 How many indentBytes each level of nesting puts at the start of a line, or 0 for the minified text. A mode, not a part of the text: reset keeps it, so a writer set up once writes every text the same way.
strict: boolean Whether a NaN or infinite f64 is JSON_WRITE_NON_FINITE rather than null. Kept by reset.
beginArray(): void Opens an array where a value is due, with the same errors as beginObject.
beginObject(): void Opens an object where a value is due. JSON_WRITE_KEY_EXPECTED inside an object before its key, JSON_WRITE_TOO_DEEP past JSON_WRITER_MAX_DEPTH open containers, JSON_WRITE_DONE after the top-level value, and JSON_WRITE_FULL when buf has no room.
boolean(v: boolean): void true or false.
complete(): boolean Whether one whole value has been written, every container closed, and nothing failed.
endArray(): void Closes the innermost array. JSON_WRITE_UNBALANCED when that is not an array, or nothing is open.
endObject(): void Closes the innermost object. JSON_WRITE_UNBALANCED when that is not an object, or nothing is open, and JSON_WRITE_VALUE_EXPECTED when a key is still waiting for its value.
f64(v: f64): void v as String(v) spells it, which is JSON.stringify's rule: the shortest digits that read back as v, -0 as 0, and null for NaN and both infinities, which JSON cannot spell — or, once setStrict(true), JSON_WRITE_NON_FINITE for those three, with nothing written.
i32(v: i32): void v in decimal, as String(v) spells it.
i64(v: i64): void The digits of v, the minimum included: the arithmetic stays on the negative side, where it fits.
key(name: string): void The next member's key, with the comma before it and the colon after it.
keyBytes(bytes: u8[], off: i32, len: i32): void key for a name held as bytes, bytes[off .. off + len), escaped exactly as key escapes a string of those bytes, and allocating nothing: what a caller that keeps its text in a byte buffer, such as nish/json-value's JsonDocument, writes without making a string per key. A window outside bytes is JSON_WRITE_BAD_WINDOW, checked once, before anything is written.
nullValue(): void null. The method is not named null, which the parser refuses as a member name.
raw(bytes: u8[], off: i32, len: i32): void bytes[off .. off + len), copied as it is and counted as one value: a fragment the caller encoded already, such as a cached object. Nothing in it is checked, so a fragment that is not one whole JSON value makes a text that is not JSON either. The bytes are copied in one move (copyBytes), as if all were read before any was written, so a window of the writer's own buffer comes out as it was. In an indented text the fragment goes in where a value would, after its line break and indent, but its own bytes are not re-indented: an object fragment stays on one line, where JSON.stringify would spread it out.
reset(buf: u8[], start: i32): void Starts a new text at buf[start ..), forgetting everything the last one wrote and every error it hit: what a server calls once per response, on the buffer it reuses. A start outside buf is JSON_WRITE_BAD_WINDOW.
setIndent(spaces: i32): void Indents the text by spaces spaces a level, as JSON.stringify(v, null, spaces) does: each member and element on a line of its own, one space after a key's colon, and an empty {} or [] left on one line. spaces is clamped to 10 as JSON.stringify clamps it, and below 1 is the minified text, the default. A mode that reset keeps; set it before the text, since a change in the middle of one applies from the next byte.
setIndentTab(): void Indents the text by one tab a level, as JSON.stringify(v, null, "\t") does.
setStrict(strict: boolean): void Whether an f64 that is NaN or an infinity fails the text with JSON_WRITE_NON_FINITE, writing nothing, rather than being written as the null JSON.stringify writes. Off by default; a mode that reset keeps. Strict is for a caller to whom a null where a number was due is a corrupt record rather than a missing one.
string(s: string): void s quoted and escaped as JSON.stringify escapes it; every other byte goes through as itself.
stringBytes(bytes: u8[], off: i32, len: i32): void string for text held as bytes, bytes[off .. off + len): the same quotes and escapes, byte for byte, and no allocation. A window outside bytes is JSON_WRITE_BAD_WINDOW, checked once, before anything is written.
text(): string The bytes written so far, buf[start .. end), as a string. A convenience that allocates a byte at a time, for a caller that wants text rather than bytes; a server sends buf itself.
u32(v: u32): void One past i32's range is ten digits, so the last digit is split off in u32 and the nine before it are an i32's.
written(): i32 How many bytes have been written since reset.

Limits

64

JSON_WRITER_MAX_DEPTH: i32

How deep a writer nests objects and arrays. The kind of each open container is one bit of two i32 fields inside the writer, so a nesting past it is JSON_WRITE_TOO_DEEP rather than a stack that grows.

Constants

-7

JSON_WRITE_BAD_WINDOW: i32

A raw window outside its array, or a start outside the writer's buffer.

-6

JSON_WRITE_DONE: i32

A second value at the top level, after the first was written whole.

-5

JSON_WRITE_FULL: i32

A write that does not fit between the cursor and buf.length; nothing past buf.length is touched.

-1

JSON_WRITE_KEY_EXPECTED: i32

A value, an opener or a raw fragment, written in an object where a key is due.

-8

JSON_WRITE_NON_FINITE: i32

An f64 that is NaN or an infinity, written by a writer setStrict(true) made strict.

0

JSON_WRITE_OK: i32

JsonWriter.failed while every call so far has been a well-formed write that fitted.

-4

JSON_WRITE_TOO_DEEP: i32

A beginObject or beginArray that would nest deeper than JSON_WRITER_MAX_DEPTH.

-3

JSON_WRITE_UNBALANCED: i32

endObject or endArray with no container open, or closing the other kind.

-2

JSON_WRITE_VALUE_EXPECTED: i32

key outside an object or straight after another key, or endObject straight after a key.