Lexical rules

Nish source is TypeScript syntax, read by the compiler's own lexer and parser (src/lexer.ts, src/parser.ts), which follow TypeScript's rules for tokens, comments, and ASI; tests/lexer-oracle.js and tests/parser-oracle.js hold them to the typescript package's scanner and parser over the test corpus. Syntax errors are reported as syntax error: in the same file:line:col shape (tests/run.js, WP10 block).

  • Semicolons are optional, by TypeScript's rule. A statement may end without ; where TypeScript would insert one: at a line break, before a }, and at the end of the file. A ; written and a ; inserted parse to the same tree, so the IR is the same byte for byte (tests/cases/asi_statements). The rule is TypeScript's so that a program means what it means under Node, and that includes its three surprises (tests/cases/asi_continuation):

    • a line that starts with ( or [ continues the expression before it (f then (x) on the next line is a call);
    • a ++ or -- at the start of a line is a prefix operator of a new statement, not a postfix one of the line before;
    • return followed by a line break returns nothing. The value on the next line is then unreachable code, which is an error here (Unreachable code after return, tests/cases/reject_asi_return_newline), where TypeScript would run it silently as a separate statement.

    Nothing is inserted between two statements on one line (expected `;`, found `const`, tests/cases/reject_asi_same_line) or in a for header (reject_asi_for_header). A template at the start of a line is not inserted before either, because TypeScript reads it as a tagged template of the line before (reject_asi_template).

  • Encoding. Source is UTF-8. String contents are stored as UTF-8 bytes (tests/cases/str_escape).

  • A shebang line. A #! line at the very start of a file (byte 0) is skipped, as TypeScript and Node skip it, so a program can be made executable and run as a script through #!/usr/bin/env -S nish run (tests/cases/entry_shebang). The line's newline is kept, so every line and column after it is the file's own (reject_shebang_keeps_lines). Anywhere else, after a blank line or a comment included, #! is '#!' can only be used at the start of a file (tests/cases/reject_shebang_not_first), as it is to TypeScript and to the kernel.

  • A diagnostic column counts UTF-16 code units, 1-based, from the start of the line — so é is one column and two bytes, and an emoji is two columns and four. That is the unit an editor indexes a line by, and an editor is who reads a column: it is what the summary line prints, what the caret under the excerpt is placed by, what --emit-checked prints, and what --json's column and endColumn carry (tests/cases/reject_diag_utf8). A DILocation column under -g is the other answer and counts bytes, because a debugger reads it back against the file's bytes (tests/cases/dbg_utf8).

  • Numeric literals. Decimal, hexadecimal (0x10), binary (0b1), octal (0o17), exponent (1e3), and separators (1_000) are accepted (CLI only); a separator goes only between two digits (see Numeric literals). A literal's type comes from its context (see Numeric literals); by default it is number. In i32 mode a literal must have an integral value: 1.5 is rejected with Non-integer literal (tests/cases/reject_float_in_i32); 1e3 is the integer 1000 and accepted, but a point spells a float, so 1.0 is refused as 1.5 is (CLI only). A literal above 2147483647 is rejected with Literal `...` does not fit in i32 (CLI only); the directly negated literal -2147483648 is accepted and is INT_MIN (tests/cases/int_min_literal).

  • String literals. "...", '...', and `...` without holes are all plain string literals; `...${x}...` is a template literal (tests/cases/str_literal, str_template). TypeScript escapes (\n, \", \\, \u...) are decoded by the parser and re-encoded as UTF-8 (tests/cases/str_escape).

    • A surrogate pair is the code point it spells. A high-surrogate escape (\uD800–\uDBFF) written immediately before a low-surrogate escape (\uDC00–\uDFFF) is one code point and encodes as its four UTF-8 bytes, so "\uD83D\uDE00", "\u{1F600}" and "😀" are the same string: === holds, length is 4, and a Map key set by one spelling is found by the others. Either half may be written \uXXXX or \u{...}, in a string or in a template's own text, as in JavaScript (tests/cases/str_surrogate_pair, str_surrogate_map_key).
    • A lone surrogate keeps its WTF-8 bytes. A surrogate escape without that partner is not refused: it encodes on its own as the three bytes UTF-8's pattern gives it ("\uD83D" is ED A0 BD), which is WTF-8 and not valid UTF-8. The pair is joined only inside one literal, so "\uD83D" + "\uDE00" === "\uD83D\uDE00" is false where JavaScript says true (tests/cases/str_surrogate_lone).
    • A \u{...} escape is at most 0x10FFFF, the last code point. A larger one is refused with TypeScript's words, An extended Unicode escape value must be between 0x0 and 0x10FFFF inclusive (tests/cases/reject_surrogate_escape_too_large), however many digits it has, so that no value can wrap round into a surrogate and join the escape after it (reject_surrogate_escape_wraps).
  • Boolean literals true, false (tests/cases/cf_logical).

  • null is a value only where a T | null type is expected (see Nullable types); with no contextual type it is `null` needs a contextual `T | null` type (CLI only). undefined is forbidden (`undefined` is forbidden in Nish; use `null` with a `T | null` type, tests/cases/reject_undefined_value).

  • Bigint literals (10n) and regex literals are forbidden (tests/cases/reject_bigint_literal, reject_regex); a pattern is a library instead, regexCompile from nish/regex (std/regex.ts).

  • Reserved prefix. Function, class, and interface names starting with nish_ are reserved for the runtime: Function names starting with `nish_` are reserved for the runtime (CLI only; src/declarations.ts).

  • Identifiers that may never appear as values: eval, Function, Proxy, Reflect, Symbol, globalThis, arguments, undefined (see Forbidden constructs).