nish/net/url
The request target's path and query, split and percent-decoded in place, and its query string read as key/value pairs; and whole URLs, parsed into their parts and references resolved against them.
import {
URL_DECODE_ALLOW_NUL,
URL_DECODE_ERR_ESCAPE,
URL_DECODE_ERR_NUL,
URL_DECODE_ERR_SPACE,
URL_DECODE_FORM,
URL_DECODE_LENIENT,
URL_DEFAULT_MAX_PAIRS,
URL_QUERY_END,
URL_QUERY_ERR_PAIRS,
URL_QUERY_PAIR,
Url,
UrlPathSegments,
UrlQuery,
UrlQueryLimits,
percentDecode,
percentDecodeErrorAt,
percentDecodePathSegment,
urlDecode,
urlDefaultPort,
urlIsHostPort,
urlOrigin,
urlParse,
urlQueryFind,
urlQueryGet,
urlQueryStart,
urlResolve,
urlSerialize,
} from "nish/net/url";
import { UrlQuery, UrlQueryLimits, URL_DECODE_FORM, URL_QUERY_PAIR, percentDecode, urlQueryFind, urlQueryStart } from "nish/net/url";
const q: i32 = urlQueryStart(target, 0, len) // -1: no `?`, so no query
if (q >= 0) {
const query: UrlQuery = new UrlQuery(new UrlQueryLimits())
query.reset(target, q + 1, len - q - 1)
if (urlQueryFind(query, "name") === URL_QUERY_PAIR) {
const n: i32 = percentDecode(target, query.valueStart, query.valueLength, out, 0, URL_DECODE_FORM)
if (n >= 0) {
// out[0, n) is the value
} // else n is a URL_DECODE_ERR_* code: a peer's `%zz`, never read out
} // else no such key, or URL_QUERY_ERR_PAIRS past the cap
}
percentDecode
Percent-decodes src[off, off + len) into out from out[at] on, and answers the number of bytes written, or a negative URL_DECODE_ERR_* code; flags is a sum of URL_DECODE_* (0 is strict: malformed escapes and %00 refused, + kept).
percentDecodeErrorAt
The offset in src of the byte percentDecode with the same arguments fails at — the % of a malformed escape or of %00, or the first byte out has no room for — or -1 when it does not fail.
percentDecodePathSegment
percentDecode for one path segment: + is literal whatever flags say, and %2F is a 0x2F byte in the segment.
urlDecode
s percent-decoded with flags, or null when percentDecode refuses it or what it decodes to is not valid UTF-8, which is where decodeURIComponent throws.
urlDefaultPort
The port a scheme uses when a URL names none.
urlIsHostPort
Whether bytes[a, b) is uri-host [ ":" port ] (RFC 3986 §3.2.2, §3.2.3) by the rules urlParse holds a URL's host and port to: a bracketed IPv6 address, a dotted-decimal IPv4 address, or a reg-name of unreserved bytes that does not read as a number, then an optional : and a port of digits up to 65535.
urlOrigin
u's origin as scheme://host:port, the port always written (the scheme's default when u names none), so origins compare as plain strings: http://A.com/x and http://a.com:80/y are both http://a.com:80.
urlParse
The absolute URL s split into its parts, or null when it is not one this module accepts.
urlQueryFind
Moves q on to the first of its remaining pairs whose key, decoded in form mode, equals name, and answers URL_QUERY_PAIR with q's windows on it; or URL_QUERY_END, or the error next() answered.
urlQueryGet
The first value for name in the query of the request target target, form-decoded strictly: "" for a key written without =, as URLSearchParams.get answers, and null when the target has no query, the key is absent, the value is malformed or not UTF-8, or the key comes after URL_DEFAULT_MAX_PAIRS pairs.
urlQueryStart
The index of the first ? in bytes[off, off + len), or -1 when there is none.
urlResolve
The reference ref resolved against the absolute URL base by RFC 3986 §5.2.2, strictly (http:g is http:g, not relative), or null when ref breaks a rule urlParse names or what it resolves to is an http, https, ws or wss URL without a host.
urlSerialize
u written back as RFC 3986 §5.3 recomposes it, an IPv6 host in brackets.
Url
A URL split into its parts (RFC 3986 §3).
UrlPathSegments
The segments of a path, split on the raw /.
UrlQuery
The pairs of a query string, one per next(), in order.
UrlQueryLimits
What a UrlQuery refuses to yield.
A handler that routes on a path or reads ?name=… from a query otherwise
writes its own byte scan, and each such scan decides differently what %zz,
%00, + or a hundred thousand & mean. This module decides once:
The byte API allocates nothing
urlQueryStart, percentDecode,
UrlPathSegments and UrlQuery read a window bytes[off, off + len) and
answer offsets into it, or write into the caller's array; reset reuses an
iterator, so a server that holds one per connection allocates nothing per
request. No byte outside the window is read. A window outside its array, or
an at outside out, is a bug in the caller and panics; nothing a peer
sends does.
Decoding is strict unless asked otherwise
With no flags,
percentDecode refuses a % not followed by two hex digits
(URL_DECODE_ERR_ESCAPE) and %00 (URL_DECODE_ERR_NUL), and copies +
as it stands. URL_DECODE_FORM reads + as a space, as
application/x-www-form-urlencoded does; URL_DECODE_LENIENT copies a
malformed escape as it stands, as the WHATWG URL Standard's percent-decode
and URLSearchParams do; URL_DECODE_ALLOW_NUL decodes %00 to a 0 byte.
For valid UTF-8, URLSearchParams is URL_DECODE_FORM | URL_DECODE_LENIENT | URL_DECODE_ALLOW_NUL; it replaces an invalid sequence with U+FFFD.
Decoded bytes are bytes
The byte API does not check that what it
decodes is UTF-8, where decodeURIComponent throws on %FF: a path segment
or a value may be binary, and whatever reads it as text checks it.
urlDecode and urlQueryGet, which answer a string, refuse anything that is
not valid UTF-8.
A path is split before it is decoded
UrlPathSegments splits on the
raw / only, and percentDecodePathSegment decodes one segment, so %2F
is a 0x2F byte inside its segment and never starts a new one: /a%2Fb/c is
the two segments a/b and c. + is literal in a path.
A query yields at most `UrlQueryLimits.maxPairs` pairs
A query yields at most UrlQueryLimits.maxPairs pairs (100 by default),
and past it next() answers URL_QUERY_ERR_PAIRS and stays spent. Empty
pairs (a&&b, a leading or trailing &) are skipped and not counted, as
URLSearchParams skips them.
A fragment is not part of a request target
A fragment is not part of a request target (RFC 9112 §3.2), so the byte
API never looks for a #: in a target that holds one, it is a byte of the
path or of the last value.
A whole URL is parsed strictly, and nothing in it is decoded
A client
that is handed http://… by a user, a redirect or a link needs its host and
port before it connects, and the place each reference points to:
import { Url, urlOrigin, urlParse, urlResolve, urlSerialize } from "nish/net/url";
const base: Url | null = urlParse("https://example.com/a/b?q")
if (base !== null) {
const next: Url | null = urlResolve(base, "../c#top") // https://example.com/c#top
// next.host, next.port (-1: the default), urlOrigin(next) === "https://example.com:443"
}
urlParse and urlResolve follow RFC 3986 (§3 for the parts, §5.2 to
resolve, dot segments removed) and refuse, with null, what two readers could
take for different hosts: a byte outside printable ASCII, a backslash, a
malformed %, a second @, a percent-encoded or otherwise non-unreserved
host byte, a host that looks numeric but is not a strict dotted-decimal IPv4
address (127.1, 0x7f.0.0.1, 2130706433), an IPv6 zone or IPvFuture
literal, and a port past 65535. The scheme and host are lower-cased; every
other part is kept as written, still percent-encoded.
The record of what this module refuses and why is docs/security/url.md.
Functions
percentDecode
Percent-decodes src[off, off + len) into out from out[at] on, and
answers the number of bytes written, or a negative URL_DECODE_ERR_* code;
flags is a sum of URL_DECODE_* (0 is strict: malformed escapes and %00
refused, + kept). Decoding never makes the input longer, so len bytes
of room always suffice; with less, URL_DECODE_ERR_SPACE is answered before
any write at or past out.length. On an error the bytes before it have been
written. What is decoded is not checked as UTF-8.
const percentDecode: (
src: u8[],
off: i32,
len: i32,
out: u8[],
at: i32,
flags: i32,
) => i32
srcu8[]requiredThe array holding the encoded text.
offi32requiredWhere the encoded text starts in src.
leni32requiredThe encoded text's length in bytes; a window outside src panics.
outu8[]requiredThe array the decoded bytes are written into.
ati32requiredThe index in out the first decoded byte goes to; one below 0 or above out.length panics.
flagsi32requiredURL_DECODE_FORM, URL_DECODE_LENIENT and URL_DECODE_ALLOW_NUL, summed, or 0 for strict.
returnsi32How many bytes were written from out[at] on, or URL_DECODE_ERR_ESCAPE, URL_DECODE_ERR_NUL or URL_DECODE_ERR_SPACE.
percentDecodeErrorAt
The offset in src of the byte percentDecode with the same arguments
fails at — the % of a malformed escape or of %00, or the first byte
out has no room for — or -1 when it does not fail. It writes what
percentDecode would have written before the error.
const percentDecodeErrorAt: (
src: u8[],
off: i32,
len: i32,
out: u8[],
at: i32,
flags: i32,
) => i32
srcu8[]requiredThe array holding the encoded text.
offi32requiredWhere the encoded text starts in src.
leni32requiredThe encoded text's length in bytes; a window outside src panics.
outu8[]requiredThe array the decoded bytes are written into, as percentDecode writes them.
ati32requiredThe index in out the first decoded byte goes to; one below 0 or above out.length panics.
flagsi32requiredThe URL_DECODE_* sum percentDecode was given.
returnsi32The index in src decoding stops at, or -1 when all of it decodes.
percentDecodePathSegment
percentDecode for one path segment: + is literal whatever flags say,
and %2F is a 0x2F byte in the segment. Split the path with
UrlPathSegments first, then decode each segment.
const percentDecodePathSegment: (
src: u8[],
off: i32,
len: i32,
out: u8[],
at: i32,
flags: i32,
) => i32
srcu8[]requiredThe array holding the path.
offi32requiredWhere the segment starts in src, as UrlPathSegments gives it.
leni32requiredThe segment's length in bytes; a window outside src panics.
outu8[]requiredThe array the decoded bytes are written into.
ati32requiredThe index in out the first decoded byte goes to; one below 0 or above out.length panics.
flagsi32requiredA URL_DECODE_* sum; URL_DECODE_FORM in it is ignored.
returnsi32How many bytes were written from out[at] on, or a negative URL_DECODE_ERR_* code, as percentDecode answers.
urlDecode
s percent-decoded with flags, or null when percentDecode refuses it
or what it decodes to is not valid UTF-8, which is where
decodeURIComponent throws. s is expected to be ASCII, as a URL on the
wire is. It allocates its answer.
const urlDecode: (s: string, flags: i32) => string | null
sstringrequiredThe encoded text; a code unit past 0xFF (possible only under Node) answers null.
flagsi32requiredURL_DECODE_FORM, URL_DECODE_LENIENT and URL_DECODE_ALLOW_NUL, summed, or 0 for strict.
returnsstring | nullThe decoded text, or null for a refused escape or %00, or text that is not UTF-8.
urlDefaultPort
The port a scheme uses when a URL names none.
const urlDefaultPort: (scheme: string) => i32
schemestringrequiredThe scheme, lower-cased and without its :, as Url.scheme holds it.
returnsi3280 for http and ws, 443 for https and wss, and -1 for any other scheme.
urlIsHostPort
Whether bytes[a, b) is uri-host [ ":" port ] (RFC 3986 §3.2.2, §3.2.3)
by the rules urlParse holds a URL's host and port to: a bracketed IPv6
address, a dotted-decimal IPv4 address, or a reg-name of unreserved bytes
that does not read as a number, then an optional : and a port of digits
up to 65535. An empty host and an empty port are allowed, as in a URL; a
percent-encoded or sub-delims byte in a reg-name is not. It is the rule an
HTTP server holds a Host field to (RFC 9112 §3.2). Allocates nothing.
const urlIsHostPort: (
bytes: u8[],
a: i32,
b: i32,
) => boolean
bytesu8[]requiredThe bytes that hold the value.
ai32requiredThe index of the value's first byte.
bi32requiredThe index past the value's last byte.
returnsbooleanTrue when the bytes are a host and an optional port; false otherwise, and for a window outside bytes.
urlOrigin
u's origin as scheme://host:port, the port always written (the scheme's
default when u names none), so origins compare as plain strings:
http://A.com/x and http://a.com:80/y are both http://a.com:80. Null for
a scheme other than http, https, ws or wss, which has no origin to compare.
Hosts are compared as written, lower-cased, so [::1] and [0::1] are
different origins, and so are a.com. and a.com: equal strings are the
same origin, and a split errs toward "different".
const urlOrigin: (u: Url) => string | null
uUrlrequiredThe URL whose origin is wanted.
returnsstring | nullThe origin, such as https://example.com:443, or null for a scheme with no default port.
urlParse
The absolute URL s split into its parts, or null when it is not one this
module accepts.
s is refused when it has no scheme, holds a byte outside printable ASCII (a
control byte, a space, DEL or anything past 0x7F), a backslash or a % not
followed by two hex digits, or its authority has more than one @, a host
with a byte outside RFC 3986's unreserved set (so a percent-encoded one), a
host whose last label is numeric but that is not a dotted-decimal IPv4
address, a malformed IPv6 literal or a port that is not digits or is
above 65535. An http, https, ws or wss URL needs a non-empty host.
The path's dot segments are removed, as RFC 3986 §5.2.2 removes them from a reference with a scheme. Nothing is decoded. It allocates its answer.
const urlParse: (s: string) => Url | null
sstringrequiredThe absolute URL, such as https://example.com/a?b#c.
returnsUrl | nullThe URL's parts, still percent-encoded, scheme and host lower-cased; or null when s is refused.
urlQueryFind
Moves q on to the first of its remaining pairs whose key, decoded in form
mode, equals name, and answers URL_QUERY_PAIR with q's windows on it;
or URL_QUERY_END, or the error next() answered. Nothing is allocated.
?na%6De=x holds the key name; a key with a malformed escape or %00
matches nothing. Call it again for the next pair with the same key.
name is ASCII: natively it is compared with the key's bytes, under Node
with its UTF-16 code units, so a non-ASCII name answers differently.
const urlQueryFind: (q: UrlQuery, name: string) => i32
qUrlQueryrequiredThe query iterator, moved on past the pair found; its windows bound that pair.
namestringrequiredThe key to look for, as it reads decoded.
returnsi32URL_QUERY_PAIR when a pair with the key was found, URL_QUERY_END when none is left, or the negative URL_QUERY_ERR_* code next() answered.
urlQueryGet
The first value for name in the query of the request target target,
form-decoded strictly: "" for a key written without =, as
URLSearchParams.get answers, and null when the target has no query, the
key is absent, the value is malformed or not UTF-8, or the key comes after
URL_DEFAULT_MAX_PAIRS pairs. It allocates its answer.
const urlQueryGet: (
target: string,
name: string,
) => string | null
targetstringrequiredA request target such as /search?q=nish; everything after its first ? is the query.
namestringrequiredThe key to look for, as it reads decoded; ASCII, as urlQueryFind asks.
returnsstring | nullThe first decoded value for name, "" when it has no =, or null in the cases above.
urlQueryStart
The index of the first ? in bytes[off, off + len), or -1 when there is
none. The path is [off, q) and the query (q, off + len); nothing is
copied. A # is not looked for: a request target holds no fragment.
const urlQueryStart: (
bytes: u8[],
off: i32,
len: i32,
) => i32
bytesu8[]requiredThe array holding the request target.
offi32requiredWhere the target starts in bytes.
leni32requiredThe target's length in bytes; a window outside bytes panics.
returnsi32The index in bytes of the ?, or -1 when the target has no query.
urlResolve
The reference ref resolved against the absolute URL base by RFC 3986
§5.2.2, strictly (http:g is http:g, not relative), or null when ref
breaks a rule urlParse names or what it resolves to is an http, https, ws
or wss URL without a host. base's fragment is never carried over. It
allocates its answer and does not change base.
const urlResolve: (base: Url, ref: string) => Url | null
baseUrlrequiredThe URL ref is relative to, as urlParse made it.
refstringrequiredThe reference: an absolute URL, a network-path //host/p, an absolute path, a relative path, a query or a fragment.
returnsUrl | nullThe resolved URL, still percent-encoded, or null when ref is refused or resolves to a URL without a host its scheme needs.
urlSerialize
u written back as RFC 3986 §5.3 recomposes it, an IPv6 host in brackets.
The userinfo is written when either half is non-empty. A path that starts
with // under no authority is written after /., as the WHATWG URL
Standard writes it, so the string does not read back with an authority.
const urlSerialize: (u: Url) => string
uUrlrequiredThe URL to write; its parts are written as they are held, so still percent-encoded.
returnsstringThe URL as one string, scheme to fragment.
Classes
Url
A URL split into its parts (RFC 3986 §3). Each part is held as it was
written, still percent-encoded, except that scheme and host are
lower-cased. urlParse and urlResolve make one; urlSerialize writes one
back.
class Url
| Member | |
|---|---|
new Url() |
|
fragment: string | null |
Without its #, or null when there is no #. |
hasAuthority: boolean |
True when the URL has a // authority; mailto:a@b has none and file:///x has an empty one. |
host: string |
A lower-cased reg-name, a dotted IPv4 address, or an IPv6 address without its brackets. |
password: string |
The userinfo after its first :, as written; empty when there is none. |
path: string |
With its dot segments removed; empty, or starting with / when there is an authority. |
port: i32 |
0 to 65535, or -1 when the URL names none; urlOrigin supplies the scheme's default. |
query: string | null |
Without its ?, or null when there is no ?. |
scheme: string |
Lower-cased, without its :: http. |
username: string |
The userinfo before its first :, as written; empty when there is none. |
UrlPathSegments
The segments of a path, split on the raw /. A leading / starts the
path rather than an empty segment, so /a//b/ is a, , `b`, , / is
one empty segment and an empty window has none. Each segment's raw bytes
are bytes[start, start + length); decode one with
percentDecodePathSegment.
class UrlPathSegments
| Member | |
|---|---|
new UrlPathSegments() |
An iterator over nothing until reset. |
bytes: u8[] |
The array the path is read from, as reset was given it. |
length: i32 |
The current segment's length in raw bytes. |
start: i32 |
The current segment's first byte. |
next(): boolean |
Moves to the next segment, whose bytes start and length then name. |
reset(bytes: u8[], off: i32, len: i32): void |
Starts over on the path bytes[off, off + len); panics when the window is outside the array. |
UrlQuery
The pairs of a query string, one per next(), in order. The window is the
query without its ?. & separates pairs and the first = in a pair ends
its key: a&b=&c=d=e is a with no value, b with an empty one and c
with d=e. A key's raw bytes are bytes[keyStart, keyStart + keyLength)
and a value's bytes[valueStart, valueStart + valueLength); decode them
with percentDecode and URL_DECODE_FORM.
class UrlQuery
| Member | |
|---|---|
new UrlQuery(limits: UrlQueryLimits) |
An iterator over nothing until reset, with the cap in limits. |
failed: i32 |
0, or the URL_QUERY_ERR_* code every later next() answers. |
hasValue: boolean |
False for a pair written without = (a), true for a= with valueLength 0. |
keyLength: i32 |
The current pair's key length in raw bytes. |
keyStart: i32 |
Where the current pair's key starts in bytes. |
valueLength: i32 |
The current pair's value length in raw bytes, 0 when it has none. |
valueStart: i32 |
Where the current pair's value starts in bytes. |
next(): i32 |
Moves to the next pair, setting the key and value windows. |
reset(bytes: u8[], off: i32, len: i32): void |
Starts over on the query bytes[off, off + len), clearing failed; panics when the window is outside the array. |
UrlQueryLimits
What a UrlQuery refuses to yield. The iterator copies the number when it
is made; a negative cap is read as zero, so every query with a pair fails.
class UrlQueryLimits
| Member | |
|---|---|
new UrlQueryLimits() |
|
maxPairs: i32 |
The most pairs one query yields, URL_DEFAULT_MAX_PAIRS (a field starts at a literal); past it next() answers URL_QUERY_ERR_PAIRS and stays spent. |
Constants
4
URL_DECODE_ALLOW_NUL: i32
%00 decodes to a 0 byte rather than being refused.
-1
URL_DECODE_ERR_ESCAPE: i32
A % not followed by two hex digits: %zz, or %4 or % at the end of the window.
-2
URL_DECODE_ERR_NUL: i32
%00 without URL_DECODE_ALLOW_NUL.
-3
URL_DECODE_ERR_SPACE: i32
out has no room for the next byte; nothing is written at or past out.length.
1
URL_DECODE_FORM: i32
+ is a space, as application/x-www-form-urlencoded writes one.
2
URL_DECODE_LENIENT: i32
A % not followed by two hex digits is copied as it stands rather than refused (WHATWG percent-decode).
100
URL_DEFAULT_MAX_PAIRS: i32
The default for UrlQueryLimits.maxPairs.
0
URL_QUERY_END: i32
The window holds no more pairs; every later next() answers the same.
-4
URL_QUERY_ERR_PAIRS: i32
The query holds more than maxPairs pairs; sticky in failed.
1
URL_QUERY_PAIR: i32
next() found a pair; the iterator's windows bound it.

