nish/compress/gzip
The gzip file format (RFC 1952) around nish/compress/deflate: a ten-byte header, the DEFLATE stream, and a trailer of the CRC-32 and the length of the data mod 2^32. What an HTTP Content-Encoding: gzip body is.
import {
GZIP_END,
GZIP_ERR_CRC,
GZIP_ERR_HEADER,
GZIP_ERR_SIZE,
GZIP_NEED_INPUT,
GzipCompressor,
GzipDecompressor,
GzipFixedCompressor,
GzipReader,
GzipWriter,
gzipCompress,
gzipDecompress,
gzipFixedBound,
} from "nish/compress/gzip";
import { DEFLATE_FAST, compressBound } from "nish/compress/deflate";
import { GzipCompressor, GzipDecompressor } from "nish/compress/gzip";
const gz = new GzipCompressor(DEFLATE_FAST); // once per thread
const out: u8[] = new Array<u8>(compressBound(8192)); // once
const end: i32 = gz.compress(body, 0, n, out, 0); // -1: no room
const gunzip = new GzipDecompressor();
const got: i32 = gunzip.decompress(src, 0, len, back, 0, 1 << 20); // < 0: an error
gzipCompress
GzipCompressor.compress at level, building the compressor for this call alone (about 330 KB, allocated and filled): keep a GzipCompressor for anything repeated.
gzipDecompress
GzipDecompressor.decompress, building the decompressor for this call alone.
gzipFixedBound
The most bytes GzipFixedCompressor.compress writes for n bytes of input.
GzipCompressor
A gzip compressor at one level: a Deflater and the CRC-32 tables, allocated once and reused by every call.
GzipDecompressor
A gzip decompressor: an Inflater and the CRC-32 tables, allocated once.
GzipFixedCompressor
A gzip compressor for short inputs: a FixedDeflater (one fixed-Huffman block, 16 KB window) and the CRC-32 tables, allocated once.
GzipReader
A streaming gzip decompressor, for a body that arrives in pieces or does not fit in memory.
GzipWriter
A streaming gzip compressor, for a body that does not fit in one buffer: write pieces, flush to make everything so far decodable, finish once.
One shot
GzipCompressor and GzipDecompressor own every table a
call needs and allocate nothing per call; gzipCompress and
gzipDecompress build one for a single call. The header written carries no
name, no comment and a zero modification time, XFL 4 at level 1 and 2 at
9, and OS 255 ("unknown"), so the same input gives the same bytes on
every machine.
Reading is strict
A header with a method other than 8 or a reserved
flag bit is GZIP_ERR_HEADER; FEXTRA, FNAME, FCOMMENT and FHCRC are
read and skipped, the header CRC checked. A wrong CRC-32 is GZIP_ERR_CRC,
a wrong length GZIP_ERR_SIZE. Members may follow one another (§2.2) and
are decompressed one after another into the same output. Bytes after the
last member are read as another member's header, so trailing garbage is
refused, not ignored: two or more bytes without the 1f 8b magic are
GZIP_ERR_HEADER, while a single byte, or fewer than 10 that start with
1f 8b, is DEFLATE_ERR_TRUNCATED. The limit bounds the whole output.
Streaming
GzipWriter compresses a body that arrives in pieces and
GzipReader decompresses one: see each class.
Written from RFC 1952.
Functions
gzipCompress
GzipCompressor.compress at level, building the compressor for this call
alone (about 330 KB, allocated and filled): keep a GzipCompressor for
anything repeated.
const gzipCompress: (
src: u8[],
off: i32,
len: i32,
out: u8[],
at: i32,
level: i32,
) => i32
srcu8[]requiredThe array that holds the data to compress.
offi32requiredWhere in src the data starts.
leni32requiredHow many bytes of data to compress.
outu8[]requiredThe caller's array the member is written into.
ati32requiredWhere in out the member starts.
leveli32requiredHow hard to compress, 0 to 9; another level panics.
returnsi32
The index in out one past the member's last byte, or
DEFLATE_NO_ROOM, as GzipCompressor.compress answers.
gzipDecompress
GzipDecompressor.decompress, building the decompressor for this call
alone.
const gzipDecompress: (
src: u8[],
off: i32,
len: i32,
out: u8[],
at: i32,
limit: i32,
) => i32
srcu8[]requiredThe array that holds the gzip data, one member or several.
offi32requiredWhere in src the first member starts.
leni32requiredHow many bytes the members are.
outu8[]requiredThe caller's array the data is written into.
ati32requiredWhere in out the data starts.
limiti32requiredThe most bytes of data to accept over all the members.
returnsi32
The index in out one past the data's last byte, or a negative
code, as GzipDecompressor.decompress answers.
gzipFixedBound
The most bytes GzipFixedCompressor.compress writes for n bytes of input.
const gzipFixedBound: (n: i32) => i32
ni32required
The length of the input, in bytes; a negative n answers the
bound of 0.
returnsi32
A size in bytes, not a position: a room of this many always
holds the member.
Classes
GzipCompressor
A gzip compressor at one level: a Deflater and the CRC-32 tables,
allocated once and reused by every call. Not shared between threads.
class GzipCompressor
| Member | |
|---|---|
new GzipCompressor(level: i32) |
A compressor at level, 0 to 9 (DEFLATE_FAST, DEFLATE_DEFAULT, ...); another level panics. |
compress(src: u8[], off: i32, len: i32, out: u8[], at: i32): i32 |
Compresses src[off .. off + len) into out from at as one gzip member, and answers where it ends, or DEFLATE_NO_ROOM (-1) when out cannot hold it; compressBound(len) bytes always can. A window outside its array panics. Allocates nothing. |
GzipDecompressor
A gzip decompressor: an Inflater and the CRC-32 tables, allocated once.
Not shared between threads.
class GzipDecompressor
| Member | |
|---|---|
new GzipDecompressor() |
Builds the tables. |
decompress(src: u8[], off: i32, len: i32, out: u8[], at: i32, limit: i32): i32 |
Decompresses every member of src[off .. off + len) into out from at, writing at most limit bytes and never past out's end, and answers where the output ends; or DEFLATE_ERR_OUTPUT_LIMIT when the data is longer, GZIP_ERR_HEADER, GZIP_ERR_CRC, GZIP_ERR_SIZE, DEFLATE_ERR_TRUNCATED or another DEFLATE_ERR_* for a malformed stream. An empty input is truncated. A window outside its array, or a negative limit, panics. Allocates nothing. |
GzipFixedCompressor
A gzip compressor for short inputs: a FixedDeflater (one fixed-Huffman
block, 16 KB window) and the CRC-32 tables, allocated once. Each call
depends on its input alone. Not shared between threads.
class GzipFixedCompressor
| Member | |
|---|---|
new GzipFixedCompressor() |
Builds the tables. |
compress(src: u8[], off: i32, len: i32, out: u8[], at: i32, room: i32): i32 |
Compresses src[off .. off + len) into out[at .. at + room) as one gzip member, and answers where it ends, or DEFLATE_NO_ROOM (-1) as soon as it would need more than room bytes, so a caller passing the most it will take stops early on an input that does not shrink. gzipFixedBound(len) always fits. A window outside its array panics. Allocates nothing. |
GzipReader
A streaming gzip decompressor, for a body that arrives in pieces or does
not fit in memory. Sans-IO in the nish/net/postgres shape: copy input
into inputBuffer() at inputEnd() and commit it (or feed a window),
then call read until it answers GZIP_NEED_INPUT; at the end of the
input, endInput(). read(out, at, max) copies up to max decoded bytes
and answers how many, GZIP_NEED_INPUT, GZIP_END once the stream and its
trailer are through, or a negative error, final. A total of more than
limit decoded bytes is DEFLATE_ERR_OUTPUT_LIMIT. Concatenated members
are read as one body, as by GzipDecompressor. About 100 KB, allocated
once; start begins the next body.
class GzipReader
| Member | |
|---|---|
new GzipReader(limit: i64) |
A reader that refuses more than limit decoded bytes in all; a negative limit panics. It is ready for a first body without a call to start. |
commit(n: i32): void |
Takes n bytes copied into inputBuffer() at inputEnd(); more than inputRoom() panics. |
endInput(): void |
Says that no input follows what has been given. |
feed(src: u8[], off: i32, len: i32): i32 |
Copies as much of src[off .. off + len) as fits and answers how much. A window outside src panics. |
inputBuffer(): u8[] |
The buffer to copy input into, from inputEnd() for at most inputRoom() bytes; then commit how many were copied. It may first move the input not yet decoded to the front, so read inputEnd() after this call. |
inputEnd(): i32 |
Where new input goes in inputBuffer(). |
inputRoom(): i32 |
How many bytes inputBuffer() has room for, from inputEnd(). Like inputBuffer(), it may first move the input not yet decoded to the front. |
read(out: u8[], at: i32, max: i32): i32 |
Copies up to max decoded bytes into out from at and answers how many (at least 1; max is at least 1), or GZIP_NEED_INPUT when it needs more input first, GZIP_END once the body has ended, or a negative error, after which it answers that error again until start. A window outside out panics. |
start(): void |
Begins a new body, dropping any input and output left over. |
GzipWriter
A streaming gzip compressor, for a body that does not fit in one buffer:
write pieces, flush to make everything so far decodable, finish once.
It buffers up to 64 KB of input (and 32 KB of history to match against),
compressing whenever that fills, so output arrives in bursts. Every call
writes into a caller window and answers where its output ends, or
DEFLATE_NO_ROOM when the window is shorter than bound(len), the most
that call can produce; nothing is consumed then. About 430 KB, allocated
once; start begins the next body.
class GzipWriter
| Member | |
|---|---|
new GzipWriter(level: i32) |
A writer at level, 0 to 9; another level panics. It is ready for a first body without a call to start. |
bound(len: i32): i32 |
The most bytes a write of len bytes, or a flush or finish after it, can produce. Less than 64 KB is pending before the call, and the call compresses once per 64 KB pending: deflateBound of all of it, with the constant part of deflateBound again for each compression after the first. |
finish(out: u8[], at: i32): i32 |
Compresses everything pending as the final block and writes the trailer; answers the new end of out, or DEFLATE_NO_ROOM below bound(0). start begins another body; until then write, flush and finish panic, as does a window outside out. |
flush(out: u8[], at: i32): i32 |
Compresses everything pending and ends with an empty stored block, so a reader can decode every byte written so far (zlib's Z_SYNC_FLUSH). Answers the new end of out, or DEFLATE_NO_ROOM below bound(0). Flushing after finish panics, as does a window outside out. |
start(): void |
Begins a new body: the next output starts with a header. |
write(src: u8[], off: i32, len: i32, out: u8[], at: i32): i32 |
Takes src[off .. off + len) and answers where the output it produced in out from at ends (often at: input waits until 64 KB are pending), or DEFLATE_NO_ROOM when out has less than bound(len) bytes from at. Writing after finish panics, as does a window outside its array. |
Constants
-100
GZIP_END: i32
GzipReader: the stream has ended and every byte of it was read.
-10
GZIP_ERR_CRC: i32
The trailer's CRC-32 is not the CRC-32 of the data.
-9
GZIP_ERR_HEADER: i32
A gzip header that is not one: the wrong magic, a method other than DEFLATE, a reserved flag, or a bad header CRC.
-11
GZIP_ERR_SIZE: i32
The trailer's length is not the data's length mod 2^32.
0
GZIP_NEED_INPUT: i32
GzipReader: no more output yet; give it input.

