nish/term-pixels

A framebuffer of pixels, and the encoders that draw it to a terminal with the best backend the terminal has: kitty graphics, Sixel, half-block cells or ASCII.

import {
  Framebuffer,
  GRAPHICS_ASCII,
  GRAPHICS_HALFBLOCK,
  GRAPHICS_KITTY,
  GRAPHICS_SIXEL,
  KITTY_TRANSFER_DIRECT,
  PixelBytes,
  PixelRect,
  SIXEL_REGISTERS,
  drawAscii,
  drawHalfBlocks,
  fitFramebuffer,
  kittyDelete,
  kittyFrame,
  scaleFramebuffer,
  sixelFrame,
} from "nish/term-pixels";
import { Framebuffer, fitFramebuffer, GRAPHICS_KITTY, kittyFrame, PixelBytes } from "nish/term-pixels"

const fb = new Framebuffer(160, 90)
fb.fill(0x000020)
fb.set(80, 45, 0xffcc00)
const rect = fitFramebuffer(fb.width, fb.height, 80, 24, 0, 0, GRAPHICS_KITTY)
const out = new PixelBytes(1 << 16)
kittyFrame(fb, rect, 1, out)
// out.text() is the frame's bytes, for one write

A program draws 0xRRGGBB pixels into a Framebuffer and never thinks about the terminal; nish/term-io picks the backend and calls one encoder per frame. Everything here is pure: no capability, no I/O, and the bytes are the same natively and under Node, because the kitty and Sixel output is ASCII and the cell drawers go through Screen, which already is.

  • Framebuffer(width, height) holds the pixels, with set, get, fill, clear and resize. scaleFramebuffer(src, divisor, dst) shrinks one by an integer divisor, nearest-neighbour, for the refresh controller's resolution downgrade.
  • GRAPHICS_ASCII, GRAPHICS_HALFBLOCK, GRAPHICS_SIXEL and GRAPHICS_KITTY name the backends, numbered from worst to best so that a capability and a cap combine with Math.min.
  • fitFramebuffer(fbW, fbH, cols, rows, cellW, cellH, backend) answers the PixelRect the image is drawn in: the cells it covers, centred in a cols × rows area (letterboxed), and its size in the backend's pixels, keeping the framebuffer's aspect ratio.
  • kittyFrame(fb, rect, imageId, out) writes the kitty graphics commands for one frame into a PixelBytes; KITTY_TRANSFER_DIRECT is its transfer. Every frame retransmits the same imageId, which the terminal swaps in atomically, so there is no per-frame delete; kittyDelete(imageId) is the command that frees the image when the program closes.
  • sixelFrame(fb, rect, dither, out) writes one Sixel image, in the fixed palette SIXEL_REGISTERS counts, optionally ordered-dithered.
  • drawHalfBlocks(fb, canvas, rect) and drawAscii(fb, screen, rect, colors) draw onto the cells of a Canvas or a Screen from nish/term, whose own diff makes an unchanged cell cost nothing at flush.

Every encoder samples the framebuffer nearest-neighbour onto the rect's pixels; none of them clears the letterbox around the rect, which is the caller's.

PixelBytes holds the kitty and Sixel encoders' working arrays as well as their output, grown once to the largest frame, so a program that keeps one PixelBytes encodes frame after frame without allocating.

Private helpers share the importing program's flat symbol namespace (docs/wp26-stdlib.md §3e), which is why each one carries the module's name.

Functions

drawAscii

Draw fb onto screen's cells over rect (from fitFramebuffer with GRAPHICS_ASCII), one character a cell for terminals with no graphics: the framebuffer pixel under the cell's centre picks a character from the ramp .:-=+*#%@ by its luminance (Rec. 601 weights, darkest first) and is the character's colour, on the terminal's own background.

When colors is 16 — the Linux console — the colour is first moved to the nearest of xterm's 16 standard colours. Screen sends colours as 24-bit or 256-palette SGR only, and a console that reduces those to its 16 itself then lands on the colour chosen here, not on a neighbour of it.

const drawAscii: (
  fb: Framebuffer,
  screen: Screen,
  rect: PixelRect,
  colors: i32,
) => void
fbFramebufferrequired

The frame's pixels.

screenScreenrequired

The screen to draw on.

rectPixelRectrequired

Where to draw it, from fitFramebuffer with GRAPHICS_ASCII.

colorsi32required

How many colours the terminal shows: 16 for the Linux console, anything else for more.

drawHalfBlocks

Draw fb onto canvas's pixels over rect (from fitFramebuffer with GRAPHICS_HALFBLOCK): two pixels a cell, the upper and lower half block. The image's own pixels are set; a rect row's lower half that the image does not reach is set to COLOR_DEFAULT. Each covered cell is repainted once, and Screen.flush sends only the cells whose glyph or colours changed, so a frame that barely moves costs almost nothing.

const drawHalfBlocks: (
  fb: Framebuffer,
  canvas: Canvas,
  rect: PixelRect,
) => void
fbFramebufferrequired

The frame's pixels.

canvasCanvasrequired

The half-block canvas of the screen to draw on.

rectPixelRectrequired

Where to draw it, from fitFramebuffer with GRAPHICS_HALFBLOCK.

fitFramebuffer

The rect a fbW × fbH framebuffer is drawn in on a cols × rows area of cells cellW × cellH pixels each, by backend (a GRAPHICS_ constant): as large as fits with the framebuffer's aspect ratio kept, and centred, so the spare cells on two sides are the letterbox. The rect starts at column and row 0 of the area; the caller adds the area's own offset.

kitty and Sixel measure in the cell's pixels, taken as 8×16 when cellW or cellH is not positive (the terminal did not answer CSI 16 t). Half-blocks and ASCII take a cell as one pixel wide and two tall, the shape of a usual terminal font, so a square framebuffer stays square. An empty framebuffer or area answers an empty rect at 0, 0.

const fitFramebuffer: (
  fbW: i32,
  fbH: i32,
  cols: i32,
  rows: i32,
  cellW: i32,
  cellH: i32,
  backend: i32,
) => PixelRect
fbWi32required

The framebuffer's width in pixels.

fbHi32required

The framebuffer's height in pixels.

colsi32required

The area's width in cells.

rowsi32required

The area's height in cells.

cellWi32required

A cell's width in pixels as the terminal reported it, or 0 or less when it did not say.

cellHi32required

A cell's height in pixels likewise; with either at 0 or less, kitty and Sixel take the cell as 8×16.

backendi32required

The GRAPHICS_ backend that will draw it, which decides what a pixel of the rect is.

returnsPixelRect

Where to draw, in the area's cells and the backend's pixels.

kittyDelete

The kitty command that deletes the image imageId, every placement of it and its data (a=d,d=I, the uppercase freeing the data), with q=2 so the terminal does not answer. A program writes it when it stops drawing, and puts it in its terminal restore string, so the last frame does not stay behind. Panics, like kittyFrame, on an id below 1.

const kittyDelete: (imageId: i32) => string
imageIdi32required

The id kittyFrame drew with.

returnsstring

The command, to write.

kittyFrame

Append one frame of fb in the kitty graphics protocol (https://sw.kovidgoyal.net/kitty/graphics-protocol/), placed over rect.

  1. The cursor moves to the rect's first cell, where kitty places an image.
  2. a=T transmits and displays at once: f=24 is packed RGB, t=d direct transfer (KITTY_TRANSFER_DIRECT), i the image and p=1 its one placement, s and v the framebuffer's own size, c and r the rect's cells, which kitty scales the image to — so a downgraded framebuffer costs fewer bytes and still fills the rect — C=1 leaves the cursor where it was, and q=2 suppresses every reply.
  3. The RGB bytes go as base64 (RFC 4648 §4) in chunks of 4096 characters, the protocol's limit: each but the last carries m=1, the last m=0. Only the first chunk carries the keys; the rest carry q=2 and m, since a terminal may answer a chunk that does not repeat q.

Every frame reuses imageId and nothing is deleted first. Transmitting an id that exists replaces that image and frees the old one's data, and the same image and placement id replace the placement, so the terminal holds one frame however many are sent. The new frame takes over only when its last chunk (m=0) has arrived, so the old one stays on screen while a frame of hundreds of chunks streams in. Deleting first would blank the image for that whole time. kittyDelete removes the image when the program is done with it.

imageId is 1 to 2^31 - 1, since 0 is the protocol's "no id"; anything else panics. An empty framebuffer or rect writes nothing.

const kittyFrame: (
  fb: Framebuffer,
  rect: PixelRect,
  imageId: i32,
  out: PixelBytes,
) => void
fbFramebufferrequired

The frame's pixels, at its own size.

rectPixelRectrequired

Where to draw it, from fitFramebuffer with GRAPHICS_KITTY.

imageIdi32required

The image's id, 1 or more (below 1 panics), the same for every frame of one program.

outPixelBytesrequired

Where the command bytes are appended.

scaleFramebuffer

Shrink src by divisor into dst, which is resized to ceil(width / divisor) × ceil(height / divisor): each pixel of dst is the top-left pixel of its divisor × divisor block, nearest-neighbour, so a frame costs a quarter or a sixteenth of the bytes at divisor 2 or 4. A divisor below 1 panics; 1 copies. dst must be another framebuffer than src, which resizing it would blank.

const scaleFramebuffer: (
  src: Framebuffer,
  divisor: i32,
  dst: Framebuffer,
) => void
srcFramebufferrequired

The full-size framebuffer, read only.

divisori32required

How many pixels across and down become one, 1 or more; 2 or 4 for the controller's steps.

dstFramebufferrequired

Where the shrunk frame goes; resized to fit.

sixelFrame

Append one frame of fb as DEC Sixel graphics, drawn over rect's width × height terminal pixels from its first cell. The format is DEC's (VT330/VT340 Programmer Reference, "Sixel Graphics"; DECSIXEL):

  • ESC P q opens the device control string with the default parameters, and ESC \ closes it.
  • "1;1;w;h is the raster attributes (DECGRA): square pixels and the image's size, so the terminal can reserve it before the data arrives.
  • #n;2;r;g;b sets colour register n to an RGB colour in percent (DECGCI). The palette is fixed: 6 levels of red, 7 of green and 6 of blue, 252 registers (SIXEL_REGISTERS), the most that fits the 256 a terminal must offer. A register always holds the same colour, so frames never fight over the palette and nothing flickers; each frame declares the registers it uses, in ascending order.
  • The image goes in bands six pixels tall. Within a band, #n selects a register and each character ?–~ is a column of six pixels, one bit each, painted in it; !n repeats a character, $ returns to the band's start for the next colour, and - moves to the next band.

With dither, each pixel's components are offset by the 4×4 Bayer matrix's threshold before they are quantised (ordered dithering), so a gradient becomes a fine, stable pattern rather than steps. An empty rect writes nothing.

const sixelFrame: (
  fb: Framebuffer,
  rect: PixelRect,
  dither: boolean,
  out: PixelBytes,
) => void
fbFramebufferrequired

The frame's pixels; shrunk or grown to the rect by nearest neighbour.

rectPixelRectrequired

Where to draw it, from fitFramebuffer with GRAPHICS_SIXEL.

ditherbooleanrequired

Whether to dither gradients instead of banding them.

outPixelBytesrequired

Where the Sixel bytes are appended.

Classes

Framebuffer

A width × height grid of 0xRRGGBB pixels, row by row, all black when made.

class Framebuffer
Member
new Framebuffer(width: i32, height: i32) A black framebuffer of width × height; a negative size, or more than 2^31 - 1 pixels, panics.
height: i32 Pixels down.
pixels: i32[] One colour per pixel, y * width + x.
width: i32 Pixels across.
clear(): void Set every pixel to black.
fill(colour: i32): void Set every pixel to colour's low 24 bits.
get(x: i32, y: i32): i32 The colour of the pixel at x, y, or black off the framebuffer.
resize(width: i32, height: i32): void Make the framebuffer width × height and black; the array is replaced only when the size changes.
set(x: i32, y: i32, colour: i32): void Set the pixel at x, y to colour's low 24 bits; a pixel off the framebuffer is not drawn.

PixelBytes

A growable buffer of the bytes an encoder writes. text() is them as a string, ready for one write; reset() empties it and keeps its capacity, so one buffer serves every frame.

It also holds the working arrays of kittyFrame and sixelFrame, grown to the largest frame and then reused, so that encoding a frame allocates nothing once the buffer has seen one as large: a frame loop calling an encoder 60 times a second runs in flat memory. An encoder's arrays could not be its own locals, because the arena cannot take a call's allocations back while that call grows a buffer it was handed (bytes above).

class PixelBytes
Member
new PixelBytes(capacity: i32) An empty buffer with room for capacity bytes before it grows.
bytes: u8[] The bytes, of which the first length are written.
length: i32 How many bytes are written.
put(b: i32): void Append the byte b.
putDecimal(v: i32): void Append v, at least 0, in decimal.
putText(s: string): void Append s, which is ASCII.
reserve(n: i32): void Make room for n more bytes, doubling the array as often as that takes.
reset(): void Forget what was written.
text(): string What was written, as one string.

PixelRect

Where a framebuffer is drawn: the cells it covers and its size in the backend's pixels. width and height are terminal pixels for kitty and Sixel, half-cell pixels for half-blocks (cols across, up to rows * 2 down) and cells for ASCII (cols × rows).

class PixelRect
Member
new PixelRect(col: i32, row: i32, cols: i32, rows: i32, width: i32, height: i32, scale: f64) A rect of the given cells and size; fitFramebuffer is how one is usually made.
col: i32 The first column covered, from 0.
cols: i32 The columns covered.
height: i32 The image's height in the backend's pixels.
row: i32 The first row covered, from 0.
rows: i32 The rows covered.
scale: f64 The backend's pixels per framebuffer pixel across; ASCII's rows are half that down.
width: i32 The image's width in the backend's pixels.

Constants

0

GRAPHICS_ASCII: i32

Text cells, one luminance character each: the Linux console's backend.

1

GRAPHICS_HALFBLOCK: i32

Two pixels a cell, through Canvas's half blocks.

3

GRAPHICS_KITTY: i32

The kitty graphics protocol, in the terminal's own pixels.

2

GRAPHICS_SIXEL: i32

DEC Sixel, in the terminal's own pixels.

0

KITTY_TRANSFER_DIRECT: i32

kitty's direct transfer, t=d: the pixels travel base64-encoded inside the escape codes themselves. It is the only transfer built. The protocol's shared-memory transfer (t=s, a POSIX shm object named in the payload) and zlib compression (o=z) are the seam for a local terminal and a slow link; a future KITTY_TRANSFER_SHM would take its own value here and its own branch in kittyFrame, which today has none to choose.

252

SIXEL_REGISTERS: i32

The Sixel registers the fixed palette uses: 6 reds × 7 greens × 6 blues.