Lists & streaming

Long lists and streaming text

A chat client is a very long list plus text that arrives a token at a time. compose-rust has a record for each: a windowed list that the renderer drives, and an append that carries only the new tail. Both work today at the boundary. The authoring API that writes them arrives with 1.0.0.

Windowed lists Works

A virtualised list cannot be described as a whole tree: the Host would have to describe items it never built so that the renderer knows they exist. So for lazy lists the flow is inverted.

1. The Host declares the total item count and a stable key per item.
2. The renderer decides which range is visible and asks for it with a range request event.
3. The Host builds the subtrees for that range plus a buffer on each side, and nothing else.

Each item carries its key, which the renderer hands to Compose's LazyColumn as the item key. Scroll position and item identity stay in the renderer, and the data stays in the Host. With 10,000 items, a request for 20 visible rows and a buffer of 4, the Host builds 28 items: the cost follows the window, not the list. A test checks exactly that.

LazyColumn, LazyRow and LazyGrid share this contract. In 0.0.1 a Runtime receives the request as a RangeRequest and answers it with records. The authoring API writes the same records, so a lazy list written with it is windowed the same way Planned.

A plain column

A ScrollColumn with one child per item is correct, and fine for the hundreds of rows that fit a screen with some scrollback. It does not window: every item becomes a real node on both sides. Updating one item still sends only that item's change, so the cost of this pattern is the node count, not the update path.

Streaming text Works

An LLM response arrives token by token. Replacing a Text's whole content on every token re-sends the entire message each time. The message grows, so the total cost grows quadratically. compose-rust sends an AppendText record instead, which carries only the new tail.

The Host coalesces: tokens that arrive between two frames merge into one AppendText per node, flushed in the next frame. A token never costs a batch of its own, and the buffers are reused, so steady streaming does not allocate. Measured on the Host, a 36 KB message stays under a 64-byte batch per token, and the p99 streaming frame is 125 ns.

// Queues a tail for the next frame. Repeated calls coalesce into one record
// per node and into a single frame request.
host.append_text(node_id, " token");

Host::append_text takes a protocol node id. It is meant for the layer that builds the tree, not for application code. A Runtime writes the same record with Batch::append_text_node. The authoring API will expose streaming without node ids Planned.

Work on worker threads

Domain work must not run on the UI thread, because that thread also runs your UI code and the whole boundary. PTY reads, HTTP, LLM streaming and file I/O belong on Host worker threads.

A worker's only job, as far as the UI is concerned, is to change state. Waking the renderer is internal: request_frame_from_worker asks for a frame from any thread, and repeated requests coalesce into one frame on the Compose frame clock. In the authoring API a worker writes the same state objects as the UI thread, and the Host requests the frame for it. Application code never calls a boundary function, and there is no queue between the two sides to fill up.

The frame budget

The target is a comparison, not a number in isolation. The baseline is the same screen written directly in Kotlin and Compose, and compose-rust may use at most 10% more frame time than that, against an 8.33 ms budget on a 120 Hz display.

Encoding a batch allocates nothing on the heap. Memory is measured as a trend: repeating the same interaction must not increase it, and growth is investigated as a leak or a cache that stopped working.