Module residency 

The expert pool: which records the GPU may touch, LRU managed here, with two backings behind one interface.

Set: a Metal residency set over the packed expert file’s mapping. Every record is its own zero-copy sub-buffer (wrapped on first use); the set holds up to a budget of them and the GPU reaches a record through its GPU address. A record leaving the set keeps its pages in the page cache until the OS needs the memory, so a miss on a recently evicted record is served from RAM (an unwired second tier with no copies); a cold miss reads the file through the cache before the record is added. Records added while a command buffer waits on an event are resident by the time the GPU is released (measured: 1 to 4 ms for 150 records, no faults).

Copy: one wired Metal buffer of record slots, filled by uncached parallel reads. No second tier, but no page-cache churn either, which matters when the machine is short of memory.

Structs

CopyPool

Landed

Per-record completion retained by the deadline policy until its slot is safe to reuse. All record IO uses the measured pread path.

PlannedRead 🔒

One record read. A zero destination means a page-cache read for a mapped residency-set record; otherwise it is a CPU address in the copy pool. need_index connects completion to the caller’s acquired records.

ReadPlan

Reads required between pool acquisition and residency completion. Built by the service thread, then moved or borrowed by reader threads.

RecordRead 🔒

A destination and its measurement ticket travel together through batching.

Residency

Enums

Pool

Functions

fetch_into_slots 🔒

Fill independent slots in parallel, one complete record per read.

read_at 🔒

read_record 🔒

A zero destination faults mapped pages in through bounded scratch.

Type Aliases

Buf 🔒