Module prefix_cache 

Bounded LRU checkpoints of complete hybrid-model sequence state.

Structs

Entry 🔒

Prefill 🔒

The cursor stays with its request; one advance processes at most one chunk.

PrefillProgress 🔒

PrefixCache 🔒

Functions

chunk_len 🔒

Balance a boundary’s chunks without exceeding memory or scheduling limits.

matches 🔒

pacing_sample 🔒

prefill_rows 🔒

Advance one prompt chunk and retain the predictions needed to resume decode.