Module prefill 

Prefill engine: a prompt chunk of up to a few thousand tokens runs through the model layer by layer, so each layer’s experts are streamed once for the whole chunk instead of once per few tokens.

Dense projections use simdgroup-matrix GEMM kernels; attention is the GEMM form (S = QK^T, causal softmax with the QSA visibility mask, O = PV) in sub-chunks of queries; the DeltaNet scan and the PLE conv walk the chunk sequentially per channel. Routed experts are handled per expert: its tokens’ rows are gathered, run through gate/up/act/down GEMMs and scatter-added back with their routing weights; records stream through a ring of pool slots, a fetch thread filling batches ahead of the GPU with two events per batch. Every side effect (KV cache, index keys, recurrent state, PLE ring, MTP cache) is left exactly as the row-batched engine would have left it, so decode continues from the result.

Modules

allocation 🔒

Prefill scratch allocation and chunk capacity.

attention 🔒

QSA masks and GEMM attention over query sub-chunks.

deltanet 🔒

Prefill DeltaNet convolution, scan, and output gate.

experts 🔒

Prefill expert grouping and event-protected streaming ring.

hyperconnection 🔒

Prefill grouped residual reads and injection.

ple 🔒

Prefill PLE n-gram gather, gating, and dilated convolution.

projection 🔒

Dense prefill projections with narrow-output fallback.

stats 🔒

Cumulative completed prefill chunks. Sequence rollback does not undo work.

Structs

ChunkStats

Timing of one prefill chunk.

MoeRef 🔒

PrefillScratch 🔒

PrefillStats

Constants

GROUP 🔒

MAX_PREFILL_ROWS

QS 🔒

Query rows per attention sub-chunk (bounds the S matrix).

RING 🔒

Slots of the expert stream ring, and experts per fetch batch.