Expand description
CLI generation, prompt processing and shared decode entry points.
Modulesยง
- decode ๐
- One complete verification step, resumable between shared-GPU turns.
- diagnostics ๐
- Optional CPU-reference comparisons, research dumps and per-step traces.
Structsยง
- Prefill
Resume ๐
Functionsยง
- check_
budget ๐ - prefill_
engine ๐ - Stream equal-sized prompt chunks so a short tail does not reread the whole store.
- prefill_
prompt ๐ - Fill the trunk and draft caches, then return the next token and draft chain.
- prefill_
row_ ๐batches - Use the decode row kernels for short prompts and explicit row-path checks.
- qwen4_
exp_ ๐gen_ once - run