Skip to main content

Module runner

Module runner 

Source
Expand description

CLI generation, prompt processing and shared decode entry points.

Modulesยง

decode ๐Ÿ”’
One complete verification step, resumable between shared-GPU turns.
diagnostics ๐Ÿ”’
Optional CPU-reference comparisons, research dumps and per-step traces.

Structsยง

PrefillResume ๐Ÿ”’

Functionsยง

check_budget ๐Ÿ”’
prefill_engine ๐Ÿ”’
Stream equal-sized prompt chunks so a short tail does not reread the whole store.
prefill_prompt ๐Ÿ”’
Fill the trunk and draft caches, then return the next token and draft chain.
prefill_row_batches ๐Ÿ”’
Use the decode row kernels for short prompts and explicit row-path checks.
qwen4_exp_gen_once ๐Ÿ”’
run