Skip to main content

prefill_prompt

Function prefill_prompt 

Source
fn prefill_prompt(
    gpu: &mut Gpu<'_>,
    cpu: Option<&CpuModel<'_>>,
    cpu_state: Option<&mut State>,
    ids: &[u32],
    n_draft: usize,
    resume: Option<PrefillResume>,
) -> Result<PrefillResume>
Expand description

Fill the trunk and draft caches, then return the next token and draft chain.