Skip to main content

prefill_rows

Function prefill_rows 

Source
fn prefill_rows(
    gpu: &mut Gpu<'_>,
    rows: &[u32],
    following: Option<u32>,
    drafts: usize,
    engine: bool,
) -> Result<PrefillResume>
Expand description

Advance one prompt chunk and retain the predictions needed to resume decode.