Skip to main content

prefill_row_batches

Function prefill_row_batches 

Source
fn prefill_row_batches(
    gpu: &mut Gpu<'_>,
    cpu: Option<&CpuModel<'_>>,
    cpu_state: Option<&mut State>,
    ids: &[u32],
    n_draft: usize,
    argmax_lines: Option<&mut Vec<String>>,
) -> Result<PrefillResume>
Expand description

Use the decode row kernels for short prompts and explicit row-path checks.