Struct CpuModel 

pub struct CpuModel<'a> {
    pub p: &'a Packed,
    pub embed: QLinear<'a>,
    pub layers: Vec<Layer<'a>>,
    pub final_mixer: HcWeights<'a>,
    pub lm_head: QLinear<'a>,
    pub mtp: Option<MtpWeights<'a>>,
}

Fields

p: &'a Packed

embed: QLinear<'a>

layers: Vec<Layer<'a>>

final_mixer: HcWeights<'a>

lm_head: QLinear<'a>

mtp: Option<MtpWeights<'a>>

Implementations

impl<'a> CpuModel<'a>

pub fn load(p: &'a Packed) -> Result<Self>

impl<'a> CpuModel<'a>

pub fn new_state(&self) -> State

pub fn mtp_forward( &self, token: u32, hyper_in: &[f32], pos: usize, state: &mut State, ) -> Result<(Vec<f32>, Vec<f32>)>

MTP draft head at trunk position pos: pairs the trunk’s wide residual for that position (hyper_in) with the embedding of the token that follows it, appends to the head’s KV cache (which must hold exactly pos entries) and returns (logits for the token after token, the head’s wide residual for chained drafting).

pub fn forward_token(&self, token: u32, state: &mut State) -> Result<Vec<f32>>

Forward one token at state.pos; returns logits.

fn gated_residual( &self, w: &HcWeights<'_>, hyper: &[f32], eps: f32, ) -> (Vec<f32>, Vec<f32>)

Gated residual read: returns the mixed hidden input for the block and the per-stream injection weights (empty for the final mixer).

fn moe(&self, w: &MoeWeights<'_>, x: &[f32]) -> Vec<f32>

fn attention( &self, w: &AttnWeights<'_>, x: &[f32], cache: &mut KvCache, pos: usize, ) -> Result<Vec<f32>>

fn select_tokens( &self, w: &AttnWeights<'_>, iq: &[f32], cache: &KvCache, pos: usize, ) -> Vec<usize>

QSA indexer for one query at pos over the causal prefix: blocks of compress_ratio tokens scored by relu(q . pooled_key) summed over index heads; the top budget / ratio blocks plus the incomplete tail are visible. Returns visible token indices in ascending order.

fn deltanet( &self, w: &DeltaWeights<'_>, x: &[f32], state: &mut DeltaState, ) -> Vec<f32>

Gated DeltaNet, recurrent form, f32 state; sigmoid output gate.

fn ngram_ids(&self, w: &PleWeights<'_>, tokens: &[u32]) -> Vec<u64>

pub fn ngram_ids_from( c: &Qwen4ExpConfig, multipliers: &[i64], head_offsets: &[u64], head_sizes: &[u64], tokens: &[u32], ) -> Vec<u64>

Hashed n-gram ids for the current (last) token: bigram heads then trigram heads. A token before the current segment (past an EOS, or before the start) reads as EOS. Multiplication wraps like torch int64.

fn ple( &self, w: &PleWeights<'_>, hyper: &[f32], state: &mut State, ) -> Result<Vec<f32>>

PLE block: n-gram embedding, per-stream key/query gating, value, dilated depthwise conv with SiLU. Returns the hc_hidden-wide addend.