Struct MemoryStats 

pub struct MemoryStats {
    pub metal_allocated_bytes_observed: u64,
    pub mapped_weight_buffer_bytes: usize,
    pub kv_index_capacity_bytes: usize,
    pub context_capacity_tokens: usize,
    pub mtp_enabled: bool,
    pub expert_pool_bytes: usize,
    pub device_working_set_bytes: usize,
    pub allocation_limit_bytes: Option<usize>,
    pub prefill_reserved_bytes: usize,
    pub expert_pool_slots: usize,
    pub resident_experts: usize,
}

Capacities and observations, not a partition of process memory. CPU prefix checkpoints and session history are reported separately by the server.

Fields

metal_allocated_bytes_observed: u64

mapped_weight_buffer_bytes: usize

Length of the shared weight buffer, counted once for all tensor views.

kv_index_capacity_bytes: usize

context_capacity_tokens: usize

mtp_enabled: bool

expert_pool_bytes: usize

Configured pool capacity; the residency-set backend may use fewer slots.

device_working_set_bytes: usize

allocation_limit_bytes: Option<usize>

prefill_reserved_bytes: usize

expert_pool_slots: usize

resident_experts: usize

Trait Implementations

impl Clone for MemoryStats

fn clone(&self) -> MemoryStats

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for MemoryStats

impl Debug for MemoryStats

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for MemoryStats

fn default() -> MemoryStats

Returns the “default value” for a type.

impl Serialize for MemoryStats

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.