Expand description
MLX affine quantization, CPU side. Layout (bits=4, group_size=64):
- weight: u32 [out, in/8], 8 nibbles per word, LSB-first along the input dim
- scales, biases: bf16 [out, in/64], one (scale, bias) per 64-element group
- dequant: x = scale * q + bias with q in 0..=15
Structs§
- QLinear
- Zero-copy view of one quantized linear layer’s tensors.