Module lowbit 

Building the low-bit expert stores from the 4-bit one, in process. experts2.bin and experts3.bin coexist beside the source store; choosing or rebuilding one precision leaves the other file intact.

experts.bin holds each expert’s three projections as MLX affine 4-bit codes (w = q * scale + bias, groups of 64). A low-bit store keeps the same scales and biases and drops the bottom bits of the codes, reconstructing at the midpoint of the range each new code covers: q >> 1 with (2q + 0.5) at three bits, q >> 2 with (4q + 1.5) at two.

The layout is the part that matters. A kernel that peels codes one at a time costs 2.3x the 4-bit kernel’s time even though it reads fewer bytes, which makes the whole exercise pointless; these layouts put a word’s codes where one masked cast (& 0x03030303, & 0x01010101) yields four codes that are CONSECUTIVE in the deinterleaved input stream, exactly as the 4-bit nibble trick does. See fn_q2_rows_h and fn_q3_rows_h in kernels/qwen4_exp/experts.metal, which must agree with the packing here bit for bit.

Structs

Layout

Expert record layout shared by the packer, prefill, and decode. Low-bit records keep the base order: three weight matrices followed by six scale/bias blocks copied verbatim from the 4-bit record.

Constants

PAGE 🔒

Functions

announce_build 🔒

build 🔒

create_output 🔒

ensure

Make sure the bits-bit store exists next to experts.bin, building it from the 4-bit records if it is missing, the wrong size, or in the older code-at-a-time layout. Returns the layout either way.

ensure_many

Build missing targets together, reading each Q4 source record once. Duplicate precisions are ignored; valid cached stores are left untouched.

ensure_selected 🔒

ensure_with_policy 🔒

is_usable 🔒

Read-only readiness check shared by indexed preparation and the runtime.

pack_matrix 🔒

Repack one matrix of codes 4-bit codes (as u32 words of 8 nibbles) into dst.

pack_record 🔒

pack_records 🔒

paths 🔒

q2_slots 🔒

Bit position of each of a 32-code chunk’s codes in the 2-bit layout: word index (0 or 1) and the offset within it. One & 0x03030303 at shift 2p then yields the four codes of group p, and those four are consecutive in xe (groups 0 and 1) or xo (groups 2 and 3).

q3_slots 🔒

Three-bit codes split into their upper two bits and lowest bit. The upper pair uses the 2-bit layout; the lowest bit goes into a third word at 8 * byte + 4 * word + group. The shader reconstructs code = 2 * upper + lowest from the corresponding masked casts.

spot_check 🔒

Re-pack a handful of records spread through the file and compare with what is stored. Reads about 15 MB, so it costs milliseconds.

usable 🔒

Whether a usable store of this layout is already on disk. The manifest is only a claim, so a few records are re-packed and compared byte for byte: that is what catches a store written by an older packing, which would otherwise be read as noise by the kernel.