Documentation
Run mise run docs on macOS to build the
website in _site/.
.gitattributes selects the files and SUMMARY.md sets the navigation.
Changes to main publish automatically.
- Rust API: generated library reference, including private items.
- Running: CLI, HTTP API, prefix caching and precision modes.
- Server configuration: TOML, memory policy and control CLI.
- Statistics: summaries, detailed JSON, and the live dashboard.
- Storage: paths, downloads and packed expert stores.
- Model index: named models, imports, retention and cleanup.
- Engine: model execution, expert streaming and synchronization.
- Developer options: diagnostic environment variables.
- Validation: tests, linters and known limitations.
- Serving architecture: ownership, scheduling and cancellation.
- Benchmarks: suite, reports and pelicans.
- Saved results: measurements and generated answers.
Repository layout
| Directory | Contents |
|---|---|
src/ | CLI, server, storage and shared runtime support |
src/server/ | HTTP framing, routes, request policy, scheduling, sessions and output |
src/control/ | Control state and statistics protocol |
src/cli_output/ | Terminal reports and dashboard |
src/runner/ | Resumable decoding and optional diagnostics |
src/model/ | Checkpoint descriptions, model index and preparation |
crates/model-data/ | Container readers, tensor encodings and byte sources |
src/qwen4_exp/ | Model config, packer, CPU reference and GPU execution |
kernels/ | Metal fragments grouped by common primitives and model subsystem |
tests/unit/ | Engine child-module tests, mirroring the source hierarchy |
xtask/src/, xtask/tests/ | Rust automation and its tests |
xtask/templates/ | HTML template used to generate benchmark galleries |
benchmarks/ | Workload definitions and measurement instructions |
results/ | Reviewed measurements; new local runs are ignored |
Model weights and packed stores use the configured data directory.
Building the documentation
Install Rust and mdBook with mise install rust github:rust-lang/mdBook, then run
mise run docs (or cargo xtask docs). The build generates documentation for all
workspace libraries, fails on rustdoc warnings, and publishes the complete
reference under _site/api/ after building the book. It needs macOS for the
Apple framework dependencies; model weights are not required.
To preview the combined site, run python3 -m http.server --directory _site 8000
and visit http://localhost:8000/. Rebuild to pick up source changes.
For just the Rust reference, run:
cargo doc --locked --workspace --no-deps --document-private-items --open