Serve a distributed MoE model

Start from the catalog-side shape — how many experts the model has, how many are active per token, and the layer layout — so you know which experts you can hold.

tenzro moe catalog-shape --model qwen3.5-397b-a17b

Load per-expert FFN weights (gate/up/down projections) into the node's expert runtime, keyed by (model, layer, expert). Shards can come from a local safetensors file or a content-addressed tenzro://blob/<hash> URI fetched over iroh-blobs. Load the gating network for each layer you participate in.

tenzro moe load-expert \
  --model qwen3.5-397b-a17b --layer 0 --expert 1 \
  --file ./l0-e1.safetensors

tenzro moe load-gate \
  --model qwen3.5-397b-a17b --layer 0 \
  --file ./l0-router.safetensors

Show the node's resident experts and gates, plus their memory footprint.

tenzro moe status

The shard map aggregates every holder's declaration into a network-wide view: expert coverage, replication per expert, which experts are hot (committed TPS over the threshold), and which are under-replicated. The default policy asks every active expert to be held by at least two distinct providers.

tenzro moe shard-map --model qwen3.5-397b-a17b

Before running a request, you can inspect how it would fan out. Feed a JSON file of top-k routing decisions — [{token_index, experts: [{layer, expert}]}] — and the planner returns the per-holder batches it would dispatch.

tenzro moe plan-dispatch \
  --model qwen3.5-397b-a17b \
  --routings ./routings.json

A distributed forward gates locally, hands the routing decisions to the planner, sends each per-holder batch — executed locally when this node holds the expert, over the holder's iroh QUIC endpoint when advertised, over HTTP otherwise — and recombines per-token outputs weighted by the gate probabilities.

tenzro moe forward \
  --model qwen3.5-397b-a17b \
  --hidden ./hidden.f32 \
  --d-model 5120 \
  --out ./combined.f32