Routing Across Many Providers, and Why Receipts Must Be Verifiable

When one model is served by many independent operators, a router has more choices than any single API gives it: different prices, different hardware, different regions, different load at this moment. Choosing well is an optimisation problem. Proving afterwards what was chosen, what ran and what was billed is a trust problem. A router that solves only the first one asks its users to take every provider's word for the second.

What a router chooses on

A useful router reads four kinds of signal, and each comes with a question about who produced it.

• Price. Providers set their own prices. On Tenzro a provider publishes a rate card: a price per unit of metered work (input tokens, output tokens, cached tokens, pixel steps, audio milliseconds) with a per-call minimum. Each model announcement carries the card, so any node can price a request at any provider. tenzro_quoteInference prices one request at every provider serving the model, cheapest first, and includes each provider's region and the model's live load.
• Latency. Measured by the router itself, never taken from the provider's claim.
• Load. Whether the model is cold, warming or warm, requests in flight against capacity, and queue depth, from the provider's signed heartbeat.
• Standing. On Tenzro, standing is read from consensus state, not from local impressions: zero without a live serving bond, otherwise a bonded component plus an availability component scaled by the share of recent hours in which validators' service-level verdicts found the provider available. Every node ranks a provider the same way.

The router combines them through a strategy: lowest price, lowest latency, highest standing, random, or a weighted score of price, latency and standing (the default).

Hedging the tail

Averages hide the requests users notice. Dean and Barroso showed why at scale: rare slow responses dominate once a request fans out, and one of the most effective short-term tools is the hedged request, sending a second copy only once the first is late.

Tenzro's router hedges on the primary provider's observed p95 latency, clamped to a band (40 ms to 500 ms by default). Each provider keeps a streaming p95 estimate using the P² algorithm, which tracks a quantile in constant memory without storing samples. Waiting until the primary's own p95 means a healthy primary has already answered 19 times in 20, so a request still pending is a genuine tail case. Then:

• at most one hedge per request, never nested;
• never to a provider whose circuit breaker is open;
• only the winning response is billed and credited;
• callers that need strict single dispatch can opt out.

Failure detection runs on heartbeats. Every serving node gossips a signed heartbeat every two seconds, signed as the machine identity its provider announcement is pinned to. Three missed intervals mark the provider down, and the next clean heartbeat restores it. tenzro_getRouterMetrics exposes requests routed, hedges dispatched, hedges won and requests that missed their whole-request deadline, which is enough to tell rescued tails from routinely slow primaries.

The trust problem: three things a provider could misreport

Routing across operators you do not control exposes three gaps.

1. Which model ran. A provider could advertise a large model and serve a smaller one, or a quantised variant.
2. What it billed. Usage reported by the party being paid is an invitation to round up.
3. What a price index says. If prices and latencies come from providers' own announcements, the market is priced on numbers the sellers wrote.

Receipts and observations close these gaps, and each closes one.

Signed receipts

Every answer a Tenzro node serves locally carries a receipt. It is signed by the serving node's session key, which its hardware identity key certified (P-256, plus ML-DSA-65 where the hardware holds one). The receipt covers the catalog model hash and engine, the serving machine's identity, the request nonce, a commitment to the input, the output hash, the billed units, the sampling parameters and a commitment hash.

Model identity is anchored in content, not names. A model in the catalog is named by the Merkle root of its manifest, every file and every byte, and a node checks weights against the catalog hashes before it loads them. Model announcements also carry a hash of the served on-disk weights, so substitution is detectable before routing.

Billing is bounded by what the router saw. A forwarded call is billed from the provider's signed receipt, and every dimension is capped by what was actually sent and received (max_tokens, bytes). A provider cannot over-report usage beyond what crossed the wire.

Verifiable answers without re-running everything

A signature proves who answered, not that the answer came from the claimed weights. For that, a request can ask for a verifiable response. The node then commits, at each decode step, to a hash over the top-k logits, an approach in the style of TOPLOC, which reported detecting changes to model, prompt or precision with compact proofs of 258 bytes per 32 tokens. It is a commitment, not a zero-knowledge proof. Any node holding the same weights can recompute the rows, and a disputed commitment goes to a stake-weighted committee that excludes the provider. A node that cannot produce the commitment refuses the request rather than answering unverifiably.

Verification is a priced choice. tenzro_quoteInference takes a verification level (none, receipt, optimistic or tee) and quotes only the providers that can serve at that level, with the governed surcharge for it.

{"jsonrpc": "2.0", "id": 1, "method": "tenzro_quoteInference",
 "params": {"model": "<model id>", "units": {"input_tokens": 1200, "output_tokens": 400}, "verification": "receipt"}}

A price feed from observations, not announcements

The compute price feed is built from what routing nodes measured, not what providers claimed. After every routed call, the routing node signs the price it charged, the latency on its own clock, the token counts capped at the wire byte count, and whether the call succeeded, then gossips the observation. Only observations signed by active validators are admitted, so every number has a bonded author. An entry is attested only once at least two distinct validators have observed it, and an observer reporting on its own address is refused. Hardware class and region stay labelled as the provider's claims beside the observed numbers.

What this means for model routers

Optimise across providers, but treat every provider-supplied number as a claim until something independent backs it: your own clock for latency, consensus state for standing, a hardware-signed receipt for what ran and what it cost, a commitment for whether the output came from the claimed weights. A router built that way can buy from operators it has never met.

Where Tenzro fits

Built: rate cards and network-wide quotes; price, latency, standing and weighted strategies; p95-based hedging with circuit breakers and heartbeat failure detection; router metrics; hardware-certified signed receipts with wire-bounded billing; top-k logit commitments with committee disputes; and the validator-attested price feed. The network is live in preview; governance decides the steps to testnet and mainnet. We publish no routing performance figures here.