From the buyer's side, an AI call looks like one request and one answer. Underneath, a single task can involve several machines run by different operators. A primary provider stalls and a second one answers. A provider fails mid-session and another takes the conversation over. A large mixture-of-experts model has its experts spread across holders, so one forward pass touches many of them.
When all of those machines belong to one company, none of this is a settlement problem; it is an internal cost allocation. When they belong to independent operators, each of whom expects to be paid for its own part and none of whom has a contract with the others, it becomes one. This article explains how Tenzro, the open settlement and coordination layer underneath those operators, routes work across them, how each operator is settled for exactly its share, and what each party to the job can see.
Routing across independent operators
Every Tenzro node accepts requests on OpenAI-compatible routes and can serve them itself or forward them to a provider that serves the model. Providers announce the models they serve and attach their own rate card to each announcement, so any node can price a request at any provider before sending it.
The router chooses a provider by one of several strategies: lowest price, lowest latency, highest reputation, or a weighted score of price, latency and standing, which is the default. It reads each provider's live state from the signed heartbeat that provider gossips every 2 seconds: whether a model is cold, warming or warm, how many requests are in flight, and how deep the queue is. A provider whose heartbeats stop for 6 seconds, or whose latest heartbeat marks it degraded, is treated as down until its next clean heartbeat. A heartbeat signed by any other key, altered in flight or outside the window is dropped.
A buyer can also ask for quotes first. A quote prices one request at every provider serving the model, cheapest first, with any surcharge for the verification level the buyer asks for, counting only providers that can meet that level.
Hedging and failover, and who gets paid
Tail latency dominates the experience of distributed services, as Dean and Barroso showed: when a request depends on one server, that server's slowest moments become the user's. The standard remedy is to hedge, sending a second copy of a slow request elsewhere and taking whichever answers first.
Tenzro's router hedges at the primary provider's own observed 95th-percentile latency, tracked with the constant-memory P² estimator and clamped to a band of 40 to 500 milliseconds by default. If the primary has not answered by then, the same request goes to the next-best provider, and the first answer wins. There is at most one hedge per request, a provider whose circuit breaker is open is never chosen as a hedge target, and a caller can opt out.
Hedging raises an obvious settlement question: two operators started the same work. The answer is simple and fixed. Only the winning response bills the buyer and credits the provider's reputation. The loser's request is dropped. The buyer pays for one answer.
If both the primary and the hedge fail, the router excludes both and retries the next-best providers up to a set limit. For streaming responses, if a provider drops mid-generation, the gateway sends the same request, with the text emitted so far and the original sampling parameters, to another provider serving the same model and continues the stream in place. That continuation is a single bounded retry; if it also fails, the provider is penalised and the stream ends.
Sessions and standby successors
Long conversations are a different case. A multi-turn session can ask for continuity. When the session starts, the serving node assigns a successor: a node that serves the same model, satisfies the session's jurisdiction pin, and is run by another operator or at another site. The session's cache state is sealed to the successor's key and sent to it periodically. If the primary's heartbeats mark it down, the next turn goes to the successor, which rebuilds the cache and serves without starting over.
Holding capacity in reserve has a cost, and Tenzro settles it explicitly. An operator can offer standby at a fee per hour. Such a node holds a session as successor only against a funded stream escrow the buyer opens to it. The successor claims the stream as it accrues; the buyer can stop it at any time. No node keeps a private charge book for it.
Many holders, one forward pass
The most fragmented case is a mixture-of-experts model whose experts are held by different providers. Each forward pass routes token batches to the holders of the experts the gate selects, and combines the results.
Every remote expert execution is signed. The holder commits to its output rows and signs the commitment, together with the model, layer, expert, token indices and a hash of the exact input it received, with a session key certified by its hardware identity key. The router verifies each receipt before accepting the batch. A missing or mismatched receipt is treated as a failure: the holder is retired for that batch, its reputation is penalised, and the work goes to a standby holder.
After the forward pass completes, the router settles the remote work per holder in the background, so response latency never waits on settlement. Each holder is paid at its own advertised price. A sample of verified batches is stored in full, about one in sixty-four, so that any node holding the same expert can re-execute it and dispute a false receipt.
When a job spans several payments
Some jobs need several payments to succeed or fail together. Tenzro's settlement plans group legs under one split rule, and record one commit-or-abort decision for the plan; on commit, the split divides the value among its recipients in the same execution. A plan whose legs are all on Tenzro decides in the transaction that opens it. A quote can name the split rule a plan must use, and a plan that names another is refused.
What each party sees
A multi-operator job is only manageable if each party sees what it needs and nothing it should not.
The buyer sees one answer, with the provider's name and a receipt where the call path produces one, and a payment record that binds the request digest, the quote digest and the price. The buyer's access token authenticates it to the node it called and is never sent on to a provider.
Each operator sees the requests routed to it and is paid for its own work at its own rate card. When a call is forwarded to it, the call is billed from the operator's own signed receipt, checked against the key its machine registered, with every dimension capped by what was actually sent and received. An operator cannot inflate its share, and no other operator's figures enter its settlement.
Anyone choosing providers sees the same public picture: each provider's model announcements, rate cards, live load and coarse region, which is a country code or a continent and never anything finer.
An auditor, given an agent's evidence pack, sees the agent's payments proven against a certified state root, and can check them offline without trusting any of the operators involved.
What this means for model routers and builders
If you route traffic across several providers today, the questions to ask of any multi-provider setup are the ones above. When two providers start the same work, who is paid? When a session moves, who pays for the standby? Whose figures set the bill, and what caps them? Can a third party check the result without your dashboard? A routing layer that cannot answer these questions per call leaves the reconciliation to you.
Where Tenzro fits
Tenzro is the settlement layer for AI work across independent operators. Everything described here is built, per the Tenzro repository documentation: routing by price, latency, reputation or weighted score; 2-second signed heartbeats with a 6-second down window; tail-latency hedging billed only to the winner; failover and streaming continuation; session successors with standby fees paid from a buyer-funded stream escrow; signed, verified receipts for remote expert execution with per-holder settlement; settlement plans with one commit-or-abort decision; bounded billing from signed receipts; and offline-verifiable evidence packs.
