Telcos as AI edge clouds: inference inside the latency budget

For a decade, telecom operators have been told that edge computing would be their next growth market. Multi-access edge computing became a standards effort at ETSI, and operators built edge sites. But much of the predicted demand never arrived, because few applications needed computation that close to the user. AI agents, voice assistants, vision on cameras and robots, and real-time translation change that. They are interactive, they are sensitive to latency, and they are expensive to run in the wrong place.

This article covers where AI inference should run along the path from device to cloud, what latency budgets actually allow, and what an operator needs in order to sell edge inference to buyers outside its existing customer base.

Physics sets the floor

Light in optical fibre travels at roughly two-thirds of its speed in a vacuum. That works out to about 5 microseconds per kilometre one way, the planning figure used in 3GPP TR 25.853. A round trip of 1,000 km of fibre therefore costs about 10 milliseconds before any router, queue or GPU is involved. Real paths are longer than straight lines, so the true figure is usually higher.

For a chat response that takes two seconds to generate, 10 ms is noise. For a voice agent, it is not. A spoken exchange needs speech recognition, a language model, possibly a tool call and speech synthesis, all before the pause starts to feel unnatural. Every round trip to a distant region comes out of the same budget. For a robot or a camera making a control decision, the budget is tighter still.

The conclusion is not that everything must run at the edge. It is that each step of a request should run as close to the user as that step needs.

Split the request, not just the traffic

Most AI applications are no longer one model call. They are model ladders. A cheap step classifies, routes, retrieves, transcribes, filters or checks policy, and a large model is called only when the request needs one. That structure suits telecom networks well:

An edge site does not need to host the frontier model to be valuable. It needs to host the steps that are called most often and are most sensitive to latency, and to call the regional tier for the rest. Done well, many requests never leave the edge, and those that do arrive at the regional GPU already transcribed, filtered and routed.

The tail matters more than the average

Telecom engineers think in percentiles, and AI systems should too. As Dean and Barroso showed, in a request that fans out to several services, the slowest component sets the user's experience. A ladder with three model calls inherits the tail of each.

Two techniques help. The first is hedging. If the chosen provider has not answered by the time a healthy one usually would, the same request goes to a second provider, and the first response wins. The second is fast failure detection. Providers that stop signalling health within seconds are removed from routing before users notice. Both work better when many edge and regional providers serve the same model, which is an argument for open pools rather than closed ones.

Selling to buyers you do not know

Operators already have network APIs. The GSMA Open Gateway initiative, launched in February 2023, exposes capabilities such as location, identity and quality on demand through APIs defined in the CAMARA open-source project. Orange's developer guide reports that more than 73 major operators covering about 80% of the market had joined by November 2025. On the radio side, the AI-RAN Alliance, formed in 2024, passed 100 members in 2025 and is working on running AI and RAN workloads on shared accelerated infrastructure.

Edge inference needs the same kind of openness, plus three things network APIs do not provide on their own.

Discovery with live state. An agent or application must find which edge sites have a given model warm, how loaded they are, and where they are, at country or regional level rather than exact site location. Coarse location is enough for routing and protects the operator's network topology.

Trust without a contract. A developer in another country will not sign a contract to send a few thousand requests to an edge site. It will send them if each response carries a signed receipt naming the model and the serving machine, and if billing is capped by what was actually sent and received.

Settlement per request. Revenue from edge inference arrives in very small amounts from many buyers. Per-request settlement, paid from funds the buyer has already committed, is the only model that works at that scale.

Regional differences

The case for telco edge AI varies by region. In the EU, data residency and sovereignty rules favour inference on domestic infrastructure, and operators are natural holders of it. In APAC, dense urban markets and high mobile use make latency the main driver. In MENA, national programmes often require in-country processing for public-sector and regulated workloads. In Africa and Latin America, where international transit can be costly and hyperscale regions are farther away, local edge inference can matter as much for cost and availability as for latency.

What this means for telcos

Telcos should treat AI inference as a workload to host, not a product to build from scratch. Put GPU capacity where the latency budget demands it. Expose it to an open pool of demand with signed receipts and per-request settlement, and let routing decide which requests stay at the edge. Use national presence for sovereignty: an edge site that can enforce a jurisdiction rule is worth more to a regulated buyer than one that cannot.

Where Tenzro fits

Tenzro is an open network where machines from edge devices to GPU servers sell AI work. It is live in preview, and governance decides the steps to testnet and mainnet. Relevant to operators, what is built:

• One node, many tiers. One codebase builds release variants: a light build for small edge devices, which forwards model calls to providers on the network, a standard build for CPU servers, and a GPU provider build. A MEC site with accelerators runs as a GPU provider.
• Discovery with coarse location. Providers declare a region at country or wide-region level, never finer. Discovery reports per-model load and warm-up state from signed heartbeats sent every two seconds.
• Tail control. The router hedges a request to a second provider at the primary's observed p95 latency and skips providers whose heartbeats have stopped for six seconds.
• Jurisdiction pins. Requests can be restricted to providers in a declared country or bloc, failing closed.
• Signed receipts and bounded billing. Responses carry hardware-certified receipts, and billing is capped by what was sent and received.
• Many modalities. Speech recognition, embeddings, detection, segmentation and forecasting run alongside language models.

Planned: a harness host role in which edge devices host agent orchestration and small local models and call GPU providers for large models, with every party paid from the caller's budget.