Skip to content
Tenzro
← All tutorials
Tutorial · Inference and training

Fine-tune on agent trajectories

Turn your agents' tool-use conversations into a decentralised chat fine-tuning run on Tenzro Network 1, with LoRA adapters and rounds replayable from their seed.

Advanced40 min

Agents get better when the model under them has practised the exact tool calls and turn structure they use. Tenzro Train has a chat fine-tuning objective, ChatSft, built for this. You give it conversations in the OpenAI chat shape, including tool calls and tool results, and it trains the model on the assistant turns only. The run is decentralised like any other Tenzro training run, and every round is replayable from its seed, so every contribution can be checked.

A common pattern is distillation from a harness: run a small student model inside your agent, let a stronger teacher correct the turns it gets wrong, and fine-tune the student on the corrected runs.

Prerequisites

  • The tenzro CLI and a Tenzro account with TNZO for the reward pool. See Post a training task for the sponsor flow.
  • Conversation logs from your agent, or a way to export them.
  • jq for checking the data.
  • Background: Training for agents.

How the objective works

  • Each line of a shard is one conversation: {"messages": [...], "tools": [...]}.
  • Assistant turns may carry tool_calls; tool results are messages with role: "tool".
  • Every assistant turn is one training example. The conversation before it is rendered with the model's own chat template and masked out of the loss. The turn itself, tool calls and end-of-turn token included, carries the loss.
  • Each turn is rendered against its own context, which is exactly what the model sees at inference, even for chat templates that rewrite earlier turns.
  • ChatSft requires the Language modality. A task that pairs it with another modality is rejected when you post it.

1. Export trajectories as JSONL

Write one conversation per line. This example is one agent run that looks up a provider and rents a machine:

json
{"tools":[{"type":"function","function":{"name":"list_providers","description":"List compute providers","parameters":{"type":"object","properties":{"accelerator":{"type":"string"}},"required":["accelerator"]}}}],"messages":[{"role":"system","content":"You book compute for the user."},{"role":"user","content":"Find me a workstation GPU for tonight."},{"role":"assistant","content":null,"tool_calls":[{"id":"call_1","type":"function","function":{"name":"list_providers","arguments":"{\"accelerator\":\"workstation\"}"}}]},{"role":"tool","tool_call_id":"call_1","content":"[{\"provider\":\"0x7a1c...\",\"rate_wei\":\"...\"}]"},{"role":"assistant","content":"I found one workstation provider available tonight. Shall I book it?"}]}

This conversation yields two examples: the tool call, and the final answer.

2. Check the shard

Make sure each line parses and that every conversation has at least one assistant turn after some context.

bash
jq -c 'select((.messages | length) < 2 or ([.messages[1:][] | select(.role=="assistant")] | length) == 0)' \
  trajectories.jsonl | wc -l
# 0

Split large files into shards of similar size, one per trainer slot or more.

bash
split -n l/8 -d --additional-suffix=.jsonl trajectories.jsonl shard-

3. Publish the shards

bash
for f in shard-*.jsonl; do tenzro iroh publish --file "$f"; done
# tenzro://blob/<hash> for each shard

If your conversations are sensitive, post the task on the Confidential tier instead. The data is then sealed to attested enclaves and only unsealed inside them.

4. Write the task spec

Set modality to Language, the objective to "ChatSft", and name the base model in architecture.metadata.hf_repo. A LoRA adapter keeps the outer gradients small; LoraAlternating is the aggregation rule built for adapter runs. The spec below is abridged: add your sponsor address, the dataset's manifest hash and the creation time as in Post a training task.

json
{
  "task_id": "agent-sft-booking-2026-10",
  "sponsor_did": "did:tenzro:human:<your-id>",
  "architecture": {
    "family": "transformer-decoder",
    "param_count": 600000000,
    "modality": "Language",
    "fragment_count": 8,
    "dtype": "bf16",
    "metadata": {
      "hf_repo": "Qwen/Qwen3-0.6B",
      "lora": { "r": 16, "alpha": 32, "alternating": true }
    }
  },
  "tier": "Open",
  "aggregation": "LoraAlternating",
  "clip_l2_norm": 1.0,
  "sync_strategy": "Full",
  "quantization": "None",
  "delayed_apply": false,
  "pipeline": null,
  "trainer_count": 8,
  "quorum": 5,
  "inner_steps": 50,
  "max_rounds": 40,
  "grace_window_ms": 180000,
  "reward_pool": "<attoTNZO>",
  "dataset_ref": "tenzro://blob/<manifest-hash>",
  "min_throughput": null,
  "objective": "ChatSft",
  "metadata": { "seq_len": 2048, "batch_size": 4, "inner_optimizer": "adamw" }
}

For a larger base model on smaller GPUs, add "quantize": "nf4" to the lora block to train a QLoRA adapter over 4-bit weights.

5. Post the task

Posting escrows the reward pool, so it is an owner call signed with your account's hardware-rooted key.

bash
tenzro train post-task --spec agent-sft.json --rpc https://rpc.tenzro.xyz

Trainers with the language stack pick it up automatically if their node runs the trainer daemon. To run one yourself:

bash
pip install 'tenzro-trainer[language]'

tenzro-trainer run \
  --task-id agent-sft-booking-2026-10 \
  --trainer-did did:tenzro:machine:<controller>:<machine-id> \
  --shard-uri tenzro://blob/<shard-hash>

6. Follow the run and verify it

bash
tenzro train get-run --task-id agent-sft-booking-2026-10 --rpc https://rpc.tenzro.xyz
tenzro train decide-round --task-id agent-sft-booking-2026-10 --rpc https://rpc.tenzro.xyz

Because every round is replayable from its seed, anyone can re-run a trainer's steps on the same checkpoint and shard and compare activation commitments with tenzro train challenge-commitment. See Train and finalize.

7. Collect the adapter and serve it

When the last round finalises, read the receipt. It records the final model hash, every round's state root and each trainer's share.

bash
tenzro train get-receipt --task-id agent-sft-booking-2026-10 --rpc https://rpc.tenzro.xyz

Serve the fine-tuned model from your own node, then point your agent at it through /v1/chat/completions with the same tool definitions you trained on.

Next steps