Routing

When more than one provider can serve a model, the router picks between them on price, latency and live health. The provider object lets you constrain that choice.

{
  "model": "meta/llama-3.3-70b",
  "messages": [{"role": "user", "content": "Hello"}],
  "provider": {
    "only": ["together", "fireworks"],
    "sort": "price",
    "max_price": {"prompt": 0.9}
  }
}

Fields

• order — try these providers first, in this order.
• only — restrict to these providers.
• ignore — exclude these providers.
• sort — price, throughput, latency or exacto (most reliable tool calls first, then fastest, then cheapest). With tools and no sort, Exacto orders the endpoints once any of them has a measured tool-call rate.
• max_price — a per-SKU ceiling in USD per million tokens.
• quantizations — restrict to given quantizations. bf16 and int4 are not the same model.
• require_parameters — only endpoints supporting every parameter you sent.
• preferred_max_latency — seconds; endpoints measured slower are tried last, never excluded.
• sort as {"by": "price", "partition": "none"} — one order across every model in models, instead of each model in turn.
• zdr and data_collection — accepted as false and allow. Requiring zero data retention or no training is a 400: whether a provider keeps or trains on your prompts is set by your own agreement with it, and we do not record it. See What we store.

Any free model: autonomousrelay/free

Send "model": "autonomousrelay/free" and the router picks a free model at random among those that can answer the request: text out, image input when you send an image, a context window that fits your prompt, and one you can route to (a BYOK credential for its provider, and allowed by your guardrails and privacy settings). Two more free models are kept as fallbacks. The response’s model names the one that answered; your logs record autonomousrelay/free as what you asked for. With none available the answer is a 404 saying what to add.

Let the router choose: autonomousrelay/auto

Send "model": "autonomousrelay/auto" and the router picks a paid model from what your request needs (text out, image input when you send an image, a context window that fits), what you can route to, and a cost tier, by output price per million tokens: low up to $2 (the default), medium to $10, high to $30, xhigh to $60, max above. Within the tier it takes the most expensive model first, then the newest — price is the only quality signal the catalogue has. Two more from the same tier are kept as fallbacks; if the tier has nothing you can route to, the nearest cheaper tier is used, then the nearest dearer one.

Set it with {"id": "auto-router", "cost_tier": "high", "allowed_models": ["anthropic/*"], "excluded_models": ["*/claude-*-mini"]} in plugins; * matches any run of characters. The router reads nothing in your prompt and keeps nothing about it, so the same request picks the same model, turn after turn. No router fee. The response’s model names the one that answered; your logs record autonomousrelay/auto.

Preferences reorder; constraints reject

sort changes the order of candidates. only, ignore, max_price and quantizations remove them. If a constraint removes every candidate the request fails with 503 and names the filter that emptied the set — it never quietly falls through to something you excluded.

{
  "error": {
    "code": 503,
    "message": "No endpoints satisfy the request",
    "metadata": {"filters": ["provider.only=[together]", "quantizations=[fp8]"]}
  }
}

Guardrails outrank the request

Organisation guardrails can restrict providers, models and data regions, and a request cannot widen them — only narrow them further. If your request asks for a provider a guardrail excludes, the guardrail wins and the error says so.