Structured outputs

response_format on a chat completion asks for JSON: {"type": "json_object"} for any JSON, or {"type": "json_schema", "json_schema": {"name", "schema", "strict"}} for JSON that follows a schema. The same field works on every model; what each provider is sent differs, and is listed below.

curl https://autonomousrelay.com/v1/chat/completions \
  -H "Authorization: Bearer $AUTONOMOUSRELAY_API_KEY" \
  -H 'content-type: application/json' \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [{"role": "user", "content": "Extract the invoice number and total from: INV-2041, $1,250.00"}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "invoice",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {"number": {"type": "string"}, "total_usd": {"type": "number"}},
          "required": ["number", "total_usd"],
          "additionalProperties": false
        }
      }
    }
  }'

What each provider receives

• OpenAI-protocol providers (OpenAI, xAI, Mistral, Groq and the rest): the field as you sent it.
• Gemini: JSON mode, with your schema as the response schema.
• Anthropic: a json_schema becomes Anthropic’s structured outputs. Bounds Anthropic does not enforce (minimum, maxLength, maxItems and the like) are left unchecked, and additionalProperties: false is assumed where you did not set it. A schema Anthropic cannot express — additionalProperties other than false, a recursive or external $ref — is sent without a format, and json_object has no Anthropic counterpart; in both cases the prompt is what asks for JSON.

Only endpoints that support it

Each catalogue endpoint declares the parameters it supports (Models & providers). With "provider": {"require_parameters": true} the request is routed only to endpoints that declare every parameter it sends — response_format among them, with tools, seed, the sampling parameters, stop, logprobs and reasoning. An endpoint that declares no parameter set at all is not excluded.

Repairing a reply that is not JSON

"plugins": [{"id": "response-healing"}]

A model asked for JSON sometimes wraps it in a markdown fence or a sentence, or leaves a trailing comma. The response-healing plugin repairs fences, surrounding text, trailing commas, comments, bare keys and unclosed brackets, and keeps the repair only if it parses: it never hands back text that is still not JSON, and never touches a reply that already is. A reply max_tokens cut short (finish_reason: "length") is returned as written — closing the brackets of half an object would invent an answer. Not on streams. When it changed the answer, the request’s metadata pipeline includes response_healing.

Hosted agents

An agent definition’s output.schema (a JSON Schema object of at most 8 KB) is sent as response_format on the run’s final answer, and the run’s result envelope then carries output_json beside the text.