Tool calling

Tool calling works the way it does with OpenAI: send tools, get tool_calls back, answer with a tool message. Coding agents and MCP clients that speak the OpenAI API run unchanged.

Katara sits between your client and the models Cortex routes to: open models on providers’ hardware and vendor models. Every reply is checked and re-serialized before it reaches you, so what you receive is the same shape whichever model served it.

What you receive

• One choice per reply. n above 1 is refused.
• tool_calls entries have an id starting with call_, type: "function", a non-empty function.name and function.arguments that parse as a JSON object.
• In a stream, each tool call arrives whole in a single chunk with its index. Text streams as it is produced; a call is held until it has been validated.
• finish_reason is tool_calls when a call was made, stop, length, or content_filter.
• usage is always present, on the final chunks of a stream too.
• Assistant text never contains a model’s raw tool syntax. If a model writes its call in its own markup instead of the structured form, Katara either recovers it into a proper tool_calls entry or refuses the reply. It is never passed through as text.

What you can send

• tool_choice: auto, none, required, or a named function. With required or a named function, a reply without that call is not delivered and the next model is tried; with none, neither is a reply that calls a tool.
• parallel_tool_calls: false gets at most one call per reply.
• strict: true on a tool: the arguments are checked against the tool’s whole schema (required fields, types, enums, nested objects, $ref, anyOf), and a call that does not fit is not delivered.
• Tool names up to 128 characters, and many tools with large schemas. A model that takes fewer, such as one with a 64-character name limit, is passed over for that request.
• Tool results that carry images or files as well as text: send the tool message’s content as a list of parts, as for a user message (Images, audio, video and PDFs).
• Custom tools, {"type": "custom", "custom": {"name", "description", "format"}}, whose call carries one string of input instead of JSON arguments. The call comes back as a custom tool call; a grammar in format is given to the model as part of the description.

/v1/models lists what Cortex takes under capabilities.tool_contract: the accepted tool_choice modes (auto, none, required, named) and max_tool_name_length.

When a model gets a call wrong

A model can produce a call that cannot be executed: a truncated argument document, a tool that was never offered, arguments that are not JSON or that break a strict schema. Katara does not deliver those. The same applies to a model that has fallen into a loop and keeps repeating the same paragraph: the reply is cut off instead of running to the token cap.

• If nothing had been sent to you yet, the request is retried on another candidate. You see one reply and pay for one reply.
• If text had already streamed, the stream ends with an error chunk with code provider_output_invalid. Nothing is charged for that attempt.
• If no candidate produced a valid reply, the request fails with 500 provider_output_invalid and nothing is charged.

A provider whose model keeps producing calls that need recovery or refusal stops receiving tool requests, however cheap it is.

Example

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.katara.com/v1", api_key=os.environ["KATARA_API_KEY"])
tools = [{"type": "function", "function": {
    "name": "write_file",
    "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}},
                   "required": ["path", "content"]}}}]
reply = client.chat.completions.create(
    model="katara/cortex",
    messages=[{"role": "user", "content": "Write weather.py that prints Florida's weather."}],
    tools=tools,
)
call = reply.choices[0].message.tool_calls[0]
print(call.function.name, call.function.arguments)

Web search

Katara can search the web for the model. On Chat Completions, add katara_web_search at the top level of the request; {} turns it on with the defaults:

{
  "model": "katara/cortex",
  "messages": [{ "role": "user", "content": "What changed in the latest Go release?" }],
  "katara_web_search": { "max_uses": 3, "allowed_domains": ["go.dev"] }
}

OpenAI’s web_search_options asks for the same search, with its user_location; katara_web_search wins when a request sends both.

max_uses lowers the cap of five searches per request. allowed_domains or blocked_domains, not both, narrows the results; user_location is accepted. The model gets a search tool named katara_web_search (katara_web_search_2 if one of your tools already has that name) and decides when to search. Katara runs each search, returns five results to the model, and calls it again until it answers. Once the searches are used up, the last call gets their results as text and is asked to answer from them. The search tool’s calls never appear in tool_calls; if the model also calls your tools in the same step, those come back as usual.

The reply’s katara object lists every search under web_searches (its id, query, results with url, title, page_age and content, and an error when it failed) and the charge under web_search (requests and its cost). In a stream, each search is announced as it starts by a chunk with an empty choices array and a top-level katara_web_search_call entry (the call’s id and query), and its results follow in a chunk of the same kind with a top-level katara_web_search entry, before the text that uses them. Messages and Responses take their own format’s web search tool and return the results in that format.

A search costs 0.006 USDC, or 0.006 in credits. A failed search is never charged: the model is told it failed (invalid_input, query_too_long, too_many_requests, unavailable, max_uses_exceeded, or insufficient_quota when the balance cannot pay for it) and the answer goes on without it. The model calls that read the results are charged at their token cost, and usage sums them.

The first model call is routed as any request is, Katara Community included. Once the model asks to search, the rest of the request is served by managed models, never by a Katara Community provider.

A request that ends without an answer charges nothing for its searches or for the model calls after the search began. That covers a model that searched and still wrote nothing, because it asked for yet another search or its reply was filtered empty, which ends with 500 provider_output_invalid. The error carries the katara object, with the searches that ran in web_searches and cost_usd (or cost_usdc) at 0. If a Katara Community model is the one that asked for the search, its reply has already been paid to its provider and stays charged, and the cost shows that reply’s charge.

Search is not offered, and the request is served without it, on the Test network, without a key (x402), on a marketplace-only key, in agent runs, for a request that names a Katara Community model, or when tool_choice is none or names a function. A request that names both domain lists is refused with 400 invalid_web_search.

Billing

A tool call is output you received, so it is billed: the tool name and the argument document count as output tokens. See Usage and cost.