Skip to content
KataraKatara documentation

Images, audio, video and PDFs

A chat message’s content can be an array of parts. Besides text, a part can carry an image, a recording, a video or a PDF. Each needs a model that reads it: /v1/models lists vision, audio, video and documents under a model’s capabilities. The Deep, Standard and Light models read all four; a backup model reads images. Katara Cortex sends a request with media only to a candidate that reads it.

Terminal window
curl https://api.katara.com/v1/chat/completions \
  -H "Authorization: Bearer $KATARA_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<JSON
{
  "model": "katara/cortex",
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "Summarize the call and check it against the contract."},
    {"type": "input_audio", "input_audio": {"data": "$(base64 < call.mp3)", "format": "mp3"}},
    {"type": "file", "file": {"filename": "contract.pdf", "file_data": "data:application/pdf;base64,$(base64 < contract.pdf)"}}
  ]}]
}
JSON

Part shapes

Section titled “Part shapes”
PartShape
Image{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}. PNG, JPEG, WebP, GIF, HEIC or HEIF as a base64 data URL, or an https URL the model fetches itself.
Audio{"type": "input_audio", "input_audio": {"data": "<base64>", "format": "mp3"}}. format is one of wav, mp3, aiff, aac, ogg, flac, m4a, opus, webm.
PDF{"type": "file", "file": {"filename": "a.pdf", "file_data": "data:application/pdf;base64,..."}}. Up to 1,000 pages.
Video{"type": "file", "file": {"filename": "a.mp4", "file_data": "data:video/mp4;base64,..."}}. MP4, MPEG, QuickTime, WebM, AVI, FLV, WMV or 3GPP.

These are the shapes the OpenAI client sends, so client.chat.completions.create works unchanged. A file part carries file_data, or the file_id of a file uploaded once with POST /v1/files (Long conversations).

Limits

Section titled “Limits”

One request carries at most 20 MB of media, counted after base64 decoding. More is refused with 413 media_too_large. A request paid per call with x402, without a key, is limited to a 4 MB body in all.

A part of the wrong shape, an audio format or file type not in the lists above, or a PDF of more than 1,000 pages is refused with 400 invalid_request. Media sent to a model that does not read it is refused with 400 feature_not_supported. Nothing is charged for a refused request.

Price

Section titled “Price”

Media is billed as input tokens at the model’s input price, counted by the model from its length, not its size: per image, per second of sound or video, and per PDF page. The Deep model counts 1,120 tokens for an image, 560 for a PDF page, 25 a second of sound and about 100 a second of video. The usage in the reply has the count.

The quote for a request paid with x402, and the hold on account credits, are made before the model counts. They read the length of each recording from its file header and the pages of each PDF, and allow for the highest resolution the model may use, so they are larger than the charge. You are charged what the reply cost.