Streaming

odnoga proxies SSE (Server-Sent Events) end-to-end. Add "stream": true and consume the OpenAI-shaped delta stream.

cURL

curl -N "$ODNOGA_BASE_URL/chat/completions" \
  -H "authorization: Bearer $ODNOGA_API_KEY" \
  -H "content-type: application/json" \
  -H "x-airouter-end-user: user_123" \
  -d '{
    "model": "gpt-4o-mini",
    "stream": true,
    "messages": [{"role":"user","content":"count to 5"}]
  }'

You receive data: { … }\n\n chunks until a final data: [DONE].

Node (OpenAI SDK)

const stream = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  stream: true,
  messages: [{ role: 'user', content: 'count to 5' }],
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}

Long calls and the 150-second limit

The platform in front of odnoga closes any response that sends no bytes for about 150 seconds, and that includes silence in the middle of a stream. A reasoning model at high effort can think longer than that before its first token.

Streamed calls are protected automatically: while the model is silent, odnoga sends an SSE comment every 15 seconds,

: keepalive

which every conforming client (the OpenAI and Anthropic SDKs, EventSource, fetch readers that look only at data: lines) ignores. You don't need to do anything, apart from making sure your own HTTP client's read timeout is longer than the call.

Streaming removes the idle cut-off, not the deadline: odnoga gives every call to the model at most 300 seconds, streamed or not (shorten it per request with x-airouter-timeout-ms). See Retries and timeouts.

A non-streamed call has no such protection. If the model needs more than ~150 s before it answers, the call is cut off, so use one of the two stream modes below for long calls.

Final-only mode: one answer, without the token stream

Streaming token by token is the right choice for a chat UI. A server-side caller that streams only to survive long calls, such as a Supabase Edge Function, a Vercel or Cloudflare function, or a worker, pays for every frame instead. A long answer is thousands of frames, and parsing each one uses the caller's own CPU budget just to stitch them back together.

Ask for the final-only mode with one header:

x-odnoga-stream: final

together with "stream": true. odnoga still streams from the model, so metering and logging are unchanged, but it sends you:

1. : keepalive comments every 15 s while the model works,
2. one data: frame with the complete chat.completion: choices[0].message (content, or complete tool_calls), finish_reason, usage, and the same _odnoga block a non-streamed response carries (request_id, cost_usd, model, vendor, tier, region),
3. data: [DONE].

If the model fails mid-stream, the one frame is an {"error": {…}} object instead. The response header x-odnoga-stream reads final when the mode was honoured, or tokens for the normal stream. Final-only mode is available on /v1/chat/completions. The Anthropic-shaped /v1/messages endpoint always streams normally.

Supabase Edge Function example

const res = await fetch(`${ODNOGA_BASE_URL}/chat/completions`, {
  method: "POST",
  headers: {
    authorization: `Bearer ${Deno.env.get("ODNOGA_API_KEY")}`,
    "content-type": "application/json",
    "x-odnoga-stream": "final",
  },
  body: JSON.stringify({ model: "gpt-6-sol", stream: true, messages }),
  // Read deadline: odnoga gives the model at most 300 s, so a little more
  // covers the final frame. Keep it under your own function's wall-clock limit
  // (~400 s on Supabase paid plans); keepalives hold the line meanwhile.
  signal: AbortSignal.timeout(310_000),
});

// The body is a few keepalive comments and ONE data frame: read it whole.
const text = await res.text();
const frame = text.split("\n\n").find((f) => f.startsWith("data: {"));
const completion = JSON.parse(frame!.slice(6));
if (completion.error) throw new Error(completion.error.message);
const answer = completion.choices[0].message.content;
const cost = completion._odnoga?.cost_usd;

Recommended read deadline: a little over odnoga's 300-second upstream deadline (about 310 s), and below the wall-clock limit of whatever runs your code. odnoga sends a keepalive at least every 15 s, so the connection never sits idle long enough to be dropped; a call that runs into the 300 s deadline ends with an error instead of an answer.

Cancellation

Pass an AbortSignal. odnoga will cancel the upstream vendor call and bill only the tokens already produced.

const ctrl = new AbortController();
setTimeout(() => ctrl.abort(), 5000);
await client.chat.completions.create({ /* … */ }, { signal: ctrl.signal });

Streams are always OpenAI-shaped

Even when the workspace's response format is set to native (raw vendor bodies), streamed responses are delivered in the OpenAI delta shape — a raw vendor stream carries no usage frame to meter. The stream's x-airouter-response-format header always reads openai (plus an explanatory x-airouter-response-format-note when the workspace is on native mode), so a native-mode client never has to guess.

Headers on streams

Headers are sent at stream open, before any tokens exist, so streams carry routing evidence, not billing totals: x-airouter-vendor, x-airouter-model, x-airouter-fallback, x-airouter-service-tier, x-airouter-region, x-airouter-region-honored x-airouter-response-format and x-odnoga-stream (tokens or final). Final tokens and cost are metered from the stream server-side and land on the request record — check the Requests page or requests.search over MCP, keyed by end-user id, model and time (streams carry no request-id header; in final-only mode the request_id is in the frame's _odnoga block).

Don't double-buffer

If you proxy the stream to your own client, forward the chunks raw — don't reassemble and re-emit. Latency matters.