import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.odnoga.com/v1", // <- the AI gateway
apiKey: process.env.ODNOGA_API_KEY, // <- your odnoga key
});
const res = await client.chat.completions.create({
model: "auto", // or "gpt-5-mini", "claude-haiku-4-5", ...
messages: [{ role: "user", content: "Hello" }],
});
Routing in front, accounting behind.
One OpenAI-compatible endpoint
Point any SDK or agent at odnoga and call OpenAI, Anthropic, Gemini, Mistral, Groq and more through a single AI gateway URL.
Routing policies and failover
Route by cost, latency, capability or residency, with automatic fallback when a vendor degrades.
Budgets, quotas and concurrency
Hard and soft spend caps per tenant, workspace, end user and model, plus a tenant-wide concurrency pool.
Catch a process that stops
odnoga learns the normal traffic of every app and function, then alerts you when one silently stops or suddenly surges. The same view shows recent history and failed calls, so you know where to begin investigating.
Per-request cost ledger
Every call writes an exact ledger entry: tokens in and out, vendor price, your margin, who to bill.
Regional routing with evidence
Choose EU or US vendor endpoints and keep recorded evidence of where each request was actually served.
Versioned prompts and A/B
Prompts live in the gateway, not hardcoded in your repo — version them, A/B them, roll back in one click.
odnoga vs Kong vs Portkey
Kong routes HTTP. Portkey routes LLMs. odnoga routes AI workloads with multi-tenant billing and Supabase-native integration.
| Capability | odnoga | Kong | Portkey |
|---|---|---|---|
| Supabase-native integration | |||
| Multi-tenant billing and wallets | Partial | ||
| Per-end-user cost attribution | Partial | ||
| EU/US residency enforcement | |||
| OpenAI-compatible endpoint | |||
| Prompt management and A/B | |||
| API transformation layer |
Comparison reflects publicly documented capabilities as of the page date. Contact us if anything looks out of date.
How an AI gateway works, request by request
A gateway is a control plane in front of the model vendors. Your app authenticates once against the gateway; the gateway resolves policy, picks a model, calls the vendor with its own stored credentials, and writes the accounting record before the response is handed back.
Authenticate and identify
The gateway API key resolves the tenant, workspace and — when you send an end-user header — the individual customer the call belongs to.
Admission control
Budget, quota and concurrency checks run before any vendor is contacted, so an overspend is refused rather than discovered on next month’s invoice.
Routing decision
A routing policy resolves the requested alias to a concrete model based on cost, latency, capability and residency, and keeps ordered fallbacks for degraded vendors.
Normalized vendor call
One OpenAI-shaped request is translated into each vendor’s dialect — tools, vision, reasoning effort and web search differ per provider.
Metering and ledger write
Token counts and vendor list prices produce an exact per-request ledger entry with your margin, attributable to a workspace or an end user.
Observability and evidence
Request logs keep the model actually used, the fallback path, the region served and the latency, so incidents and audits have a record to read.
your app ──▶ AI gateway ──▶ auth + budget check ──▶ routing policy ──▶ vendor (EU/US endpoint)
│ │
└──────────── ledger entry + request log ◀───────────────┘When you do not need one: a single app calling a single model, with no per-customer cost question and no compliance requirement, is usually better off calling the vendor directly. A gateway earns its place the moment there is more than one model, more than one customer, or more than one person changing prompts.
Two kinds of caching, named honestly
Most gateways advertise "caching" as one feature. It is two, they save money in different ways, and only one of them can change what your users read. You switch each on separately and see what each one saved.
odnoga cache — we reuse our stored answer
An identical request gets back the answer we already have. No provider call, so the call costs nothing. The honest catch: it is the earlier answer, not a fresh one — which is why we reuse only deterministic calls unless you say otherwise.
Provider prompt cache — cheaper re-reads
The provider remembers the long prompt you keep resending and charges less to read it again. The answer is always generated fresh, so this can never change a result.
- Safety mode: reuse only when the call was deterministic (temperature 0 or a fixed seed).
- Scope it per workspace or per end customer, so one customer never sees another’s answer.
- Validity you choose: 5 minutes, 1 hour or 24 hours — and per-function or per-prompt overrides.
- Per-call control with a header: bypass the stored answer, refresh it, or switch caching off.
- Every reused answer comes back marked with its age, so nothing is silently stale.
- Both kinds counted in the same ledger: what was avoided, on which function, at which model rate.
Response caching is not unique to odnoga — OpenRouter and Portkey offer it too. What we add is governance and proof: who it applies to, when reuse is refused, and a savings figure traceable to your own requests rather than a marketing multiplier.
Signs your app has outgrown direct vendor calls
- Model names and prompts are hardcoded across edge functions and nobody knows which version is live.
- You cannot say how much a single customer costs you in AI this month.
- A vendor outage or an out-of-credit key takes your feature down with no fallback.
- You need EU routing for some customers and cannot evidence where requests were served.
- Adding a new provider means another SDK, another key store and another billing line.
AI gateway questions
What is an AI gateway?
An AI gateway is a control plane that sits between your application and every AI model provider you use. It gives you one API endpoint, unified authentication, policy enforcement, cost attribution, and observability across multiple vendors. Unlike a traditional API gateway, it understands model-specific concepts: tokens, prompts, reasoning effort, embeddings, vision, and regional residency.
How is an AI gateway different from Kong or an API gateway?
Traditional API gateways like Kong manage traffic, rate limits and transformations at the HTTP layer. They do not understand AI workloads: per-token pricing, model capabilities, prompt versioning, or which customer should be billed for a call. An AI gateway adds these semantics and lets you route, meter and bill AI requests natively.
How is odnoga different from Portkey?
Portkey is a strong LLM gateway. odnoga is an AI gateway built specifically for multi-tenant SaaS running on Supabase. That means tenant-aware budgets, per-end-user wallets, multi-tenant billing, Supabase Edge Function integration, and EU/US residency enforcement out of the box.
Do I have to change my code?
Change the base URL and the API key. The chat completions and embeddings surfaces are OpenAI-compatible, so official SDKs, LangChain, n8n and most tools work unchanged.
Whose vendor keys are used?
Yours. You add your own OpenAI, Anthropic or Gemini keys to a workspace and odnoga routes through them, so you keep your vendor pricing and rate limits.
Can I bill my own customers through it?
Yes. Send one header identifying your end user and every request is attributed and billable — see multi-tenant billing.
Does an AI gateway cache reuse answers?
There are two different things called caching. A response cache stores the answer and returns it again for an identical request — no model call happens, so the call is free, but you get the earlier answer, not a new one. A provider prompt cache never reuses an answer: the provider only charges less to re-read prompt text it already holds. odnoga runs both, as two separate switches you control.
Will caching change my answers?
It can, and we say so plainly. If a model is asked to be creative, the same question asked twice does not have one right answer, so returning a stored one is a different result than calling again. That is why our default only reuses an answer for calls the model was told to make deterministic — temperature 0 or a fixed seed. Reusing answers for creative calls is possible, but it is an explicit choice you make, not a default we make for you.
Prompt caching vs response caching — which saves more?
A reused answer removes the whole cost of that call; a provider prompt cache only discounts the repeated prompt text, usually a large share of a long system prompt. Response caching saves more per hit but applies to far fewer calls. odnoga counts both from your own request history, per function and per workspace, so you can see which one is actually paying off. Savings are cost you avoided, never money paid back.
Related: what an LLM gateway is · model catalog and prices · multi-tenant billing · security and residency · how odnoga compares