A prompt factory with evidence before production.
Version prompts, turn reviewed real calls into test cases, compare models on identical inputs, then A/B test and switch labels without a redeploy.
- • Prompts hardcoded across 100+ edge functions.
- • "Who changed the welcome prompt yesterday?" — git blame.
- • Want to test a new wording → branch, PR, deploy, watch logs.
- • Rollback = revert commit, redeploy, hope nothing else changed.
- • Prompts live in a registry, called by slug.
- • Every save is a version with an author and a diff.
- • Run an A/B by traffic %; compare traffic, reliability, latency and cost before choosing.
- • Rollback = re-point a label. Atomic. Zero deploys.
- • Compare 2–8 models on the same frozen evidence before live traffic.
Compare quality, cost and speed on your own workload.
A fair comparison starts with one prompt version and the same frozen test cases for every model—not a collection of unrelated demos.
Capture or create
Arm one real run or enter typed variables manually.
Review and freeze
Remove sensitive values and define the pass rule.
Run 2–8 models
Every model receives the identical version and cases.
Inspect evidence
Read responses, passes, tokens, billed cost and latency side by side.
Privacy is deliberate: capture is one-shot and admin-initiated. Failed, streamed, cached and laboratory calls are excluded. You review and redact variables before saving; an unsaved capture expires within 24 hours.
Call a prompt the way you call a function.
const r = await ai.prompts.chat({
slug: 'welcome-email',
label: 'production', // or 'staging', or 'experiment-v3'
variables: { name: 'Ada', plan: 'Pro' },
});
console.log(r.text);
console.log(r._meta.promptVersion); // version that served this call
console.log(r._meta.costUsd);
Versioned prompts
Every save is a version with a diff. Browse the history, compare, restore.
Labels: production, staging
Promote a version by re-pointing the label. SDK calls by label, not version id.
A/B by traffic %
Split traffic between two versions. Watch metrics. Ship the winner.
Natychmiastowe wycofanie
One click moves the production label back. No redeploy, no PR.
Per-tenant overrides
A tenant can pin a different prompt without forking your codebase.
Variables, typed
Pass variables; odnoga substitutes server-side and logs the resolved prompt.
Evaluation laboratory
Run the same frozen cases across 2–8 models and compare responses, pass rates, tokens, billed cost and latency.
Real-run capture
Explicitly capture one successful production call, review and redact its variables, then save it as a reusable test case.
Version analytics
Follow request volume, reliability, latency, cost and feedback by prompt version before changing production.
Frequently asked
How is this different from LangSmith or PromptLayer?
Those are observability tools you bolt on. odnoga's prompt registry is part of the gateway — calls are routed and metered together with prompt version, per tenant, with no extra SDK and no separate billing.
Can I compare models on the same prompt and inputs?
Yes. The evaluation laboratory freezes one prompt version and its test cases, then runs the identical snapshot across 2–8 selected models. The dashboard shows each response, assertion result, input and output tokens, billed cost and latency.
Can I turn a real production call into a test?
Yes, explicitly. Arm capture for one prompt, make the next successful non-streaming and non-cached call, then review and redact the captured variables before saving. Unsaved captures expire within 24 hours.
How does evaluation relate to an A/B test?
Evaluation tests a frozen workload before release. A/B testing measures two prompt versions on live traffic. Use laboratory evidence to narrow the candidates, then production evidence to decide whether to promote or roll back.
Can non-engineers edit prompts?
Yes. Prompts live in the dashboard with role-based access. Engineers ship the slug; product or ops iterate on the prompt without touching code.
What happens on rollback — do in-flight requests break?
No. The label flip is atomic. New requests use the new version, in-flight requests finish on the old one.