Sierra tests agents on its own platform. Omnia tests them through your real tools, in your cluster.
Omnia runs the tests through your real tools in your own cluster, on the model you choose, and promotes a version on its live metrics. Sierra’s simulations run on Sierra’s platform, and its rollback is a manual step.
That conversation ran a few hundred times before release, against your order system in a test environment, with simulated customers who changed their minds and pushed back. One run hit the carrier API returning a half-updated order, and the version stopped at the gate. docs
on sierra Sierra’s release testing simulates conversations against mock APIs. their docs
Tests that hit the real tools
Simulated users drive the agent through its tools, and assertions check the calls it made: that it verified the customer before changing the order, and that it didn’t issue a refund it shouldn’t have. docs
Sierra: Simulations run between agents and mock personas, against mock APIs, and can gate a release from CI. source
Your cluster, your models
Omnia installs into your Kubernetes cluster, so conversations and tool data stay inside your boundary. The agent uses whichever model you choose: Claude, GPT, Gemini, a model on Bedrock or Vertex, or one you host. docs
Sierra: A self-hosted option isn’t documented. Sierra describes its agents as built on a constellation of models it selects. source
Promotion on live numbers
A new version starts on part of the traffic, and Omnia promotes it when its live metrics hold or rolls it back when they drop. Conversations in progress stay on the version they started with. docs enterprise
Sierra: Releases are immutable snapshots with instant rollback and A/B testing. Promotion or rollback on live metrics isn’t documented. source
At a glance
| Topic | Omnia | Sierra, per its docs |
|---|---|---|
| Simulated users | yes Multi-turn, adversarial included. docs | documented Mock personas. source |
| Simulation against real tools | yes Yes, with assertions on what the agent did. docs | not documented Against mock APIs. source |
| Voice testing | yes Barge-in, overlapping speech and dead air. docs | documented Voice Sims for noise and speaker variation. source |
| Runs in your infrastructure | yes Your Kubernetes cluster. docs | not documented Not documented. source |
| Choose the model | yes Any provider, or your own. docs | not documented Not documented. source |
| Metric-gated promotion | yes Promotion waits for the numbers; rollback is automatic. docs enterprise | partly Instant manual rollback. source |
| Argument-level tool policy | yes A CEL rule on each call’s arguments. docs enterprise | partly Tools withheld until the customer is authenticated. source |
| Production failures become tests | yes A failed session becomes a regression scenario. docs enterprise | not documented Not documented. source |
Common questions
- How is Sierra priced?
- Sierra describes outcome-based pricing, where you pay when a conversation is resolved, with consumption pricing where it fits. Omnia runs on your infrastructure with no meter: free while you build, the core free in production, and an enterprise licencelicense per environment per year from £30,000.
- Is Omnia only for customer service?
- No. It runs conversational agents, voice agents, and functions with typed input and output, on a schedule if you like.
Put your own agent through the gate.
Install Omnia in your cluster and run it free while you build.
Sierra claims are from Sierra’s published documentation, checked 22 October 2026. Spot something out of date? Tell us.