# Omnia vs Braintrust

> Omnia holds a release below a score threshold, rolls out on live scores, and runs in your own cluster. Braintrust scores the eval, and what happens to the release is up to your pipeline.

Category: Tracing and evaluation. Checked against Braintrust’s published material on 2026-10-22. Canonical page: https://www.altairalabs.ai/vs/braintrust/

## Braintrust measures quality. Omnia makes quality the release gate.

In Omnia the pass threshold, the rollout and the live scores are one system, inside your cluster. Braintrust reports the score to CI; turning it into a release decision is a reporter you write.

## When it matters: A prompt change drops task completion from 94% to 81%.

**On Omnia:** The suite carries its own pass threshold. 81% is below it, so the version stops at the gate. (Omnia docs: https://promptarena.altairalabs.ai/arena/how-to/interfaces/run-in-ci/)

**On Braintrust:** bt eval returns a non-zero exit code if any eval throws an exception. Failing on a score needs a custom reporter. (source: https://www.braintrust.dev/docs/evaluate/run-in-ci)

## The gate is the quality bar

Each version faces simulated users, hostile and voice callers, and assertions on the tool calls it makes. The threshold is part of the suite, and the gate reads it directly. (Omnia docs: https://promptarena.altairalabs.ai/arena/explanation/scenario-design/)

Braintrust: CI gates on exceptions by default; voice simulation comes through a partner. (source: https://www.braintrust.dev/docs/evaluate/run-in-ci)

## Online scores that act

Online scores feed the rollout. Each step to more traffic waits for them, and a drop triggers a rollback. *(enterprise)* (Omnia docs: https://omnia.altairalabs.ai/tutorials/progressive-rollouts/)

Braintrust: Online scoring runs asynchronously in the background, on a sample. (source: https://www.braintrust.dev/docs/evaluate/score-online)

## All of it in your cluster

Testing, the runtime, the scores and the session archive stay in your Kubernetes cluster, on every edition. (Omnia docs: https://omnia.altairalabs.ai/how-to/workspaces/manage-workspaces/)

Braintrust: Self-hosting is on the Enterprise plan, with Braintrust hosting the control plane. (source: https://www.braintrust.dev/docs/guides/self-hosting)

## At a glance

| Topic | Omnia | Omnia docs | Braintrust | Braintrust docs | Source |
|---|---|---|---|---|---|
| Datasets and scorers | Scenarios, assertions and judges. *(enterprise)* | https://promptarena.altairalabs.ai/arena/reference/assertions/ | Datasets, scorers, playground. | documented | https://www.braintrust.dev/docs/evaluate/score-online |
| Quality gate in CI | Built in. | https://promptarena.altairalabs.ai/arena/how-to/interfaces/run-in-ci/ | Custom reporter. | partly documented | https://www.braintrust.dev/docs/evaluate/run-in-ci |
| Simulated users | Multi-turn, against real tools. | https://promptarena.altairalabs.ai/arena/explanation/scenario-design/ | Not documented. | not documented | https://www.braintrust.dev/docs/evaluate/score-online |
| Voice testing | Built in. | https://promptarena.altairalabs.ai/arena/how-to/voice/setup-voice-testing/ | Through a partner. | partly documented | https://www.braintrust.dev/articles/how-to-evaluate-voice-agents |
| Runs and rolls out agents | Yes. | https://omnia.altairalabs.ai/tutorials/progressive-rollouts/ | Not documented. | not documented | https://www.braintrust.dev/docs/evaluate/score-online |
| Self-hosted | Every edition. | https://omnia.altairalabs.ai/how-to/workspaces/manage-workspaces/ | Enterprise, hybrid. | partly documented | https://www.braintrust.dev/docs/guides/self-hosting |

## Questions

**How is each priced?**

Braintrust charges by data processed and scores, with plans from free to Enterprise. Omnia is free while you build, the core is free in production, and the enterprise licence is per environment per year from £30,000, with nothing metered.