# Omnia vs LangSmith

> Omnia ties the evaluation to the release, rolls out in steps with automatic rollback, and keeps tool policy in configuration. In LangSmith, gating a release on an experiment is a script you maintain in CI.

Category: Tracing and evaluation. Checked against LangChain’s published material on 2026-10-22. Canonical page: https://www.altairalabs.ai/vs/langsmith/

## LangSmith evaluates the new version. In Omnia, the evaluation is the release decision.

Omnia picks up where the experiment ends: the gate, the rollout and the rules on each tool call, all in your cluster. LangSmith traces and evaluates, self-hosted on its Enterprise plan, and gating a release on the result is a CI script you maintain.

## When it matters: An experiment comes back four points down.

**On Omnia:** That suite is also the release gate, so the version stays where it is. One that passes goes to part of the traffic first and is rolled back if its live metrics drop. (Omnia docs: https://promptarena.altairalabs.ai/arena/how-to/interfaces/run-in-ci/)

**On LangSmith:** The docs’ example gate is a script that exits non-zero below a threshold. For deployments, the rollback advice is to tag your images with version information. (source: https://docs.langchain.com/langsmith/deploy-with-control-plane)

## A release gate

Simulated users, hostile and voice callers among them, run the agent through its real tools. The gate is part of the platform rather than a script in your CI. (Omnia docs: https://promptarena.altairalabs.ai/arena/explanation/scenario-design/)

LangSmith: Multi-turn simulation through the open-source openevals package, run from pytest, Vitest or evaluate. The gate is your own script. (source: https://docs.langchain.com/langsmith/multi-turn-simulation)

## Rollout and rollback

A new version gets traffic in steps, and each step waits for its metrics to hold. A conversation that started on the old version finishes there. If the metrics drop, the version is rolled back. *(enterprise)* (Omnia docs: https://omnia.altairalabs.ai/tutorials/progressive-rollouts/)

LangSmith: Deployments ship as revisions, and assistant versions can be promoted to active. Canary and automatic rollback aren’t documented. (source: https://docs.langchain.com/langsmith/deployment-revisions)

## Policy as configuration

Tool policy is a CEL rule in the agent’s pack, checked against each call’s arguments and caller. If the broker can’t answer, the call is refused. Security can review the rule without reading your Python. *(enterprise)* (Omnia docs: https://omnia.altairalabs.ai/how-to/security/configure-tool-policies/)

LangSmith: Human-in-the-loop middleware interrupts per tool, with an optional Python predicate on the call’s arguments. (source: https://docs.langchain.com/oss/python/langchain/human-in-the-loop)

## At a glance

| Topic | Omnia | Omnia docs | LangSmith | LangSmith docs | Source |
|---|---|---|---|---|---|
| Runs in your Kubernetes cluster | Helm, in every edition. | https://omnia.altairalabs.ai/how-to/workspaces/manage-workspaces/ | Self-hosted on the Enterprise plan. | documented | https://docs.langchain.com/langsmith/platform-setup |
| Simulated users | Built in, adversarial included. | https://promptarena.altairalabs.ai/arena/explanation/scenario-design/ | Via openevals. | documented | https://docs.langchain.com/langsmith/multi-turn-simulation |
| Voice testing | Barge-in, overlapping speech and dead air. | https://promptarena.altairalabs.ai/arena/how-to/voice/setup-voice-testing/ | Not documented. | not documented | https://docs.langchain.com/oss/python/langchain/voice-agent |
| Eval-gated release | Built in. | https://promptarena.altairalabs.ai/arena/how-to/interfaces/run-in-ci/ | Write your own CI script. | partly documented | https://docs.langchain.com/langsmith/read-local-experiment-results |
| Stepped rollout, auto rollback | Yes. | https://omnia.altairalabs.ai/tutorials/progressive-rollouts/ | Not documented. | not documented | https://docs.langchain.com/langsmith/deployment-revisions |
| OpenTelemetry out to your collector | Generally available, with a documented span inventory. | https://omnia.altairalabs.ai/reference/platform/telemetry/ | Alpha on Agent Server. | partly documented | https://docs.langchain.com/langsmith/env-var-cloud |
| End-user identity to tools | Carried through the agent to every call. | https://omnia.altairalabs.ai/explanation/platform/delegation-tokens/ | Custom auth exposes the user to the graph. | partly documented | https://docs.langchain.com/langsmith/auth |

## Questions

**Can I keep LangSmith for tracing and use Omnia to run agents?**

Yes. Omnia emits standard OpenTelemetry, which LangSmith ingests, and runs agents built in other frameworks under its own policy and testing.

**How is each priced?**

LangSmith charges per seat and per trace, with deployments metered per vCPU-hour; self-hosting is on the Enterprise plan. Omnia is free while you build, the core is free in production, and the enterprise licence is one price per environment per year from £30,000, with nothing metered.