Copilot Studio gets an agent into Microsoft 365. Omnia tests it before the whole company does.
Omnia runs thousands of simulated conversations and holds the release on them. A Copilot Studio test set tops out at twenty, and publishing doesn’t wait for the result.
Thousands of simulated employees try it first through the real HR system: one asking about someone else’s salary, one gaming the leave policy, one on the phone talking over it. It ships only if it passes, and then reaches part of the staff before the rest. docs
on copilot studio Conversation test sets support up to 20 test cases of up to 12 messages each. Evaluations run before and after publishing; a gate that blocks a failing publish isn’t documented. their docs
Testing at the volume you’ll run at
Load testing, evaluation and synthetic data are three settings on the same job. It runs thousands of simulated conversations at once against the agent’s real tools, voice and adversarial callers among them. docs enterprise
Copilot Studio: Up to 20 generated or imported conversations per test set, run as the signed-in user. Voice and adversarial testing aren’t documented. source
A release that can say no
A version that fails the suite stays where it is. One that passes reaches staff in steps, and is promoted while its metrics hold or rolled back when they don’t. docs enterprise
Copilot Studio: Agents move between environments with Power Platform pipelines. An eval gate, staged rollout and automatic rollback aren’t documented. source
Telemetry your SRE team can use
Traces are standard OpenTelemetry with a documented span inventory, sent to the collector you already run. Prometheus metrics and Grafana dashboards come with it. docs
Copilot Studio: Environment-level export to Application Insights is in preview, and trace and span IDs use a GUID format instead of the OpenTelemetry standard. source
At a glance
| Topic | Omnia | Copilot Studio, per its docs |
|---|---|---|
| Tests hit real tools | yes Yes, with assertions on what the agent did. docs | documented Yes, as the configured user. source |
| Simulated users at scale | yes Thousands of concurrent conversations. docs enterprise | partly Up to 20 conversations per set. source |
| Eval-gated release | yes The suite decides whether a version ships. docs | not documented Not documented. source |
| End-user identity to tools | yes OIDC or edge trust, carried to every call. docs | documented Entra on-behalf-of. source |
| Argument-level tool policy | yes A CEL rule on every call, before the tool runs. docs enterprise | partly Connector policies and DLP at tenant level. source |
| Runs in your infrastructure | yes Your Kubernetes cluster, or AgentCore, Vertex or Foundry. docs | not documented Microsoft’s cloud. source |
| Any model | yes Seven native protocols and any OpenAI-compatible URL. docs | partly OpenAI and Anthropic models. source |
Common questions
- Can Omnia agents reach Microsoft 365 users?
- Omnia serves agents over WebSocket, REST, A2A or MCP, and deploys the same pack to Azure AI Foundry, so it can sit behind whatever front end your users already have.
- How is it priced?
- Copilot Studio is billed in Copilot Credits. Omnia runs on your infrastructure with no meter: free while you build, the core free in production, and an enterprise licencelicense per environment per year from £30,000.
Put your own agent through the gate.
Install Omnia in your cluster and run it free while you build.
Copilot Studio claims are from Microsoft’s published documentation, checked 22 October 2026. Spot something out of date? Tell us.