Is Promptfoo still open source?

Its README says it is now part of OpenAI and remains open source under the MIT licence.

Yes. PromptArena is Apache 2.0 and installs with npm. The paid parts are Omnia’s enterprise features in production.

omnia vs promptfoo

Already red-teaming with Promptfoo? PromptArena does the same job on whichever providers you choose.

Then check where the traffic goes. Promptfoo, now part of OpenAI and still MIT licensed, runs its simulated users and several multi-turn attacks on hosted inference by default. PromptArena is the open test harness under Omnia.

A bank red-teams its payments agent before launch.

on omnia

The simulated customers and the attacker trying to reroute a payment run on the model provider the bank already approved, against the agent’s real tools, inside the bank’s own boundary. The transcripts stay there, and the agent ships only if the suite passes. docs

on promptfoo

By default, the simulated user runs on Promptfoo’s hosted conversation models. Several multi-turn attack strategies use remote inference in the community edition. their docs

Every simulated user runs where you say

Simulated users, attackers and judges run on whichever provider you configure: Claude, GPT, Gemini, Bedrock, Vertex, or a model you host. docs

Promptfoo: Hosted conversation models by default for simulated users; remote inference for GOAT, Goblin and Hydra in the community edition. source

Voice, as a phone call

PromptArena’s voice tests hold a two-way audio conversation with the agent, so the simulated caller can cut in while the agent is speaking. Each session is recorded. docs

Promptfoo: The audio strategy converts prompt text into speech and encodes it, and requires remote generation. source

Red-team results gate the release

Omnia runs the agent as well as the tests, so a passing suite starts the rollout. If live metrics drop, the version is rolled back and the production failure is added to the suite. docs enterprise

Promptfoo: A CI flag fails the build on test failures. Promptfoo doesn’t run agents, so what happens after the test is up to you. source

At a glance

Topic	Omnia	Promptfoo, per its docs
Open source	yes PromptArena, Apache 2.0. docs	documented MIT. source
Red-team strategies	yes Adversarial callers probing for policy bypass, against real tools. docs	documented A broad library, including Crescendo and GOAT. source
Simulated users on your own provider	yes Always. docs	partly Hosted by default. source
Duplex voice testing	yes Barge-in, overlapping speech and dead air. docs	partly Text converted to audio. source
CI gate	yes Built into the release. docs	documented Fail the build on errors. source
Self-hosted server for production	yes Omnia, installed with Helm. docs	not documented Not recommended for production; no built-in SSO. source
Runs and rolls out the agent	yes Yes, with automatic rollback. docs	not documented Testing only. source

Common questions

Is Promptfoo still open source?: Its README says it is now part of OpenAI and remains open source under the MIT licencelicense.
Is PromptArena free?: Yes. PromptArena is Apache 2.0 and installs with npm. The paid parts are Omnia’s enterprise features in production.

Put your own agent through the gate.

Install Omnia in your cluster and run it free while you build.

Promptfoo claims are from Promptfoo, now part of OpenAI’s published documentation, checked 22 October 2026. Spot something out of date? Tell us.

Install it in your cluster Or walk through it with us →