promptfoo review

promptfoo FAQ

Is promptfoo free and open source?

Yes. promptfoo's source code is on GitHub (promptfoo/promptfoo) under the MIT license, so you can use it for free. Its website also links to a pricing page (https://promptfoo.dev/pricing/) for any paid plans.

How do I install promptfoo?

Run npm install -g promptfoo. It needs Node.js >=22.22.0. The install section above has the other options from the project's README.

Is promptfoo still maintained?

Yes. The promptfoo/promptfoo repository was last updated on Oct 25, 2026, and the latest release, 0.125.1 on GitHub, came out on Oct 25, 2026.

How popular is promptfoo?

promptfoo has 26.7k GitHub stars as of Nov 1, 2026. It gained 306 new stars in the week of Oct 25–31, 2026, against a 12-week average of 231 a week. Its main package had about 1.2M downloads that week, +16% against its own 4-week average. That made it #6 among LLM evals and observability tools on the Devtoolwatch Board.

promptfoo vs DeepEval: how do they compare?

promptfoo and DeepEval are both tracked by Devtoolwatch as LLM evals and observability tools. promptfoo has 26.7k GitHub stars and DeepEval has 19.2k. In the week of Oct 25–31, 2026, promptfoo's main package had 1.2M downloads and DeepEval's had 769.5k. DeepEval is an open-source Python framework for unit testing and evaluating LLM apps, agents, chatbots, voice agents, and RAG pipelines. DeepEval review.

promptfoo vs Langfuse: how do they compare?

promptfoo and Langfuse are both tracked by Devtoolwatch as LLM evals and observability tools. promptfoo has 26.7k GitHub stars and Langfuse has 36.8k. In the week of Oct 25–31, 2026, promptfoo's main package had 1.2M downloads and Langfuse's had 9.8M. Langfuse is an open source LLM engineering platform for tracing, evaluating, prompt management, and debugging LLM applications and agents. Langfuse review.

What are the best alternatives to promptfoo?

Other LLM evals and observability tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: SkillSpector (#1), Judgeval (#2), A.I.G (AI-Infra-Guard) (#3), Langfuse (#4), Opik (#5). The alternatives table above has their numbers.

What is promptfoo used for?

promptfoo is used to evaluate and red-team LLM-based apps. Developers use it to test prompts and models, compare providers side by side, scan for vulnerabilities such as prompt injection and data leaks, and automate these checks in CI/CD.

How do you run promptfoo?

promptfoo runs from the command line. Run promptfoo init --example getting-started, set your provider API key, then run promptfoo eval and promptfoo view inside the project folder. You can also run any command without installing by using npx promptfoo@latest.

How does promptfoo work?

promptfoo reads declarative config files that list prompts, providers, and tests. It sends the tests to the configured LLM APIs, scores the outputs, and shows the results in the CLI or a web viewer. For red teaming, it generates attacks tailored to your target application.

Is promptfoo joining OpenAI?

Yes. The promptfoo README states that Promptfoo is now part of OpenAI. It also states that Promptfoo remains open source and MIT licensed.

Is promptfoo safe for private prompts?

promptfoo runs LLM evals locally, so your prompts never leave your machine, according to the README. Prompts are still sent to whichever LLM provider APIs you configure.

Does promptfoo work with Python?

Yes. promptfoo is available through pip install promptfoo. The README says it works with any LLM API or programming language. It also ships as a Node.js package.