Promptfoo is a testing and evaluation framework for large language model applications, including prompts, agents, and RAG systems. It lets you compare performance across different LLM providers (GPT, Claude, Gemini, DeepSeek, etc.), run red team security tests for vulnerabilities, and define test cases through declarative configuration files with CLI and CI/CD integration. Developers use it to systematically validate and improve their LLM-based applications before deployment.

439 open issues · last updated Jul 21, 2026