promptfoo / promptfooLinks
Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
☆7,501Updated this week
Alternatives and similar repositories for promptfoo
Users that are interested in promptfoo are comparing it to the libraries listed below
Sorting:
- 🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with Open…☆13,586Updated this week
- The LLM Evaluation Framework☆9,134Updated this week
- Adding guardrails to large language models.☆5,229Updated last month
- 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓☆4,095Updated this week
- Supercharge Your LLM Application Evaluations 🚀☆9,864Updated this week
- Superfast AI decision making and intelligent processing of multi-modal data.☆2,671Updated this week
- Harness LLMs with Multi-Agent Programming☆3,462Updated this week
- Structured Outputs☆12,084Updated this week
- An open-source visual programming environment for battle-testing prompts to LLMs.☆2,678Updated 2 months ago
- AI Observability & Evaluation☆6,253Updated this week
- Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean…☆11,873Updated this week
- structured outputs for llms☆10,907Updated this week
- Build Conversational AI in minutes ⚡️☆10,143Updated this week
- The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.☆2,918Updated last week
- Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sag…☆25,176Updated this week
- An awesome & curated list of best LLMOps tools for developers☆5,063Updated 2 weeks ago
- NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.☆4,872Updated this week
- [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…☆5,262Updated 4 months ago
- A framework for serving and evaluating LLM routers - save LLM costs without compromising quality☆4,083Updated 11 months ago
- Agent Framework / shim to use Pydantic with LLMs☆10,703Updated this week
- Zep | Examples, Integrations, & More☆3,359Updated last week
- DSPy: The framework for programming—not prompting—language models☆26,177Updated this week
- A language for constraint-guided and efficient LLM programming.☆3,984Updated last month
- an ambient intelligence library☆5,808Updated last week
- Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chro…☆2,894Updated 10 months ago
- A language model programming library.☆5,798Updated last month
- The AI framework that adds the engineering to prompt engineering (Python/TS/Ruby/Java/C#/Rust/Go compatible)☆4,615Updated this week
- The open-source visual AI programming environment and TypeScript library☆3,988Updated last week
- Letta (formerly MemGPT) is the stateful agents framework with memory, reasoning, and context management.☆17,263Updated this week
- RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry☆4,137Updated 4 months ago