Alternatives and similar repositories for promptfoo:

Users that are interested in promptfoo are comparing it to the libraries listed below

• instructor-ai / instructor
  structured outputs for llms
  ☆9,808Updated this week
• guardrails-ai / guardrails
  Adding guardrails to large language models.
  ☆4,646Updated this week
• dottxt-ai / outlines
  Structured Text Generation
  ☆11,072Updated this week
• microsoft / LLMLingua
  [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…
  ☆4,939Updated last week
• confident-ai / deepeval
  The LLM Evaluation Framework
  ☆5,585Updated this week
• BerriAI / litellm
  Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sag…
  ☆18,941Updated this week
• Unstructured-IO / unstructured
  Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.
  ☆10,513Updated this week
• explodinggradients / ragas
  Supercharge Your LLM Application Evaluations 🚀
  ☆8,457Updated last week
• langfuse / langfuse
  🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with Open…
  ☆9,390Updated this week
• stanfordnlp / dspy
  DSPy: The framework for programming—not prompting—language models
  ☆22,470Updated this week
• NVIDIA / NeMo-Guardrails
  NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
  ☆4,513Updated last week
• hegelai / prompttools
  Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chro…
  ☆2,810Updated 7 months ago
• 1rgs / jsonformer
  A Bulletproof Way to Generate Structured JSON from Language Models
  ☆4,643Updated last year
• ianarawjo / ChainForge
  An open-source visual programming environment for battle-testing prompts to LLMs.
  ☆2,530Updated this week
• Arize-ai / phoenix
  AI Observability & Evaluation
  ☆5,075Updated this week
• Chainlit / chainlit
  Build Conversational AI in minutes ⚡️
  ☆8,883Updated this week
• microsoft / promptbase
  All things prompt engineering
  ☆5,569Updated 9 months ago
• lm-sys / RouteLLM
  A framework for serving and evaluating LLM routers - save LLM costs without compromising quality
  ☆3,705Updated 7 months ago
• SciPhi-AI / R2R
  The most advanced AI retrieval system. Agentic Retrieval-Augmented Generation (RAG) with a RESTful API.
  ☆5,507Updated this week
• tensorchord / Awesome-LLMOps
  An awesome & curated list of best LLMOps tools for developers
  ☆4,586Updated last month
• weaviate / Verba
  Retrieval Augmented Generation (RAG) chatbot powered by Weaviate
  ☆6,937Updated 2 weeks ago
• Helicone / helicone
  🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
  ☆3,413Updated this week
• bentoml / OpenLLM
  Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
  ☆10,964Updated last week
• zilliztech / GPTCache
  Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
  ☆7,456Updated 6 months ago
• eth-sri / lmql
  A language for constraint-guided and efficient LLM programming.
  ☆3,864Updated 9 months ago
• langroid / langroid
  Harness LLMs with Multi-Agent Programming
  ☆3,158Updated this week
• guidance-ai / guidance
  A guidance language for controlling large language models.
  ☆19,883Updated this week
• Portkey-AI / gateway
  A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
  ☆7,310Updated last week
• PrefectHQ / marvin
  ✨ AI agents that spark joy
  ☆5,558Updated last week
• ShishirPatil / gorilla
  Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
  ☆11,891Updated this week