lm-sys / RouteLLM
A framework for serving and evaluating LLM routers - save LLM costs without compromising quality
☆3,884Updated 8 months ago
Alternatives and similar repositories for RouteLLM:
Users that are interested in RouteLLM are comparing it to the libraries listed below
- [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…☆5,048Updated last month
- Deploy your agentic worfklows to production☆2,002Updated last week
- RAG that intelligently adapts to your use case, data, and queries☆3,206Updated last month
- Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…☆3,426Updated 2 months ago
- The easiest way to use Agentic RAG in any enterprise☆4,210Updated 3 months ago
- Optimizing inference proxy for LLMs☆2,201Updated last week
- SoTA production-ready AI retrieval system. Agentic Retrieval-Augmented Generation (RAG) with a RESTful API.☆6,683Updated this week
- Harness LLMs with Multi-Agent Programming☆3,265Updated this week
- The Open Source Memory Layer For Autonomous Agents☆2,190Updated 6 months ago
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆2,671Updated last week
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆2,964Updated last week
- ☆2,928Updated 7 months ago
- Supercharge Your LLM Application Evaluations 🚀☆9,025Updated last week
- Superfast AI decision making and intelligent processing of multi-modal data.☆2,569Updated last week
- Easy token price estimates for 400+ LLMs. TokenOps.☆1,647Updated 3 weeks ago
- Cohere Toolkit is a collection of prebuilt components enabling users to quickly build and deploy RAG applications.☆3,038Updated this week
- A language model programming library.☆5,754Updated 2 months ago
- Knowledge Agents and Management in the Cloud☆3,934Updated this week
- [NeurIPS'24] HippoRAG is a novel RAG framework inspired by human long-term memory that enables LLMs to continuously integrate knowledge a…☆2,297Updated this week
- Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models☆2,735Updated 3 months ago
- Go ahead and axolotl questions☆9,258Updated this week
- Build a Perplexity-Inspired Answer Engine Using Next.js, Groq, Llama-3, Langchain, OpenAI, Upstash, Brave & Serper☆4,893Updated 7 months ago
- Zep | The Memory Foundation For Your AI Stack☆3,259Updated last month
- The LLM Evaluation Framework☆6,147Updated this week
- Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.☆1,055Updated last month
- Python & JS/TS SDK for running AI-generated code/code interpreting in your AI app☆1,710Updated this week
- Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Ge…☆6,395Updated this week
- Retrieval Augmented Generation (RAG) chatbot powered by Weaviate☆7,076Updated last month
- Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sag…☆21,842Updated this week
- AdalFlow: The library to build & auto-optimize LLM applications.☆2,971Updated last month