Optimizing inference proxy for LLMs
☆4,177Jul 18, 2026Updated this week
Alternatives and similar repositories for optillm
Users that are interested in optillm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Open-source implementation of AlphaEvolve☆6,745Updated this week
- Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails…☆6,871Updated this week
- Structured Outputs☆14,547Updated this week
- DSPy: The framework for programming—not prompting—language models☆36,221Updated this week
- Entropy Based Sampling and Parallel CoT Decoding☆3,435Nov 13, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Pivotal Token Search☆154Updated this week
- Tools for merging pretrained large language models.☆7,243Jun 17, 2026Updated last month
- Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement…☆10,492Updated this week
- Go ahead and axolotl questions☆12,215Updated this week
- 💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows☆12,733Updated this week
- The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails…☆54,020Updated this week
- aider is AI pair programming in your terminal☆47,510May 22, 2026Updated last month
- SGLang is a high-performance serving framework for large language models and multimodal models.☆30,498Updated this week
- The LLM Evaluation Framework☆16,950Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.☆23,864Jul 3, 2026Updated 2 weeks ago
- Build Real-Time Knowledge Graphs for AI Agents☆28,912Updated this week
- A guidance language for controlling large language models.☆21,685May 21, 2026Updated last month
- Create Custom LLMs☆1,858Jun 27, 2026Updated 3 weeks ago
- Large-scale LLM inference engine☆1,801Updated this week
- 🙌 OpenHands: AI-Driven Development☆81,293Updated this week
- Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.☆3,143Updated this week
- ☆1,033Dec 17, 2024Updated last year
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layer☆10,706Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- structured outputs for llms☆13,568Jul 13, 2026Updated last week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆86,634Updated this week
- A fast inference library for running LLMs locally on modern consumer-class GPUs☆4,586Mar 4, 2026Updated 4 months ago
- g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains☆4,176Dec 30, 2025Updated 6 months ago
- Universal memory layer for AI Agents☆61,213Updated this week
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆3,816May 28, 2026Updated last month
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆3,332Updated this week
- Our library for RL environments + evals☆4,387Updated this week
- SoTA production-ready AI retrieval system. Agentic Retrieval-Augmented Generation (RAG) with a RESTful API.☆7,931Nov 7, 2025Updated 8 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A flexible, adaptive classification system for dynamic text classification☆567Oct 7, 2025Updated 9 months ago
- Build, run, and manage agent platforms.☆41,287Updated this week
- Recursive-Open-Meta-Agent v0.1 (Beta). A meta-agent framework to build high-performance multi-agent systems.☆5,093Feb 16, 2026Updated 5 months ago
- Get your documents ready for gen AI☆63,456Updated this week
- Harness LLMs with Multi-Agent Programming☆4,077Jul 12, 2026Updated last week
- ☆972Jan 23, 2025Updated last year
- Agentless🐱: an agentless approach to automatically solve software development problems☆2,083Dec 22, 2024Updated last year