A Practitioner handbook for production llm serving.
☆203Sep 5, 2026Updated 3 weeks ago
Alternatives and similar repositories for llm-inference-at-scale
Users that are interested in llm-inference-at-scale are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A context layer for coding agents — build a code knowledge graph from a real codebase, then measure whether the agent actually uses it. A…☆16Jun 29, 2026Updated 3 months ago
- A physics-grounded, cost-aware optimization loop for vLLM☆68Aug 22, 2026Updated last month
- Dasein managed vector index service - Python SDK☆23May 11, 2026Updated 4 months ago
- An Efficient Compression Framework for LLM☆69Jun 27, 2026Updated 3 months ago
- The implementation of "Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization" (NeurIPS…☆22Nov 4, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆10Jul 20, 2023Updated 3 years ago
- Frontier-grade answers from any mix of models — a local MCP server bringing OpenRouter's Fusion panel architecture to any MCP client. Bri…☆35Sep 13, 2026Updated 2 weeks ago
- Become GPU kernel engineer step by step.☆70May 7, 2026Updated 4 months ago
- Use any AI model, locally or in the cloud, while tracking on-device token usage and collecting token telemetry data.☆18Aug 14, 2026Updated last month
- A dbt package to run natural language queries☆10Jan 13, 2023Updated 3 years ago
- Curated collection of AI inference engineering resources — LLM serving, GPU kernels, quantization, distributed inference, and production …☆326Feb 4, 2026Updated 7 months ago
- CDRAG is a new retrieval framework that uses hierarchical document clustering, and LLM-guided document selection from those clusters to c…☆38Apr 13, 2026Updated 5 months ago
- A conversational AI system using Ollama with persistent memory capabilities. Features hybrid context management (sliding window + vector …☆22Mar 20, 2026Updated 6 months ago
- I developed a fine-tuned retrieval head for RAG that learns to more reliably retrieve relevant passages by transforming the query embeddi…☆17May 21, 2026Updated 4 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Please see the newer: https://github.com/Quazmoz/openvino-windows-llm☆27Sep 18, 2026Updated 2 weeks ago
- One script to replace every outdated tool that Apple ships on macOS with modern versions via Homebrew☆16Jul 18, 2026Updated 2 months ago
- Long-Term Memory plugin for Claude Code — semantic search, context injection, session learning☆26Updated this week
- A local-first, self-organizing AI RAG GRAPH knowledge system that reads, links and reasons over your documents — offline by default, clou…☆29Apr 22, 2026Updated 5 months ago
- ☆25Dec 15, 2025Updated 9 months ago
- The code implementation of HyGRAG, accepted by WWW'26.☆15May 31, 2026Updated 4 months ago
- FastMemory is a topological representation of text data using concepts as the primary input. It helps in improving the RAG(by replacing e…☆53Updated this week
- GPU Engineering for AI Systems☆598May 17, 2026Updated 4 months ago
- ☆13Dec 12, 2021Updated 4 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- This project demonstrates how you can enhance standard CRUD operations in your application using Semantic Search mechanism.☆12Oct 23, 2024Updated last year
- ☆21Sep 7, 2026Updated 3 weeks ago
- Offline-first synchronization engine for SQLite and PostgreSQL, built in Rust with column-level CRDTs, authoritative partial replicas, …☆80Jun 28, 2026Updated 3 months ago
- Data Science Course☆15Jan 8, 2024Updated 2 years ago
- Demo of fine-tuning QA models for answering FAQ of cloud providers documentation☆11Jun 20, 2026Updated 3 months ago
- An "LLM wiki" upgraded to a real database — typed entities, graph relations, HTTP API, and a built-in natural-language agent.☆110Sep 12, 2026Updated 3 weeks ago
- A fast & easy way to train ML models in your cloud, directly from your laptop.☆14Mar 28, 2022Updated 4 years ago
- AIGuru - R2AI Stage 1☆39Jul 14, 2026Updated 2 months ago
- ☆24Apr 22, 2026Updated 5 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Open Source client side code of INDRZ indoor wayfinding, mapping, routing system. Mirrored from https://gitlab.com/indrz.☆15Feb 16, 2025Updated last year
- 🔍📃 LLM-powered PDF Table Extractor☆19Jun 26, 2025Updated last year
- A simple SPA to implement User-Owns-Data Embedding in Salesforce.☆12Aug 17, 2022Updated 4 years ago
- Streaming Retrieval-Augmented Generation (RAG) agent in Go. It consumes real-time data from Kafka topics, processes it in configurable wi…☆27Jun 7, 2025Updated last year
- ☆13Apr 16, 2025Updated last year
- Bulk Salesforce Files downloader built with Robot Framework and Python. Supports parallel batches, validated downloads, CI, and Data Load…☆24Aug 27, 2026Updated last month
- Enterprise-grade distributed AI agent framework | Develop → Deploy → Observe | K8s-native | Dynamic DI | Auto-failover | Multi-LLM | Pyth…☆41Updated this week