Everything you need to know about LLM inference
☆411Sep 2, 2026Updated this week
Alternatives and similar repositories for llm-inference-handbook
Users that are interested in llm-inference-handbook are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Benchmark and optimize LLM inference across frameworks with ease☆201Jul 14, 2026Updated last month
- how to build a sentence embedding application using BentoML☆15Jul 14, 2026Updated last month
- 🐳 Build OCI images for Bentos in k8s☆19Jul 14, 2026Updated last month
- learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen☆4,538Updated this week
- ☆16Oct 15, 2025Updated 10 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Command-line tool for debugging MCP servers☆37Updated this week
- Postgres extension that speeds up analytics queries by upto 90%☆52Jun 8, 2024Updated 2 years ago
- VibeRL is a Reinforcement Learning framework built essentially through vibe coding with Kimi K2.☆18Updated this week
- A simple Rust crate to cache data both in-memory and on disk☆11Dec 26, 2021Updated 4 years ago
- Personal Site☆21Aug 10, 2026Updated 3 weeks ago
- A concise, beginner-friendly introduction to the core ideas of linear algebra.☆1,997Mar 16, 2026Updated 5 months ago
- react app to help automate attack modifiers, monster hp tracking, monster ability cards, elements☆12Feb 28, 2021Updated 5 years ago
- ☆483Nov 25, 2025Updated 9 months ago
- A reverse literate programming tool.☆16Jun 26, 2024Updated 2 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A domain specific language to build complex workflows☆15Jun 26, 2026Updated 2 months ago
- Modern, Fast, Fun - A Semantic-First Data/BI Platform☆15Updated this week
- This is a python implementation for stitching images.☆229Oct 3, 2024Updated last year
- Terminal user interface for a Kanban board☆11Nov 5, 2021Updated 4 years ago
- Online Inference API for NLP Transformer models - summarization, text classification, sentiment analysis and more☆45Mar 16, 2024Updated 2 years ago
- ☆16May 23, 2026Updated 3 months ago
- ☆18Sep 17, 2025Updated 11 months ago
- Elixir-powered BitTorrent Tracker☆341Mar 1, 2026Updated 6 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆90,889Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- ☆22Feb 6, 2026Updated 6 months ago
- A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual h…☆194Updated this week
- Private, on-device voice-to-text for macOS. Free, open source, no subscription.☆25Jan 10, 2026Updated 7 months ago
- ☆164Mar 5, 2026Updated 5 months ago
- Context management tool for LLM collaboration☆93Aug 19, 2025Updated last year
- Simple Agents Made Easy☆617Mar 16, 2026Updated 5 months ago
- A comprehensive cookbook demonstrating how to implement CrewAI with Anthropic's prompt caching feature for efficient LLM operations☆16Aug 11, 2025Updated last year
- A demonstration of text/GUI bi-directional editing via an LSP server☆38Jul 1, 2025Updated last year
- ☆17Oct 21, 2025Updated 10 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Nano vLLM☆15,297Apr 26, 2026Updated 4 months ago
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆799Updated this week
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆4,947May 17, 2026Updated 3 months ago
- SGLang is a high-performance serving framework for large language models and multimodal models.☆33,856Updated this week
- Persistent memory for LLMs and apps. Content-addressed storage with dedupe, compression, full-text and vector search.☆373Updated this week
- FlashInfer: Kernel Library for LLM Serving☆6,326Updated this week
- Handwritten GEMM using Intel AMX (Advanced Matrix Extension)☆17Jan 11, 2025Updated last year