Deployment & experimentation of LLM inference at production scale (P/D disaggregation, KV offloading, parallelism, and autoscaling).
☆65Sep 26, 2026Updated last month
Alternatives and similar repositories for deployment-guide
Users that are interested in deployment-guide are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆12Aug 19, 2026Updated 2 months ago
- everything you need to know about inference engineering☆45Sep 29, 2026Updated last month
- A distributed inference framework for large language models that routes requests to workers, manages KV cache lifecycle, handles failures…☆27Oct 6, 2026Updated last month
- 100 days of LLM inference engineering — daily posts, experiments, and visualizations☆1,912Apr 30, 2026Updated 6 months ago
- Plug-and-play PyTorch transformer research framework. Swap attention, FFN, norm, positional, connection, projection, optimizer, scheduler…☆18Oct 25, 2026Updated 2 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Find moments in your videos — agent-first CLI for Marlin-2B (dense captioning + temporal grounding). Local on Apple Silicon (MLX) / NVIDI…☆13Jul 1, 2026Updated 4 months ago
- documenting my work in inference engineering☆25Apr 19, 2026Updated 6 months ago
- Educational distributed training and inference library for heterogeneous local hardware: Mac minis, Raspberry Pis and GPUs over plain Pyt…☆90Oct 7, 2026Updated last month
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,354Oct 17, 2026Updated 3 weeks ago
- ☆104Oct 4, 2026Updated last month
- A novel temporal fusion framework for propelling autoregressive model inference☆11Updated this week
- A collection of GPU experiments and benchmarks for my personal understanding and research.☆43Aug 19, 2026Updated 2 months ago
- ☆16Jun 28, 2026Updated 4 months ago
- Information for the Intro to Cluster System Administration for Non-Sysadmins class☆10Dec 12, 2021Updated 4 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- mumwelt provides the environment for an agent to effectively commune with the Marin project☆30Updated this week
- A Lean Formalization Benchmark for Advanced Mathematics☆14Nov 2, 2026Updated last week
- Speed-of-Light SW efficiency by using ultra low-latency primitives for comms collectives☆13Apr 8, 2026Updated 7 months ago
- Framework that makes your LLM self-aware, loops with self-learning and persistent memory that's private. Open-source & free.☆24Aug 17, 2026Updated 2 months ago
- Explore training for quantized models☆29Jul 12, 2025Updated last year
- A training codebase for Zeroth-Order Optimization☆14Updated this week
- ☆13May 21, 2026Updated 5 months ago
- a collection of CUDA Kernels☆18Jul 24, 2026Updated 3 months ago
- Agent Skill to prepare learning schedule for any technical topic.☆30Apr 24, 2026Updated 6 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Kernels by MakoraGenerate☆15May 26, 2026Updated 5 months ago
- A course offered by Louis-Philippe Morency from Carnegie Mellon University☆23Oct 8, 2020Updated 6 years ago
- python opengl protein viewer☆10Nov 29, 2025Updated 11 months ago
- This is a CLI tool for playing music quickly made in python.☆10Jul 8, 2024Updated 2 years ago
- Repository for ML Reproducibility Challenge 2020 for the Neurips paper, "The Value Equivalence Principle for Model-Based Reinforcement Le…☆18Apr 13, 2021Updated 5 years ago
- A meta-harness for Google Gemini 3: An agentic web UI with tool use and reasoning visualization, built by an AI agent.☆14Nov 24, 2025Updated 11 months ago
- Sequential Monte Carlo Speculative Decoding☆53Aug 25, 2026Updated 2 months ago
- ☆12Dec 30, 2020Updated 5 years ago
- ☆11Oct 15, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆11Oct 25, 2025Updated last year
- observability for speculative decoding☆15Aug 23, 2026Updated 2 months ago
- Open-source transpiler for CUDA Tile (13.1) migration☆19Dec 9, 2025Updated 11 months ago
- fine tuning mistral 7B using Huggingface, Weights and Biases, Choline, and Vast AI☆41Oct 4, 2023Updated 3 years ago
- ☆13Aug 17, 2024Updated 2 years ago
- NVFP4 KV cache for vLLM on SM120 (RTX PRO 6000) via FlashInfer FA2 explicit-SF-stride patch — ~1.5x fp8 pool at ~95-104% speed☆20Jun 5, 2026Updated 5 months ago
- Face swap using Python + OpenGL☆14Aug 21, 2022Updated 4 years ago
Are these results useful?