DeepInfer: High-performance, large-scale inference framework for DeepSeek models
☆25,855Nov 3, 2026Updated last week
Alternatives and similar repositories for DeepInfer
Users that are interested in DeepInfer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆2,819Oct 19, 2026Updated 3 weeks ago
- DeepSeek Harness: Everything is a Plugin.☆278,856Updated this week
- Towards a More Autonomous Agent System☆22,440Nov 3, 2026Updated last week
- SGLang is a high-performance serving framework for large language models and multimodal models.☆38,744Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆96,749Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels☆8,949Updated this week
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆7,024Updated this week
- DeepGEMM: clean and efficient BLAS kernel library on GPU☆9,186Oct 20, 2026Updated 3 weeks ago
- Nano vLLM☆16,388Apr 26, 2026Updated 6 months ago
- AI agent toolkit: unified LLM API, agent loop, TUI, coding agent CLI☆121,422Updated this week
- Reverse engineer anything with agents, from app behavior down to native binaries.☆452,735Updated this week
- FlashInfer: Kernel Library for LLM Serving☆6,897Updated this week
- ☆23,455Nov 2, 2026Updated last week
- DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms☆7,306Jul 9, 2026Updated 4 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Lightweight coding agent that runs in your terminal☆134,895Updated this week
- DeepEP: an efficient expert-parallel communication library☆10,431Oct 21, 2026Updated 3 weeks ago
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆5,561May 17, 2026Updated 5 months ago
- FlashMLA: Efficient Multi-head Latent Attention Kernels☆13,261Oct 27, 2026Updated 2 weeks ago
- slime is an LLM training framework for RL Scaling.☆8,814Updated this week
- TokenSpeed is a speed-of-light LLM inference engine.☆2,918Updated this week
- LMCache: Supercharge Your LLM with the Fastest KV Cache Layer☆12,340Updated this week
- A kernel library written in tilelang☆2,039Oct 14, 2026Updated 3 weeks ago
- The agent that grows with you☆262,641Updated this week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework☆24,184Updated this week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆43,141Updated this week
- The open source coding agent.☆226,081Updated this week
- DeepSelect: TopK kernels for DeepSeek Sparse Attention (DSA) and Samplers☆540Sep 30, 2026Updated last month
- Fast and memory-efficient exact attention☆25,275Updated this week
- 《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验☆7,491Oct 1, 2026Updated last month
- MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts☆1,210Oct 12, 2026Updated last month
- Kernel Design Agents (KDA) is a agent-centric workflow to write high-performance CUDA Kernels.☆1,568Oct 19, 2026Updated 3 weeks ago
- Ongoing research training transformer models at scale☆18,377Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A Datacenter Scale Distributed Inference Serving Framework☆8,470Updated this week
- ☆17,502Oct 30, 2026Updated last week
- A framework for efficient model inference with omni-modality models☆7,699Updated this week
- TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizat…☆15,062Updated this week
- AgentENV (AENV) is a distributed platform for running agent environments at scale.☆3,742Updated this week
- DeepGEMM-Ascend: clean and efficient matrix multiplication kernel library for Huawei Ascend NPUs☆639Oct 14, 2026Updated 3 weeks ago
- LLM inference in C/C++☆134,072Updated this week
Are these results useful?