A PyTorch native library for training speculative decoding models
☆258Sep 29, 2026Updated 2 weeks ago
Alternatives and similar repositories for TorchSpec
Users that are interested in TorchSpec are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆1,221Updated this week
- TokenSpeed is a speed-of-light LLM inference engine.☆2,246Updated this week
- Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini &…☆572Updated this week
- [EMNLP2026 Main] Official implementation of “Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding”.☆139Jul 25, 2026Updated 2 months ago
- ☆78Apr 30, 2026Updated 5 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆883Updated this week
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆331Jul 29, 2026Updated 2 months ago
- Draft-Target Disaggregation LLM Serving System via Parallel Speculative Decoding.☆223Mar 18, 2026Updated 6 months ago
- Code for the paper *Attention Drift: What Speculative Decoding Models Learn*.☆33May 12, 2026Updated 5 months ago
- Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime.☆3,108Updated this week
- ☆974Updated this week
- [ASPLOS'26] Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter☆182Feb 27, 2026Updated 7 months ago
- Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…☆515Updated this week
- mKernel: fast multi-node, multi-GPU fused kernels☆297Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- IREE Tokenizer Python Bindings☆21Mar 5, 2026Updated 7 months ago
- DeepSpec: a full-stack codebase for training and evaluating speculative decoding algorithms☆7,216Jul 9, 2026Updated 3 months ago
- Humming is a high-performance, lightweight, and highly flexible JIT (Just-In-Time) compiled GEMM kernel library specifically designed for…☆254Updated this week
- ☆42Jun 23, 2026Updated 3 months ago
- Perplexity GPU Kernels☆608Nov 7, 2025Updated 11 months ago
- FlashInfer: Kernel Library for LLM Serving☆6,618Updated this week
- Official Implementation of SAM-Decoding: Speculative Decoding via Suffix Automaton☆54May 12, 2026Updated 5 months ago
- ☆429Updated this week
- ☆397Apr 16, 2026Updated 5 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- An LLM post-training framework with vLLM for RL Scaling☆485Updated this week
- SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.☆1,459Updated this week
- Tile-Based Runtime for Ultra-Low-Latency LLM Inference☆1,861Sep 24, 2026Updated 2 weeks ago
- Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).☆2,552Feb 20, 2026Updated 7 months ago
- ☆97Sep 15, 2025Updated last year
- htop-like TUI for real-time RDMA network monitoring.☆116Oct 2, 2026Updated last week
- An Optimizer for Nvidia Compilers.☆139Updated this week
- 📰 Must-read papers and blogs on Speculative Decoding ⚡️☆1,299Jun 27, 2026Updated 3 months ago
- slime is an LLM post-training framework for RL Scaling.☆8,661Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- ☆98Mar 26, 2025Updated last year
- High Performance LLM Inference Operator Library☆1,176Sep 7, 2026Updated last month
- A high-performance and light-weight router for vLLM large scale deployment☆467Updated this week
- DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling☆45Oct 5, 2026Updated last week
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆415Apr 22, 2025Updated last year
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆6,794Updated this week
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆773Updated this week