☆68Dec 3, 2024Updated last year
Alternatives and similar repositories for OSD
Users that are interested in OSD are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A selective knowledge distillation algorithm for efficient speculative decoders☆40Nov 27, 2025Updated 11 months ago
- ☆12Feb 20, 2026Updated 8 months ago
- official code for GliDe with a CaPE☆21Aug 13, 2024Updated 2 years ago
- ☆33May 24, 2025Updated last year
- Multi-Candidate Speculative Decoding☆41Apr 22, 2024Updated 2 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆71Nov 4, 2024Updated 2 years ago
- 📰 Must-read papers and blogs on Speculative Decoding ⚡️☆1,304Jun 27, 2026Updated 4 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆162Dec 4, 2024Updated last year
- An Attention Superoptimizer☆22Jan 20, 2025Updated last year
- Repository for the COLM 2025 paper SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths☆19Jul 10, 2025Updated last year
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆414Apr 22, 2025Updated last year
- Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction | A tiny BERT model can tell you the verbosity of an …☆53Jun 1, 2024Updated 2 years ago
- Python package for rematerialization-aware gradient checkpointing☆27Oct 31, 2023Updated 3 years ago
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆228Oct 14, 2026Updated 3 weeks ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆68May 19, 2025Updated last year
- Fast inference from large lauguage models via speculative decoding☆927Aug 22, 2024Updated 2 years ago
- Official Implementation of SAM-Decoding: Speculative Decoding via Suffix Automaton☆53May 12, 2026Updated 5 months ago
- (COLING2025) SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding☆10Dec 18, 2024Updated last year
- Paper list of safety in reasoning of Large Reasoning Models (LRMs).☆10Mar 27, 2025Updated last year
- Artifacts for our SIGCOMM'23 paper Ditto☆16Oct 17, 2023Updated 3 years ago
- Official Repo for "SplitQuant / LLM-PQ: Resource-Efficient LLM Offline Serving on Heterogeneous GPUs via Phase-Aware Model Partition and …☆41Aug 29, 2025Updated last year
- [ACL 2025 main] FR-Spec: Frequency-Ranked Speculative Sampling☆56Jul 15, 2025Updated last year
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆173Dec 23, 2025Updated 10 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- [NeurIPS'23] Speculative Decoding with Big Little Decoder☆99Feb 6, 2024Updated 2 years ago
- SpotServe: Serving Generative Large Language Models on Preemptible Instances☆140Feb 22, 2024Updated 2 years ago
- 西安电子科技大学软件工程-大三上计网复习笔记与实验与试题☆27Feb 8, 2025Updated last year
- [ICLR 2025] SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration☆70Feb 21, 2025Updated last year
- Efficient Inference for Large Language Model-based Generative Recommendation (ICLR'25)☆15May 27, 2025Updated last year
- REST: Retrieval-Based Speculative Decoding, NAACL 2024☆220Mar 5, 2026Updated 8 months ago
- [COLM 2024] TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding☆280Aug 31, 2024Updated 2 years ago
- [Project] Tune LLaMA with Prefix/LoRA on English/Chinese instruction datasets☆12Mar 31, 2023Updated 3 years ago
- A Cluster-Wide Model Manager to Accelerate DNN Training via Automated Training Warmup☆36Jan 9, 2023Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- [ASPLOS'26] Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Drafter☆187Feb 27, 2026Updated 8 months ago
- Official Implementation of "GRIFFIN: Effective Token Alignment for Faster Speculative Decoding"[NeurIPS 2025]☆21May 12, 2025Updated last year
- Implementation of AdaCQR(COLING 2025)☆15Dec 30, 2024Updated last year
- Vortex: Programmable Sparse Attention for Agents as Algorithm Designers☆69Aug 22, 2026Updated 2 months ago
- Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation☆35Feb 27, 2024Updated 2 years ago
- Explorations into some recent techniques surrounding speculative decoding☆308Dec 22, 2024Updated last year
- PyTorch Implementation of Speculative decoding☆11Jun 25, 2025Updated last year