Implementation of Speculative Sampling as described in "Accelerating Large Language Model Decoding with Speculative Sampling" by Deepmind
☆112Feb 29, 2024Updated 2 years ago
Alternatives and similar repositories for Speculative-Sampling
Users that are interested in Speculative-Sampling are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Fast inference from large lauguage models via speculative decoding☆927Aug 22, 2024Updated 2 years ago
- Keyformer proposes KV Cache reduction through key tokens identification and without the need for fine-tuning☆59Mar 26, 2024Updated 2 years ago
- Chat interface for your computer☆16Mar 14, 2024Updated 2 years ago
- REST: Retrieval-Based Speculative Decoding, NAACL 2024☆220Mar 5, 2026Updated 7 months ago
- ☆69Dec 3, 2024Updated last year
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Implementation of the paper "Opcodes as predictor for malware " by Daniel Bilar☆11Oct 17, 2020Updated 5 years ago
- [ICLR 2024] Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond☆23Apr 29, 2024Updated 2 years ago
- An approximate implementation of the OpenAI paper - An Empirical Model of Large-Batch Training for MNIST☆11Nov 19, 2022Updated 3 years ago
- DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting☆19Mar 4, 2025Updated last year
- Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**☆229Feb 13, 2025Updated last year
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆415Apr 22, 2025Updated last year
- Parallel waveform generation with DiffusionGAN☆17Mar 26, 2022Updated 4 years ago
- ☆23Aug 14, 2024Updated 2 years ago
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆37Oct 9, 2025Updated last year
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Official implementation of "SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching" (COLM 2025). A novel KV cache com…☆15Sep 29, 2025Updated last year
- Implementation of the paper "Variable Bitrate Residual Vector Quantization for Audio Coding"☆11Apr 10, 2025Updated last year
- ☆16Jun 4, 2024Updated 2 years ago
- Official implementation of DGP-based multi-speaker speech synthesis with PyTorch☆24Mar 23, 2021Updated 5 years ago
- ☆24Mar 15, 2022Updated 4 years ago
- ☆19Jan 10, 2023Updated 3 years ago
- Efficient Neural Architecture Search via Straight-Through Gradients☆13Nov 12, 2020Updated 5 years ago
- [ICLR 2025] PEARL: Parallel Speculative Decoding with Adaptive Draft Length☆172Dec 23, 2025Updated 9 months ago
- The official code for Dropping Backward Propagation (DropBP)☆32Oct 29, 2024Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- G2pw's inference speed is accelerated by about 8-10 times. Change loop generated predictive data to only once and model loop prediction b…☆14Dec 30, 2023Updated 2 years ago
- A social networking website in Django☆26Aug 17, 2022Updated 4 years ago
- Implementation of FlashAttention (FA1-FA4) in PyTorch for educational and algorithmic clarity☆227Apr 12, 2026Updated 6 months ago
- [SpeechCom Journal] Learning and controlling the source-filter representation of speech with a variational autoencoder☆46Apr 18, 2023Updated 3 years ago
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,341Mar 6, 2025Updated last year
- Multi-Candidate Speculative Decoding☆41Apr 22, 2024Updated 2 years ago
- NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates [WIP]☆25Jul 5, 2022Updated 4 years ago
- Automatically Discovering Fast Parallelization Strategies for Distributed Deep Neural Network Training☆1,900Updated this week
- My vocoder experiments☆31Jul 26, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Code for EMNLP 2021 paper: Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting☆17Nov 30, 2021Updated 4 years ago
- [ICASSP 2025] AnCoGen: Analysis, Control and Generation of Speech with a Masked Autoencoder☆16Mar 11, 2025Updated last year
- ☆28Mar 14, 2024Updated 2 years ago
- ICASSP 2024 - Generative De-Quantization for Neural Speech Codec via Latent Diffusion.☆56Jul 31, 2026Updated 2 months ago
- [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…☆74Jun 26, 2024Updated 2 years ago
- ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation☆25Oct 2, 2020Updated 6 years ago
- PyTorch Implementation of NCSOFT's FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis☆74Aug 3, 2021Updated 5 years ago