FasterDecoding / BitDeltaLinks
☆197Updated 5 months ago
Alternatives and similar repositories for BitDelta
Users that are interested in BitDelta are comparing it to the libraries listed below
Sorting:
- PB-LLM: Partially Binarized Large Language Models☆152Updated last year
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆160Updated 11 months ago
- ☆125Updated last year
- ☆129Updated 3 months ago
- ☆222Updated 11 months ago
- The official repo for "LLoCo: Learning Long Contexts Offline"☆116Updated 11 months ago
- Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…☆132Updated 9 months ago
- Official PyTorch implementation of QA-LoRA☆135Updated last year
- Multipack distributed sampler for fast padding-free training of LLMs☆188Updated 9 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆275Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆199Updated 10 months ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆237Updated 4 months ago
- Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models☆236Updated last year
- Explorations into some recent techniques surrounding speculative decoding☆266Updated 5 months ago
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆163Updated 10 months ago
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆143Updated 8 months ago
- Experiments on speculative sampling with Llama models☆126Updated last year
- Token Omission Via Attention☆126Updated 7 months ago
- The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction☆386Updated 10 months ago
- ☆79Updated 4 months ago
- A family of compressed models obtained via pruning and knowledge distillation☆341Updated 6 months ago
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆84Updated 11 months ago
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆149Updated last month
- ☆248Updated last year
- [NeurIPS 24 Spotlight] MaskLLM: Learnable Semi-structured Sparsity for Large Language Models☆166Updated 5 months ago
- ☆68Updated 10 months ago
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆301Updated last month
- Code accompanying the paper "Massive Activations in Large Language Models"☆162Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆116Updated 6 months ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆153Updated 7 months ago