dust-tt / llama-ssp
Experiments on speculative sampling with Llama models
☆118Updated last year
Related projects ⓘ
Alternatives and complementary repositories for llama-ssp
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆199Updated 6 months ago
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆139Updated this week
- REST: Retrieval-Based Speculative Decoding, NAACL 2024☆176Updated this week
- Multipack distributed sampler for fast padding-free training of LLMs☆178Updated 3 months ago
- Code repository for the c-BTM paper☆105Updated last year
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆135Updated 5 months ago
- Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆79Updated this week
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆89Updated last year
- Official implementation for 'Extending LLMs’ Context Window with 100 Samples'☆74Updated 10 months ago
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆77Updated last month
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆262Updated last year
- ☆189Updated 6 months ago
- ☆122Updated 10 months ago
- ☆184Updated last month
- KV cache compression for high-throughput LLM inference☆87Updated last week
- A simple unified framework for evaluating LLMs☆145Updated 2 weeks ago
- The official repo for "LLoCo: Learning Long Contexts Offline"☆113Updated 5 months ago
- Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"☆128Updated last month
- ☆96Updated 2 months ago
- scalable and robust tree-based speculative decoding algorithm☆318Updated 3 months ago
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆112Updated last year
- Explorations into some recent techniques surrounding speculative decoding☆212Updated last year
- Codes for the paper "∞Bench: Extending Long Context Evaluation Beyond 100K Tokens": https://arxiv.org/abs/2402.13718☆285Updated last month
- Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper☆124Updated 4 months ago
- GPTQLoRA: Efficient Finetuning of Quantized LLMs with GPTQ☆97Updated last year
- A pipeline for LLM knowledge distillation☆78Updated 3 months ago
- Implementation of Speculative Sampling as described in "Accelerating Large Language Model Decoding with Speculative Sampling" by Deepmind☆82Updated 8 months ago
- Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers☆195Updated 3 months ago
- Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)☆139Updated 2 months ago
- ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…☆216Updated 7 months ago