mayank31398 / ladder-residual-inferenceLinks
☆14Updated 2 months ago
Alternatives and similar repositories for ladder-residual-inference
Users that are interested in ladder-residual-inference are comparing it to the libraries listed below
Sorting:
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆60Updated 11 months ago
- The evaluation framework for training-free sparse attention in LLMs☆100Updated 3 months ago
- Muon fsdp 2☆44Updated 2 months ago
- Fast and memory-efficient exact attention☆70Updated 7 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- Using FlexAttention to compute attention with different masking patterns☆45Updated last year
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆118Updated 3 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆130Updated 10 months ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆82Updated last year
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆89Updated 2 months ago
- ☆46Updated last year
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Updated last year
- JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)☆36Updated last year
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆68Updated 10 months ago
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆84Updated 11 months ago
- ☆14Updated last year
- ☆52Updated 11 months ago
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- ☆83Updated last year
- ☆129Updated 4 months ago
- Work in progress.☆74Updated 3 months ago
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆73Updated last year
- Experiment of using Tangent to autodiff triton☆81Updated last year
- ☆41Updated 6 months ago
- FlexAttention w/ FlashAttention3 Support☆27Updated last year
- [NeurIPS 2025] Scaling Speculative Decoding with Lookahead Reasoning☆44Updated 2 weeks ago
- Odysseus: Playground of LLM Sequence Parallelism☆77Updated last year
- Make triton easier☆47Updated last year
- ring-attention experiments☆152Updated 11 months ago
- Code for studying the super weight in LLM☆119Updated 10 months ago