GeeeekExplorer / transformers-patchLinks
patches for huggingface transformers to save memory
☆32Updated 6 months ago
Alternatives and similar repositories for transformers-patch
Users that are interested in transformers-patch are comparing it to the libraries listed below
Sorting:
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆51Updated 5 months ago
- ☆114Updated 6 months ago
- ☆97Updated 8 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆135Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆78Updated last year
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆112Updated 8 months ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆137Updated last year
- ☆77Updated 2 weeks ago
- ☆132Updated 6 months ago
- Transformers components but in Triton☆34Updated 7 months ago
- KV cache compression for high-throughput LLM inference☆146Updated 10 months ago
- ☆155Updated 10 months ago
- A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…☆91Updated 3 months ago
- ☆52Updated 6 months ago
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆61Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆156Updated 2 months ago
- APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generation. A system-level optimization for scalable LLM tra…☆44Updated 2 months ago
- CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…☆208Updated 2 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- [NeurIPS 2025] Scaling Speculative Decoding with Lookahead Reasoning☆56Updated last month
- ☆56Updated 6 months ago
- REST: Retrieval-Based Speculative Decoding, NAACL 2024☆212Updated 3 months ago
- 🔥 LLM-powered GPU kernel synthesis: Train models to convert PyTorch ops into optimized Triton kernels via SFT+RL. Multi-turn compilation…☆105Updated last month
- ☆122Updated 6 months ago
- ☆124Updated 3 months ago
- Fast and memory-efficient exact kmeans☆130Updated last month
- Quantized Attention on GPU☆44Updated last year
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆170Updated last year
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆219Updated last year
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆52Updated last year