GeeeekExplorer / transformers-patchLinks
patches for huggingface transformers to save memory
☆30Updated 5 months ago
Alternatives and similar repositories for transformers-patch
Users that are interested in transformers-patch are comparing it to the libraries listed below
Sorting:
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆52Updated 3 months ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆137Updated last year
- ☆102Updated 5 months ago
- A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…☆68Updated 2 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆78Updated last year
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆130Updated 10 months ago
- ☆97Updated 7 months ago
- ☆69Updated last week
- ☆50Updated 5 months ago
- ☆130Updated 5 months ago
- Make SGLang go brrr☆37Updated last month
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang☆58Updated 11 months ago
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆111Updated 7 months ago
- Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth☆197Updated last week
- KV cache compression for high-throughput LLM inference☆143Updated 8 months ago
- Transformers components but in Triton☆34Updated 5 months ago
- ☆55Updated 4 months ago
- An efficient implementation of the NSA (Native Sparse Attention) kernel☆121Updated 4 months ago
- [NeurIPS 2025] Scaling Speculative Decoding with Lookahead Reasoning☆51Updated this week
- ☆95Updated 8 months ago
- The evaluation framework for training-free sparse attention in LLMs☆102Updated 2 weeks ago
- CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…☆201Updated 3 weeks ago
- Fast and memory-efficient exact kmeans☆113Updated last week
- 🔥 LLM-powered GPU kernel synthesis: Train models to convert PyTorch ops into optimized Triton kernels via SFT+RL. Multi-turn compilation…☆89Updated 3 weeks ago
- Quantized Attention on GPU☆44Updated 11 months ago
- ☆120Updated 2 months ago
- Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs☆191Updated 3 weeks ago
- Hydragen: High-Throughput LLM Inference with Shared Prefixes☆43Updated last year
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆148Updated 2 weeks ago