ZRayZzz / flash-attention-v100
☆32Updated last year
Alternatives and similar repositories for flash-attention-v100:
Users that are interested in flash-attention-v100 are comparing it to the libraries listed below
- Triton Documentation in Chinese Simplified / Triton 中文文档☆67Updated last week
- Summary of system papers/frameworks/codes/tools on training or serving large model☆56Updated last year
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆109Updated 11 months ago
- Official PyTorch implementation of "FlatQuant: Flatness Matters for LLM Quantization"☆123Updated last month
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆42Updated last month
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆35Updated 2 months ago
- An easy-to-use package for implementing SmoothQuant for LLMs☆96Updated 2 weeks ago
- ☆78Updated last year
- Official implementation of the EMNLP23 paper: Outlier Suppression+: Accurate quantization of large language models by equivalent and opti…☆47Updated last year
- ☆82Updated last month
- ☆68Updated 3 months ago
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆36Updated 3 weeks ago
- 📚FFPA(Split-D): Yet another Faster Flash Attention with O(1) GPU SRAM complexity large headdim, 1.8x~3x↑🎉 faster than SDPA EA.☆169Updated 2 weeks ago
- Implement Flash Attention using Cute.☆76Updated 4 months ago
- ☆69Updated last week
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆116Updated 2 weeks ago
- ☆92Updated 7 months ago
- 使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention☆63Updated 8 months ago
- 使用 CUDA C++ 实现的 llama 模型推理框架☆50Updated 5 months ago
- GPTQ inference TVM kernel☆38Updated last year
- Official PyTorch implementation of "Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models"☆28Updated this week
- Quantized Attention on GPU☆45Updated 5 months ago
- A practical way of learning Swizzle☆18Updated 2 months ago
- ☆131Updated last month
- Code implementation of GPTQv2 (https://arxiv.org/abs/2504.02692)☆29Updated last week
- Summary of the Specs of Commonly Used GPUs for Training and Inference of LLM☆40Updated last month
- Code Repository for the NeurIPS 2024 Paper "Toward Efficient Inference for Mixture of Experts".☆16Updated 5 months ago
- ☆139Updated last year
- A MoE impl for PyTorch, [ATC'23] SmartMoE☆62Updated last year
- ☆55Updated 2 weeks ago