Alternatives and similar repositories for flash-attention-turing:

Users that are interested in flash-attention-turing are comparing it to the libraries listed below

• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆173Updated this week
• zhaochenyang20 / ModelServer
  Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang
  ☆49Updated 5 months ago
• OpenGVLab / EfficientQAT
  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆263Updated 6 months ago
• neuralmagic / AutoFP8
  ☆185Updated 6 months ago
• ModelCloud / GPTQModel
  Production ready LLM model compression/quantization toolkit with hw accelerated inference support for both cpu/gpu via HF, vLLM, and SGLa…
  ☆494Updated this week
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆243Updated last week
• Cornell-RelaxML / qtip
  ☆130Updated last month
• xuyuzhuang11 / OneBit
  The homepage of OneBit model quantization framework.
  ☆175Updated 2 months ago
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆242Updated last year
• InternLM / turbomind
  ☆83Updated last month
• ROCm / flash-attention
  Fast and memory-efficient exact attention
  ☆171Updated this week
• mit-han-lab / deepcompressor
  Model Compression Toolbox for Large Language Models and Diffusion Models
  ☆446Updated last month
• LLM-inference-router / vllm-router
  vLLM Router
  ☆26Updated last year
• pzhao-eng / FlashMLA
  ☆43Updated last week
• xlite-dev / Awesome-Diffusion-Inference
  📖A curated list of Awesome Diffusion Inference Papers with codes: Sampling, Caching, Multi-GPUs, etc. 🎉🎉
  ☆212Updated last month
• intel / auto-round
  Advanced Quantization Algorithm for LLMs/VLMs.
  ☆438Updated this week
• JL-er / RWKV-PEFT
  ☆119Updated last week
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆168Updated 3 weeks ago
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆630Updated this week
• EfficientMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆171Updated last week
• xorbitsai / xllamacpp
  xllamacpp - a Python wrapper of llama.cpp
  ☆35Updated 2 weeks ago
• foldl / chatllm.cpp
  Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
  ☆578Updated this week
• PipeFusion / PipeFusion
  A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters
  ☆45Updated 9 months ago
• ubergarm / r1-ktransformers-guide
  run DeepSeek-R1 GGUFs on KTransformers
  ☆224Updated last month
• IST-DASLab / marlin
  FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
  ☆809Updated 7 months ago
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆71Updated this week
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆102Updated this week
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆98Updated last year
• interestingLSY / swiftLLM
  A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …
  ☆163Updated 9 months ago
• Cornell-RelaxML / quip-sharp
  ☆532Updated 6 months ago