Alternatives and similar repositories for online-softmax

Users that are interested in online-softmax are comparing it to the libraries listed below

• cauyxy / bilivideos
  ☆52Updated last year
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆63Updated last year
• L1aoXingyu / llm-infer-bench
  ☆11Updated last year
• ironartisan / awesome-compression1
  模型压缩的小白入门教程
  ☆22Updated 11 months ago
• mdy666 / Qwen-Native-Sparse-Attention
  qwen-nsa
  ☆66Updated last month
• DD-DuDa / BitDistiller
  [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.
  ☆114Updated last year
• microsoft / chunk-attention
  ☆76Updated last month
• TRT2022 / ControlNet_TensorRT
  天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案
  ☆49Updated last year
• TRT2022 / trtllm-llama
  ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化
  ☆48Updated last year
• Adlik / smoothquantplus
  [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
  ☆23Updated last year
• DD-DuDa / TensorRT-in-Action
  TensorRT-in-Action 是一个 GitHub 代码库，提供了使用 TensorRT 的代码示例，并有对应 Jupyter Notebook。
  ☆16Updated 2 years ago
• preacher-1 / MLA_tutorial
  from MHA, MQA, GQA to MLA by 苏剑林, with code
  ☆19Updated 3 months ago
• mdy666 / mdy_triton
  ☆131Updated last month
• FFY0 / AdaKV
  The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
  ☆76Updated 4 months ago
• Tlntin / trt2023
  ☆24Updated last year
• Ascend / AscendSpeed
  ☆79Updated last year
• shreyansh26 / FlashAttention-PyTorch
  Implementation of FlashAttention in PyTorch
  ☆150Updated 4 months ago
• Rayrtfr / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆17Updated last year
• caiwanxianhust / FasterLLaMA
  使用 CUDA C++ 实现的 llama 模型推理框架
  ☆57Updated 6 months ago
• zhzihao / QPruningKV
  More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
  ☆11Updated 4 months ago
• vllm-project / vllm-nccl
  Manages vllm-nccl dependency
  ☆17Updated last year
• CoinCheung / gdGPT
  Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
  ☆95Updated last year
• dhcode-cpp / NSA-pytorch
  DeepSeek Native Sparse Attention pytorch implementation
  ☆70Updated 3 months ago
• Tlntin / ChatGLM2-6B-TensorRT
  ☆90Updated last year
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆68Updated 5 months ago
• thuwyh / InferLight
  lightweighted deep learning inference service framework
  ☆39Updated 3 years ago
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆56Updated last year
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆99Updated 2 months ago
• Bruce-Lee-LY / decoding_attention
  Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.
  ☆36Updated 2 months ago
• luchangli03 / onnxsim_large_model
  simplify >2GB large onnx model
  ☆57Updated 6 months ago