Alternatives and similar repositories for AngelSlim

Users that are interested in AngelSlim are comparing it to the libraries listed below

• ModelTC / EasyLLM
  Built upon Megatron-Deepspeed and HuggingFace Trainer, EasyLLM has reorganized the code logic with a focus on usability. While enhancing …
  ☆48Updated 10 months ago
• Tlntin / qwen-ascend-llm
  ☆48Updated 8 months ago
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆102Updated 3 months ago
• DD-DuDa / BitDistiller
  [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.
  ☆115Updated last year
• madsys-dev / deepseekv2-profile
  ☆145Updated 4 months ago
• ruikangliu / FlatQuant
  [ICML 2025] Official PyTorch implementation of "FlatQuant: Flatness Matters for LLM Quantization"
  ☆148Updated this week
• ByteDance-Seed / decoupleQ
  A quantization algorithm for LLM
  ☆141Updated last year
• Rayrtfr / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆17Updated last year
• bytedance / AffineQuant
  Official implementation of the ICLR 2024 paper AffineQuant
  ☆26Updated last year
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆135Updated 3 months ago
• pzhao-eng / FlashMLA
  ☆50Updated last week
• luchangli03 / onnxsim_large_model
  simplify >2GB large onnx model
  ☆60Updated 7 months ago
• facebookresearch / LLM-QAT
  Code repo for the paper "LLM-QAT Data-Free Quantization Aware Training for Large Language Models"
  ☆304Updated 4 months ago
• shreyansh26 / FlashAttention-PyTorch
  Implementation of FlashAttention in PyTorch
  ☆154Updated 6 months ago
• ClubieDong / QAQ-KVCacheQuantization
  QAQ: Quality Adaptive Quantization for LLM KV Cache
  ☆51Updated last year
• ModelTC / LightCompress
  [EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a V…
  ☆516Updated this week
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆84Updated 7 months ago
• Macaronlin / LLaMA3-Quantization
  A repository dedicated to evaluating the performance of quantizied LLaMA3 using various quantization methods..
  ☆191Updated 6 months ago
• Bruce-Lee-LY / decoding_attention
  Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.
  ☆39Updated last month
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆261Updated last month
• weishengying / tiny-flash-attention
  使用 cutlass 实现 flash-attention 精简版，具有教学意义
  ☆44Updated 11 months ago
• weishengying / cute_gemm
  ☆14Updated 11 months ago
• inisis / OnnxLLM
  Large Language Model Onnx Inference Framework
  ☆36Updated 6 months ago
• harleyszhang / llm_counts
  llm theoretical performance analysis tools and support params, flops, memory and latency analysis.
  ☆99Updated last week
• DeepLink-org / dlinfer
  ☆52Updated last week
• luchangli03 / export_llama_to_onnx
  export llama to onnx
  ☆131Updated 6 months ago
• OpenPPL / ppl.nn.llm
  ☆139Updated last year
• Equationliu / Kangaroo
  [NeurIPS 2024] The official implementation of "Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exitin…
  ☆59Updated last year
• alipay / PainlessInferenceAcceleration
  Accelerate inference without tears
  ☆320Updated 4 months ago
• shishishu / LLM-Inference-Acceleration
  LLM Inference with Deep Learning Accelerator.
  ☆49Updated 6 months ago