Alternatives and similar repositories for AngelSlim

Users that are interested in AngelSlim are comparing it to the libraries listed below

• JT-Ushio / MHA2MLA
  Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
  ☆204Updated 2 months ago
• stepfun-ai / Step3
  ☆449Updated 5 months ago
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆225Updated 3 weeks ago
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆274Updated 6 months ago
• qingkelab / qingketalk
  青稞Talk
  ☆190Updated 2 weeks ago
• ByteDance-Seed / decoupleQ
  A quantization algorithm for LLM
  ☆148Updated last year
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆251Updated last year
• zai-org / GLM-Edge
  GLM Series Edge Models
  ☆158Updated 7 months ago
• neuralmagic / AutoFP8
  ☆206Updated 9 months ago
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆154Updated 5 months ago
• ModelTC / LightCompress
  [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.
  ☆675Updated 2 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆139Updated last year
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆153Updated last year
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆110Updated 10 months ago
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆166Updated 4 months ago
• flagos-ai / FlagScale
  FlagScale is a large model toolkit based on open-sourced projects.
  ☆474Updated this week
• TencentARC / mllm-npu
  mllm-npu: training multimodal large language models on Ascend NPUs
  ☆95Updated last year
• luchangli03 / onnxsim_large_model
  simplify >2GB large onnx model
  ☆70Updated last year
• NVlabs / COAT
  [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training
  ☆258Updated 6 months ago
• intelligent-machine-learning / atorch
  An industrial extension library of pytorch to accelerate large scale model training
  ☆58Updated 5 months ago
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆184Updated 10 months ago
• Adlik / smoothquantplus
  [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
  ☆23Updated last year
• modelscope / easydistill
  a toolkit on knowledge distillation for large language models
  ☆266Updated this week
• inclusionAI / Ling-V2
  Ling-V2 is a MoE LLM provided and open-sourced by InclusionAI.
  ☆253Updated 4 months ago
• OpenGVLab / EfficientQAT
  [ACL 2025 Main] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆327Updated 2 months ago
• OpenBMB / infllmv2_cuda_impl
  ☆87Updated last week
• NVIDIA / Star-Attention
  Efficient LLM Inference over Long Sequences
  ☆394Updated 7 months ago
• MuLabPKU / TransMLA
  TransMLA: Multi-Head Latent Attention Is All You Need (NeurIPS 2025 Spotlight)
  ☆429Updated 4 months ago
• mit-han-lab / duo-attention
  [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
  ☆524Updated 11 months ago
• DeepLink-org / dlinfer
  ☆74Updated this week