Alternatives and similar repositories for AngelSlim

Users that are interested in AngelSlim are comparing it to the libraries listed below

• JT-Ushio / MHA2MLA
  Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
  ☆198Updated 2 weeks ago
• stepfun-ai / Step3
  ☆440Updated 4 months ago
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆212Updated 2 months ago
• ByteDance-Seed / decoupleQ
  A quantization algorithm for LLM
  ☆146Updated last year
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆270Updated 4 months ago
• intelligent-machine-learning / atorch
  An industrial extension library of pytorch to accelerate large scale model training
  ☆55Updated 4 months ago
• qingkelab / qingketalk
  青稞Talk
  ☆175Updated last week
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆250Updated last year
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆151Updated 4 months ago
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆110Updated 8 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆137Updated last year
• OpenGVLab / EfficientQAT
  [ACL 2025 Main] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆319Updated 3 weeks ago
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆149Updated 2 months ago
• sgl-project / SpecForge
  Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
  ☆557Updated this week
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆184Updated 8 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆148Updated 10 months ago
• ModelTC / LightCompress
  [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLM, VLM, and video generation models.
  ☆638Updated last month
• zhaochenyang20 / ModelServer
  Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang
  ☆61Updated last year
• mit-han-lab / duo-attention
  [ICLR 2025] DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
  ☆515Updated 10 months ago
• NVlabs / COAT
  [ICLR 2025] COAT: Compressing Optimizer States and Activation for Memory-Efficient FP8 Training
  ☆252Updated 4 months ago
• vllm-project / vllm-omni
  A framework for efficient model inference with omni-modality models
  ☆927Updated this week
• zai-org / GLM-Edge
  GLM Series Edge Models
  ☆156Updated 6 months ago
• Efficient-ML / Qwen3-Quantization
  ☆65Updated 3 months ago
• flagos-ai / FlagScale
  FlagScale is a large model toolkit based on open-sourced projects.
  ☆426Updated this week
• inclusionAI / Ling-V2
  Ling-V2 is a MoE LLM provided and open-sourced by InclusionAI.
  ☆248Updated 2 months ago
• neuralmagic / AutoFP8
  ☆206Updated 7 months ago
• TencentARC / mllm-npu
  mllm-npu: training multimodal large language models on Ascend NPUs
  ☆95Updated last year
• thunlp / Delta-CoMe
  Delta-CoMe can achieve near loss-less 1-bit compressin which has been accepted by NeurIPS 2024
  ☆59Updated last year
• pangu-tech / pangu-ultra
  ☆74Updated 6 months ago
• madsys-dev / deepseekv2-profile
  ☆154Updated 9 months ago