Alternatives and similar repositories for AngelSlim

Users that are interested in AngelSlim are comparing it to the libraries listed below

• zai-org / GLM-Edge
  GLM Series Edge Models
  ☆149Updated 3 months ago
• OpenBMB / CPM.cu
  CPM.cu is a lightweight, high-performance CUDA implementation for LLMs, optimized for end-device inference and featuring cutting-edge tec…
  ☆178Updated last week
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆136Updated last year
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆247Updated last year
• JT-Ushio / MHA2MLA
  Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs
  ☆188Updated 2 months ago
• Adlik / smoothquantplus
  [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
  ☆23Updated last year
• woct0rdho / transformers-qwen3-moe-fused
  Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth
  ☆171Updated this week
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆264Updated last month
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.
  ☆177Updated 5 months ago
• thunlp / Delta-CoMe
  Delta-CoMe can achieve near loss-less 1-bit compressin which has been accepted by NeurIPS 2024
  ☆56Updated 9 months ago
• OpenBMB / infllmv2_cuda_impl
  ☆46Updated this week
• zhaochenyang20 / ModelServer
  Efficient, Flexible, and Highly Fault-Tolerant Model Service Management Based on SGLang
  ☆56Updated 10 months ago
• ByteDance-Seed / decoupleQ
  A quantization algorithm for LLM
  ☆141Updated last year
• pangu-tech / pangu-ultra
  ☆67Updated 3 months ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆189Updated last year
• Efficient-ML / Qwen3-Quantization
  ☆39Updated 3 months ago
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆67Updated 2 years ago
• SkyworkAI / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆17Updated last year
• rlite-project / RLite
  A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…
  ☆67Updated 2 weeks ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆136Updated 7 months ago
• intelligent-machine-learning / atorch
  An industrial extension library of pytorch to accelerate large scale model training
  ☆49Updated 3 weeks ago
• qingkelab / qingketalk
  青稞Talk
  ☆141Updated this week
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆139Updated 3 weeks ago
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆105Updated 5 months ago
• neuralmagic / AutoFP8
  ☆197Updated 4 months ago
• casper-hansen / AutoAWQ_kernels
  ☆78Updated 9 months ago
• TencentARC / mllm-npu
  mllm-npu: training multimodal large language models on Ascend NPUs
  ☆91Updated last year
• OpenGVLab / EfficientQAT
  [ACL 2025 Main] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  ☆301Updated 3 months ago
• bigai-nlco / TokenSwift
  [ICML 2025] |TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation
  ☆113Updated 3 months ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year