Alternatives and similar repositories for ZhiLight

Users that are interested in ZhiLight are comparing it to the libraries listed below

• bytedance / ABQ-LLM
  An acceleration library that supports arbitrary bit-width combinatorial quantization operations
  ☆224Updated 8 months ago
• alibaba / rtp-llm
  RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
  ☆791Updated 2 weeks ago
• Qihoo360 / 360-LLaMA-Factory
  adds Sequence Parallelism into LLaMA-Factory
  ☆511Updated this week
• Zefan-Cai / KVCache-Factory
  Unified KV Cache Compression Methods for Auto-Regressive Models
  ☆1,138Updated 5 months ago
• NetEase-Media / grps_trtllm
  Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+…
  ☆140Updated last month
• hyperai / tvm-cn
  TVM Documentation in Chinese Simplified / TVM 中文文档
  ☆1,696Updated 2 months ago
• Simple-Efficient / RL-Factory
  Train your Agent model via our easy and efficient framework
  ☆1,144Updated this week
• FlagOpen / FlagPerf
  FlagPerf is an open-source software platform for benchmarking AI chips.
  ☆338Updated this week
• Tencent / KsanaLLM
  ☆334Updated 5 months ago
• xdit-project / xDiT
  xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism
  ☆2,028Updated this week
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆256Updated 3 weeks ago
• dhcode-cpp / X-R1
  minimal-cost for training 0.5B R1-Zero
  ☆742Updated last month
• Qcompiler / MIXQ
  MIXQ: Taming Dynamic Outliers in Mixed-Precision Quantization by Online Prediction
  ☆90Updated 7 months ago
• FlagOpen / FlagScale
  FlagScale is a large model toolkit based on open-sourced projects.
  ☆301Updated this week
• antgroup / glake
  GLake: optimizing GPU memory management and IO transmission.
  ☆467Updated 2 months ago
• NetEase-Media / grps
  Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming mo…
  ☆164Updated last month
• sgl-project / sgl-learning-materials
  Materials for learning SGLang
  ☆443Updated this week
• bytedance / flux
  A fast communication-overlapping library for tensor/expert parallelism on GPUs.
  ☆978Updated 3 weeks ago
• PaddleJitLab / CUDATutorial
  A self-learning tutorail for CUDA High Performance Programing.
  ☆641Updated 2 months ago
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆242Updated last year
• wei-potato / Train-llm-from-scratch
  使用deepspeed从头开始训练一个LLM,经过pretrain和sft阶段,验证llm学习知识、理解语言、回答问题的能力
  ☆154Updated 11 months ago
• DeepLink-org / deeplink.framework
  ☆67Updated 7 months ago
• bytedance / ByteTransformer
  optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052
  ☆474Updated last year
• DeepLink-org / DIOPI
  ☆70Updated 6 months ago
• BBuf / how-to-learn-deep-learning-framework
  how to learn PyTorch and OneFlow
  ☆434Updated last year
• harleyszhang / llm_note
  LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.
  ☆780Updated this week
• LLMServe / DistServe
  Disaggregated serving system for Large Language Models (LLMs).
  ☆617Updated 2 months ago
• FlagOpen / FlagGems
  FlagGems is an operator library for large language models implemented in the Triton Language.
  ☆573Updated this week
• MLSys-Learner-Resources / Awesome-MLSys-Blogger
  The repository has collected a batch of noteworthy MLSys bloggers (Algorithms/Systems)
  ☆241Updated 5 months ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆519Updated 3 weeks ago