Alternatives and similar repositories for Megatron-DeepSpeed

Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below

• deepspeedai / DeepSpeed-Kernels
  ☆71Updated 7 months ago
• graphcore-research / unit-scaling-demo
  Unit Scaling demo and experimentation code
  ☆16Updated last year
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆121Updated 11 months ago
• stanford-futuredata / stk
  ☆113Updated last year
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆131Updated 11 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆42Updated last year
• argonne-lcf / LLM-Inference-Bench
  LLM-Inference-Bench
  ☆57Updated 4 months ago
• axonn-ai / axonn
  Parallel framework for training and fine-tuning deep neural networks
  ☆68Updated last week
• sgl-project / tensorrt-demo
  TensorRT LLM Benchmark Configuration
  ☆13Updated last year
• vedantroy / gpu_kernels
  ☆27Updated last year
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆85Updated last year
• chu-tianxiang / QuIP-for-all
  QuIP quantization
  ☆60Updated last year
• ROCm / TransformerEngine
  ☆51Updated this week
• intel / intel-extension-for-deepspeed
  Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…
  ☆63Updated 4 months ago
• HabanaAI / Model-References
  Reference models for Intel(R) Gaudi(R) AI Accelerator
  ☆167Updated last month
• SqueezeBits / QUICK
  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference
  ☆118Updated last year
• groq / mlagility
  Machine Learning Agility (MLAgility) benchmark and benchmarking tools
  ☆40Updated 3 months ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆27Updated 2 years ago
• tridao / flash-attention-wheels
  ☆57Updated last year
• IBM / triton-dejavu
  Framework to reduce autotune overhead to zero for well known deployments.
  ☆85Updated 2 months ago
• foundation-model-stack / foundation-model-stack
  🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.
  ☆216Updated last week
• tile-ai / tvm
  Open deep learning compiler stack for cpu, gpu and specialized accelerators
  ☆19Updated this week
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆40Updated last year
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆68Updated 11 months ago
• vllm-project / dashboard
  vLLM performance dashboard
  ☆37Updated last year
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆170Updated last year
• tgale96 / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆127Updated 5 months ago
• NVIDIA / LDDL
  Distributed preprocessing and data loading for language datasets
  ☆39Updated last year
• IST-DASLab / Quartet
  ☆107Updated this week
• INT-FlashAttention2024 / INT-FlashAttention
  ☆83Updated 9 months ago