Alternatives and similar repositories for Megatron-LM:

Users that are interested in Megatron-LM are comparing it to the libraries listed below

• ROCm / rccl-tests
  RCCL Performance Benchmark Tests
  ☆59Updated last month
• merthidayetoglu / HiCCL
  A hierarchical collective communications library with portable optimizations
  ☆29Updated 2 months ago
• ROCm / TransformerEngine
  ☆20Updated this week
• Azure / msccl
  Microsoft Collective Communication Library
  ☆62Updated 3 months ago
• ROCm / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆67Updated this week
• ROCm / pytorch-micro-benchmarking
  ☆19Updated 3 months ago
• ROCm / rocmProfileData
  ☆19Updated last week
• ROCm / amd_matrix_instruction_calculator
  A tool for generating information about the matrix multiplication instructions in AMD Radeon™ and AMD Instinct™ accelerators
  ☆76Updated last year
• Azure / msccl-executor-nccl
  ☆36Updated 2 months ago
• triton-lang / kernels
  ☆71Updated 3 months ago
• HabanaAI / vllm-fork
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆56Updated this week
• ColfaxResearch / cfx-article-src
  ☆75Updated last week
• intel / torch-xpu-ops
  ☆34Updated this week
• sunlex0717 / DissectingTensorCores
  ☆89Updated 10 months ago
• yifuwang / symm-mem-recipes
  ☆48Updated 2 months ago
• codeplaysoftware / cutlass-fork
  CUDA Templates for Linear Algebra Subroutines
  ☆14Updated this week
• ROCm / TransferBench
  TransferBench is a utility capable of benchmarking simultaneous copies between user-specified devices (CPUs/GPUs)
  ☆39Updated this week
• ColfaxResearch / cutlass-kernels
  ☆185Updated 7 months ago
• wangsiping97 / FastGEMV
  High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.
  ☆100Updated 7 months ago
• ROCm / rdc
  RDC
  ☆27Updated this week
• parasailteam / coconet
  ☆75Updated 2 years ago
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆103Updated last month
• flexflow / flexflow-serve
  FlexFlow Serve: Low-Latency, High-Performance LLM Serving
  ☆22Updated this week
• microsoft / mscclpp
  MSCCL++: A GPU-driven communication stack for scalable AI applications
  ☆304Updated this week
• ParCIS / Chimera
  Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.
  ☆60Updated last year
• alibaba / llm-scheduling-artifact
  Artifact of OSDI '24 paper, ”Llumnix: Dynamic Scheduling for Large Language Model Serving“
  ☆60Updated 8 months ago
• microsoft / nnscaler
  nnScaler: Compiling DNN models for Parallel Training
  ☆97Updated 2 weeks ago
• HabanaAI / Habana_Custom_Kernel
  Provides the examples to write and build Habana custom kernels using the HabanaTools
  ☆20Updated 3 months ago
• microsoft / SuperScaler
  An experimental parallel training platform
  ☆54Updated 11 months ago