Alternatives and similar repositories for pytorch_distributed:

Users that are interested in pytorch_distributed are comparing it to the libraries listed below

• TiledTensor / TiledBench
  Benchmark tests supporting the TiledCUDA library.
  ☆15Updated 4 months ago
• L1aoXingyu / llm-infer-bench
  ☆11Updated last year
• BBuf / flash-rwkv
  ☆30Updated 10 months ago
• microsoft / AttentionEngine
  ☆52Updated this week
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆68Updated 9 months ago
• megvii-research / IntLLaMA
  IntLLaMA: A fast and light quantization solution for LLaMA
  ☆18Updated last year
• bojone / softtopk
  differentiable top-k operator
  ☆21Updated 3 months ago
• ModelTC / awesome-lm-system
  Summary of system papers/frameworks/codes/tools on training or serving large model
  ☆56Updated last year
• yester31 / Cutlass_EX
  study of cutlass
  ☆21Updated 4 months ago
• Harry-Chen / InfMoE
  Inference framework for MoE layers based on TensorRT with Python binding
  ☆41Updated 3 years ago
• yuzhenmao / IceFormer
  Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).
  ☆22Updated 9 months ago
• megvii-research / basedet
  An object detection codebase based on MegEngine.
  ☆28Updated 2 years ago
• cassiewilliam / cuda_op_benchmark
  方便扩展的Cuda算子理解和优化框架，仅用在学习使用
  ☆13Updated 9 months ago
• pprp / Awesome-Efficient-MoE
  Efficient Mixture of Experts for LLM Paper List
  ☆47Updated 3 months ago
• BBuf / megatron-lm-parallel-group-playground
  ☆15Updated last year
• zhuzilin / pytorch-malloc
  An external memory allocator example for PyTorch.
  ☆14Updated 3 years ago
• GeeeekExplorer / cupytorch
  A small framework mimics PyTorch using CuPy or NumPy
  ☆27Updated 3 years ago
• lvyufeng / easy_mindspore_bk
  ☆18Updated 2 years ago
• zxytim / arithmetic-encoding-compression
  ☆11Updated last year
• Aleph-Alpha / NeurIPS-WANT-submission-efficient-parallelization-layouts
  ☆22Updated last year
• li199603 / sgemm_with_cuda
  ☆17Updated last year
• OpenGVLab / InternLMM
  ☆17Updated last year
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆61Updated last year
• LeiWang1999 / AutoGPTQ.tvm
  GPTQ inference TVM kernel
  ☆38Updated 11 months ago
• Oneflow-Inc / vision
  Datasets, Transforms and Models specific to Computer Vision
  ☆85Updated last year
• Oneflow-Inc / serving
  OneFlow Serving
  ☆20Updated 3 months ago
• Bruce-Lee-LY / decoding_attention
  Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.
  ☆35Updated 3 weeks ago
• feifeibear / ChituAttention
  Quantized Attention on GPU
  ☆45Updated 4 months ago
• intelligent-machine-learning / atorch
  An industrial extension library of pytorch to accelerate large scale model training
  ☆29Updated last month
• sustcsonglin / fla-tilelang
  ☆19Updated 3 weeks ago