Alternatives and similar repositories for oh-my-server:

Users that are interested in oh-my-server are comparing it to the libraries listed below

• LiuXiaoxuanPKU / GACT-ICML
  ☆42Updated 2 years ago
• zhuohan123 / terapipe
  ☆72Updated 3 years ago
• ParCIS / Chimera
  Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.
  ☆62Updated 3 weeks ago
• thu-pacman / FasterMoE
  ☆82Updated 3 years ago
• hpcaitech / ColossalAI-Benchmark
  Performance benchmarking with ColossalAI
  ☆39Updated 2 years ago
• microsoft / nnscaler
  nnScaler: Compiling DNN models for Parallel Training
  ☆106Updated 2 months ago
• feifeibear / ChituAttention
  Quantized Attention on GPU
  ☆45Updated 4 months ago
• UofT-EcoSystem / Tempo
  Memory footprint reduction for transformer models
  ☆11Updated 2 years ago
• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆207Updated 7 months ago
• tgale96 / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆81Updated 5 months ago
• NUS-HPC-AI-Lab / LARS-ImageNet-PyTorch
  Accuracy 77%. Large batch deep learning optimizer LARS for ImageNet with PyTorch and ResNet, using Horovod for distribution. Optional acc…
  ☆38Updated 3 years ago
• DachengLi1 / AMP
  (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.
  ☆38Updated 2 years ago
• cat538 / SKVQ
  SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
  ☆19Updated 6 months ago
• uwsampl / dtr-prototype
  Dynamic Tensor Rematerialization prototype (modified PyTorch) and simulator. Paper: https://arxiv.org/abs/2006.09616
  ☆132Updated last year
• fanshiqing / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆118Updated 3 months ago
• yuezhouhu / 2by4-pretrain
  Efficient 2:4 sparse training algorithms and implementations
  ☆54Updated 4 months ago
• alibaba / easydist
  Automated Parallelization System and Infrastructure for Multiple Ecosystems
  ☆79Updated 4 months ago
• NUS-HPC-AI-Lab / pytorch-lamb
  PyTorch implementation of LAMB for ImageNet/ResNet-50 training
  ☆13Updated 3 years ago
• MayDomine / Seq1F1B
  Sequence-level 1F1B schedule for LLMs.
  ☆17Updated 10 months ago
• Youhe-Jiang / IJCAI2023-OptimalShardedDataParallel
  [IJCAI2023] An automated parallel training system that combines the advantages from both data and model parallelism. If you have any inte…
  ☆51Updated last year
• andy-yang-1 / DoubleSparse
  16-fold memory access reduction with nearly no loss
  ☆89Updated 3 weeks ago
• mit-han-lab / Block-Sparse-Attention
  A sparse attention kernel supporting mix sparse patterns
  ☆192Updated 2 months ago
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆68Updated 10 months ago
• DD-DuDa / BitDecoding
  A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.
  ☆32Updated 3 weeks ago
• xuqifan897 / Optimus
  ☆27Updated 3 years ago
• machilusZ / FastGen
  This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
  ☆36Updated 8 months ago
• kwai / Megatron-Kwai
  [USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Paral…
  ☆51Updated 8 months ago
• Gaffey / ExCP
  Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".
  ☆47Updated 9 months ago
• ByteDance-Seed / ByteCheckpoint
  ByteCheckpoint: An Unified Checkpointing Library for LFMs
  ☆184Updated 2 weeks ago
• CalvinXKY / mfu_calculation
  A simple calculation for LLM MFU.
  ☆34Updated last month