DS3Lab / DT-FM

Related projects ⓘ

Alternatives and complementary repositories for DT-FM

• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆74Updated last year
• SymbioticLab / Oobleck
  A resilient distributed training framework
  ☆85Updated 7 months ago
• thu-pacman / FasterMoE
  ☆70Updated 2 years ago
• DachengLi1 / AMP
  (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.
  ☆34Updated 2 years ago
• zhuohan123 / terapipe
  ☆65Updated 3 years ago
• hao-ai-lab / MuxServe
  ☆46Updated 5 months ago
• RulinShao / FastCkpt
  Python package for rematerialization-aware gradient checkpointing
  ☆23Updated last year
• TorchMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆103Updated 3 months ago
• LiuXiaoxuanPKU / OSD
  ☆35Updated 3 months ago
• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆195Updated 3 months ago
• DS3Lab / Decentralized_FM_alpha
  ☆19Updated last year
• HPMLL / BurstGPT
  A ChatGPT(GPT-3.5) & GPT-4 Workload Trace to Optimize LLM Serving Systems
  ☆132Updated last month
• stanford-futuredata / stk
  ☆88Updated 2 months ago
• alpa-projects / mms
  AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving (OSDI 23)
  ☆78Updated last year
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆61Updated 3 weeks ago
• Hsword / SpotServe
  SpotServe: Serving Generative Large Language Models on Preemptible Instances
  ☆101Updated 8 months ago
• tonyzhao-jt / LLM-PQ
  Official Repo for "LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization"
  ☆27Updated 8 months ago
• DS3Lab / AC-SGD
  Code associated with the paper **Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees**.
  ☆26Updated last year
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆147Updated 4 months ago
• microsoft / nnscaler
  nnScaler: Compiling DNN models for Parallel Training
  ☆74Updated 3 weeks ago
• SymbioticLab / ModelKeeper
  A Cluster-Wide Model Manager to Accelerate DNN Training via Automated Training Warmup
  ☆34Updated last year
• spcl / substation
  Research and development for optimizing transformers
  ☆125Updated 3 years ago
• Distributed-AI / PipeTransformer
  PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021
  ☆55Updated 3 years ago
• AlibabaResearch / flash-llm
  Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
  ☆180Updated last year
• kssteven418 / BigLittleDecoder
  [NeurIPS'23] Speculative Decoding with Big Little Decoder
  ☆86Updated 9 months ago
• YaoJiayi / CacheBlend
  ☆51Updated last month
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆185Updated last month
• Relaxed-System-Lab / HexGen
  [ICML 2024] Serving LLMs on heterogeneous decentralized clusters.
  ☆15Updated 6 months ago
• alibaba / easydist
  Automated Parallelization System and Infrastructure for Multiple Ecosystems
  ☆75Updated this week
• Shigangli / Chimera
  Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.
  ☆46Updated 11 months ago