Alternatives and similar repositories for LargeScale:

Users that are interested in LargeScale are comparing it to the libraries listed below

• Harry-Chen / InfMoE
  Inference framework for MoE layers based on TensorRT with Python binding
  ☆41Updated 3 years ago
• Ascend / AscendSpeed
  ☆78Updated last year
• MayDomine / Burst-Attention
  Distributed IO-aware Attention algorithm
  ☆20Updated 8 months ago
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆68Updated 10 months ago
• OpenBMB / cpm_kernels
  ☆23Updated last year
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated last year
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Updated 10 months ago
• ProjectD-AI / LLaMA-Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆69Updated last year
• Oneflow-Inc / one-glm
  A more efficient GLM implementation!
  ☆55Updated 2 years ago
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆62Updated last year
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆56Updated last year
• ModelTC / awesome-lm-system
  Summary of system papers/frameworks/codes/tools on training or serving large model
  ☆56Updated last year
• vllm-project / vllm-nccl
  Manages vllm-nccl dependency
  ☆17Updated 10 months ago
• THUDM / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆39Updated 2 years ago
• BBuf / megatron-lm-parallel-group-playground
  ☆16Updated last year
• sail-sg / VocabularyParallelism
  Vocabulary Parallelism
  ☆17Updated last month
• Aleph-Alpha / NeurIPS-WANT-submission-efficient-parallelization-layouts
  ☆22Updated last year
• Rayrtfr / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆17Updated last year
• CoinCheung / gdGPT
  Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
  ☆95Updated last year
• eedalong / Dpex
  Distributed DataLoader For Pytorch Based On Ray
  ☆24Updated 3 years ago
• void-main / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆59Updated last year
• LeeJuly30 / BERTCpp
  implement bert in pure c++
  ☆36Updated 4 years ago
• CLUEbenchmark / SuperCLUE-Math6
  SuperCLUE-Math6：新一代中文原生多轮多步数学推理数据集的探索之旅
  ☆54Updated last year
• bojone / analytical-classification
  逻辑回归和单层softmax的解析解
  ☆12Updated 3 years ago
• L1aoXingyu / llm-infer-bench
  ☆11Updated last year
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆63Updated last year
• keezen / ntk_alibi
  NTK scaled version of ALiBi position encoding in Transformer.
  ☆67Updated last year
• anyscale / llm-continuous-batching-benchmarks
  ☆117Updated last year
• Zheng0428 / COIG-Kun
  ☆36Updated 7 months ago
• WalkerMitty / Fast-Llama2
  Fast instruction tuning with Llama2
  ☆11Updated last year