Alternatives and similar repositories for alpa

Users that are interested in alpa are comparing it to the libraries listed below

• facebookincubator / AITemplate
  AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (N…
  ☆4,695Updated last week
• ELS-RD / kernl
  Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackab…
  ☆1,586Updated last year
• NVIDIA / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆6,382Updated last year
• deepspeedai / DeepSpeed-MII
  MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
  ☆2,088Updated 6 months ago
• flexflow / flexflow-train
  Automatically Discovering Fast Parallelization Strategies for Distributed Deep Neural Network Training
  ☆1,854Updated last week
• CarperAI / trlx
  A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
  ☆4,736Updated 2 years ago
• google-research / t5x
  ☆2,937Updated this week
• young-geng / EasyLM
  Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…
  ☆2,508Updated last year
• facebookresearch / fairscale
  PyTorch extensions for high performance and large scale training.
  ☆3,393Updated 8 months ago
• deepspeedai / Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆2,220Updated 5 months ago
• bigscience-workshop / Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆1,431Updated last year
• IST-DASLab / gptq
  Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
  ☆2,245Updated last year
• bigscience-workshop / bigscience
  Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
  ☆1,009Updated last year
• facebookresearch / metaseq
  Repo for external large-scale work
  ☆6,543Updated last year
• bitsandbytes-foundation / bitsandbytes
  Accessible large language models via k-bit quantization for PyTorch.
  ☆7,896Updated this week
• pytorch / torchdynamo
  A Python-level JIT compiler designed to make unmodified PyTorch programs faster.
  ☆1,071Updated last year
• FMInference / FlexLLMGen
  Running large language models on a single GPU for throughput-oriented scenarios.
  ☆9,380Updated last year
• qwopqwop200 / GPTQ-for-LLaMa
  4 bits quantization of LLaMA using GPTQ
  ☆3,077Updated last year
• NVIDIA / TransformerEngine
  A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on H…
  ☆3,081Updated last week
• EleutherAI / pythia
  The hub for EleutherAI's work on interpretability and learning dynamics
  ☆2,712Updated 2 months ago
• tensorflow / mesh
  Mesh TensorFlow: Model Parallelism Made Easier
  ☆1,624Updated 2 years ago
• EleutherAI / the-pile
  ☆1,627Updated 2 years ago
• microsoft / mup
  maximal update parametrization (µP)
  ☆1,657Updated last year
• microsoft / torchscale
  Foundation Architecture for (M)LLMs
  ☆3,131Updated last year
• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆566Updated last year
• lucidrains / toolformer-pytorch
  Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI
  ☆2,057Updated last year
• openxla / xla
  A machine learning compiler for GPUs, CPUs, and ML accelerators
  ☆3,892Updated this week
• hpcaitech / EnergonAI
  Large-scale model inference.
  ☆627Updated 2 years ago
• JonasGeiping / cramming
  Cramming the training of a (BERT-type) language model into limited compute.
  ☆1,361Updated last year
• togethercomputer / RedPajama-Data
  The RedPajama-Data repository contains code for preparing large datasets for training large language models.
  ☆4,914Updated last year