Alternatives and similar repositories for Megatron-LM:

Users that are interested in Megatron-LM are comparing it to the libraries listed below

• Dao-AILab / flash-attention
  Fast and memory-efficient exact attention
  ☆16,835Updated this week
• huggingface / accelerate
  🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (i…
  ☆8,608Updated this week
• NVIDIA / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆6,116Updated last year
• huggingface / peft
  🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
  ☆18,082Updated this week
• huggingface / trl
  Train transformer language models with reinforcement learning.
  ☆13,166Updated this week
• deepspeedai / DeepSpeed
  DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
  ☆37,834Updated this week
• facebookresearch / fairscale
  PyTorch extensions for high performance and large scale training.
  ☆3,293Updated this week
• bitsandbytes-foundation / bitsandbytes
  Accessible large language models via k-bit quantization for PyTorch.
  ☆6,901Updated this week
• artidoro / qlora
  QLoRA: Efficient Finetuning of Quantized LLMs
  ☆10,366Updated 10 months ago
• triton-lang / triton
  Development repository for the Triton language and compiler
  ☆15,146Updated this week
• microsoft / unilm
  Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities
  ☆21,052Updated last month
• microsoft / LoRA
  Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
  ☆11,699Updated 3 months ago
• google-research / text-to-text-transfer-transformer
  Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"
  ☆6,325Updated last month
• google / sentencepiece
  Unsupervised text tokenizer for Neural Network-based text generation.
  ☆10,771Updated last week
• EleutherAI / lm-evaluation-harness
  A framework for few-shot evaluation of language models.
  ☆8,595Updated this week
• jessevig / bertviz
  BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)
  ☆7,306Updated last year
• karpathy / minGPT
  A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
  ☆21,717Updated 7 months ago
• deepspeedai / DeepSpeedExamples
  Example models using DeepSpeed
  ☆6,414Updated 2 weeks ago
• facebookresearch / xformers
  Hackable and optimized Transformers building blocks, supporting a composable construction.
  ☆9,319Updated this week
• deepspeedai / Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆2,042Updated 3 weeks ago
• triton-inference-server / server
  The Triton Inference Server provides an optimized cloud and edge inferencing solution.
  ☆9,037Updated this week
• NVIDIA / NeMo
  A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Auto…
  ☆13,613Updated this week
• CarperAI / trlx
  A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
  ☆4,621Updated last year
• vllm-project / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆44,418Updated this week
• UKPLab / sentence-transformers
  State-of-the-Art Text Embeddings
  ☆16,415Updated last week
• huggingface / text-generation-inference
  Large Language Model Text Generation Inference
  ☆9,992Updated this week
• google-research / t5x
  ☆2,782Updated this week
• EleutherAI / gpt-neox
  An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
  ☆7,154Updated last week
• facebookresearch / metaseq
  Repo for external large-scale work
  ☆6,522Updated 11 months ago
• AutoGPTQ / AutoGPTQ
  An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
  ☆4,802Updated 3 weeks ago