Alternatives and similar repositories for PaLM-colossalai

Users that are interested in PaLM-colossalai are comparing it to the libraries listed below

• hpcaitech / ColossalAI-Benchmark
  Performance benchmarking with ColossalAI
  ☆39Updated 2 years ago
• RulinShao / LightSeq
  Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
  ☆208Updated 9 months ago
• AI21Labs / Parallel-Context-Windows
  ☆105Updated last year
• HuangLK / transpeeder
  train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism
  ☆220Updated last year
• fpgaminer / GPTQ-triton
  GPTQ inference Triton kernel
  ☆300Updated 2 years ago
• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆564Updated 7 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆118Updated last year
• hpcaitech / ColossalAI-Examples
  Examples of training models with hybrid parallelism using ColossalAI
  ☆340Updated 2 years ago
• LAION-AI / Open-Instruction-Generalist
  Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks
  ☆209Updated last year
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆57Updated last year
• bytedance / effective_transformer
  Running BERT without Padding
  ☆471Updated 3 years ago
• THUDM / icetk
  A unified tokenization tool for Images, Chinese and English.
  ☆152Updated 2 years ago
• qhliu26 / Dive-into-Big-Model-Training
  📑 Dive into Big Model Training
  ☆113Updated 2 years ago
• Dahoas / reward-modeling
  ☆96Updated 2 years ago
• huggingface / transformers_bloom_parallel
  Techniques used to run BLOOM at inference in parallel
  ☆37Updated 2 years ago
• ProjectD-AI / LLaMA-Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆68Updated last year
• CoinCheung / gdGPT
  Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
  ☆95Updated last year
• epfml / dynamic-sparse-flash-attention
  ☆144Updated 2 years ago
• lxe / llama-tune
  LLaMa Tuning with Stanford Alpaca Dataset using Deepspeed and Transformers
  ☆51Updated 2 years ago
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆95Updated last year
• imoneoi / multipack_sampler
  Multipack distributed sampler for fast padding-free training of LLMs
  ☆188Updated 9 months ago
• bigscience-workshop / data-preparation
  Code used for sourcing and cleaning the BigScience ROOTS corpus
  ☆313Updated 2 years ago
• hpcaitech / EnergonAI
  Large-scale model inference.
  ☆630Updated last year
• bojone / rerope
  Rectified Rotary Position Embeddings
  ☆367Updated last year
• Ascend / AscendSpeed
  ☆79Updated last year
• facebookresearch / LLM-QAT
  Code repo for the paper "LLM-QAT Data-Free Quantization Aware Training for Large Language Models"
  ☆287Updated 3 months ago
• triton-inference-server / fastertransformer_backend
  ☆411Updated last year
• lucidrains / triton-transformer
  Implementation of a Transformer, but completely in Triton
  ☆265Updated 3 years ago
• hpcaitech / Titans
  A collection of models built with ColossalAI
  ☆32Updated 2 years ago
• Langboat / mengzi-retrieval-lm
  An experimental implementation of the retrieval-enhanced language model
  ☆74Updated 2 years ago