Alternatives and similar repositories for PaLM-colossalai

Users that are interested in PaLM-colossalai are comparing it to the libraries listed below

• AI21Labs / Parallel-Context-Windows
  ☆105Updated 2 years ago
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆56Updated 2 years ago
• LAION-AI / Open-Instruction-Generalist
  Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks
  ☆209Updated last year
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆96Updated 2 years ago
• Dahoas / reward-modeling
  ☆98Updated 2 years ago
• hpcaitech / ColossalAI-Examples
  Examples of training models with hybrid parallelism using ColossalAI
  ☆339Updated 2 years ago
• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆565Updated last year
• HuangLK / transpeeder
  train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism
  ☆224Updated last year
• CoinCheung / gdGPT
  Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
  ☆98Updated last year
• bigscience-workshop / data-preparation
  Code used for sourcing and cleaning the BigScience ROOTS corpus
  ☆315Updated 2 years ago
• fpgaminer / GPTQ-triton
  GPTQ inference Triton kernel
  ☆309Updated 2 years ago
• RulinShao / LightSeq
  Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training
  ☆215Updated last year
• qhliu26 / Dive-into-Big-Model-Training
  📑 Dive into Big Model Training
  ☆114Updated 2 years ago
• ProjectD-AI / LLaMA-Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆68Updated 2 years ago
• huggingface / transformers_bloom_parallel
  Techniques used to run BLOOM at inference in parallel
  ☆37Updated 2 years ago
• anyscale / llm-continuous-batching-benchmarks
  ☆121Updated last year
• princeton-nlp / CoFiPruning
  [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
  ☆197Updated 2 years ago
• SimiaoZuo / MoEBERT
  This PyTorch package implements MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation (NAACL 2022).
  ☆112Updated 3 years ago
• THUDM / icetk
  A unified tokenization tool for Images, Chinese and English.
  ☆151Updated 2 years ago
• zsc / llama_infer
  Inference script for Meta's LLaMA models using Hugging Face wrapper
  ☆109Updated 2 years ago
• p-lambda / dsir
  DSIR large-scale data selection framework for language model training
  ☆259Updated last year
• SeanNaren / minGPT
  A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!
  ☆113Updated 2 years ago
• Glaciohound / LM-Infinite
  Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆148Updated 6 months ago
• imoneoi / multipack
  Multipack distributed sampler for fast padding-free training of LLMs
  ☆201Updated last year
• shjwudp / c4-dataset-script
  Inspired by google c4, here is a series of colossal clean data cleaning scripts focused on CommonCrawl data processing. Including Chinese…
  ☆131Updated 2 years ago
• thomfoster / minRLHF
  A (somewhat) minimal library for finetuning language models with PPO on human feedback.
  ☆86Updated 2 years ago
• FasterDecoding / REST
  REST: Retrieval-Based Speculative Decoding, NAACL 2024
  ☆210Updated last month
• dropreg / efficient_alpaca
  The aim of this repository is to utilize LLaMA to reproduce and enhance the Stanford Alpaca
  ☆98Updated 2 years ago
• huggingface / datablations
  Scaling Data-Constrained Language Models
  ☆342Updated 3 months ago
• Langboat / mengzi-retrieval-lm
  An experimental implementation of the retrieval-enhanced language model
  ☆75Updated 2 years ago