Alternatives and similar repositories for PaLM-colossalai:

Users that are interested in PaLM-colossalai are comparing it to the libraries listed below

• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆204Updated 5 months ago
• hpcaitech / ColossalAI-Benchmark
  Performance benchmarking with ColossalAI
  ☆39Updated 2 years ago
• AI21Labs / Parallel-Context-Windows
  ☆106Updated last year
• anyscale / llm-continuous-batching-benchmarks
  ☆114Updated 10 months ago
• fpgaminer / GPTQ-triton
  GPTQ inference Triton kernel
  ☆292Updated last year
• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆563Updated 3 months ago
• Dahoas / reward-modeling
  ☆96Updated last year
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆102Updated 2 months ago
• hpcaitech / ColossalAI-Examples
  Examples of training models with hybrid parallelism using ColossalAI
  ☆337Updated last year
• bytedance / effective_transformer
  Running BERT without Padding
  ☆468Updated 2 years ago
• qhliu26 / Dive-into-Big-Model-Training
  📑 Dive into Big Model Training
  ☆110Updated 2 years ago
• FasterDecoding / REST
  REST: Retrieval-Based Speculative Decoding, NAACL 2024
  ☆190Updated last month
• THUDM / icetk
  A unified tokenization tool for Images, Chinese and English.
  ☆151Updated last year
• triton-inference-server / fastertransformer_backend
  ☆411Updated last year
• bigscience-workshop / data-preparation
  Code used for sourcing and cleaning the BigScience ROOTS corpus
  ☆307Updated last year
• thomfoster / minRLHF
  A (somewhat) minimal library for finetuning language models with PPO on human feedback.
  ☆86Updated 2 years ago
• LAION-AI / Open-Instruction-Generalist
  Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks
  ☆207Updated last year
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆58Updated last year
• microsoft / DeepSpeed-Kernels
  ☆58Updated 8 months ago
• fanshiqing / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆86Updated 3 weeks ago
• princeton-nlp / CoFiPruning
  [ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408
  ☆192Updated last year
• huggingface / transformers_bloom_parallel
  Techniques used to run BLOOM at inference in parallel
  ☆37Updated 2 years ago
• HuangLK / transpeeder
  train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism
  ☆213Updated last year
• facebookresearch / LLM-QAT
  Code repo for the paper "LLM-QAT Data-Free Quantization Aware Training for Large Language Models"
  ☆266Updated 4 months ago
• sail-sg / zero-bubble-pipeline-parallelism
  Zero Bubble Pipeline Parallelism
  ☆317Updated 2 months ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆415Updated 3 weeks ago
• p-lambda / dsir
  DSIR large-scale data selection framework for language model training
  ☆242Updated 9 months ago
• ProjectD-AI / LLaMA-Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆69Updated last year
• stanford-futuredata / stk
  ☆97Updated 5 months ago
• jaymody / speculative-sampling
  Simple implementation of Speculative Sampling in NumPy for GPT-2.
  ☆90Updated last year