Alternatives and similar repositories for pretraining-BERT:

Users that are interested in pretraining-BERT are comparing it to the libraries listed below

• KhoomeiK / complexity-scaling
  gzip Predicts Data-dependent Scaling Laws
  ☆34Updated 10 months ago
• abacaj / train-with-fsdp
  ☆92Updated last year
• AblateIt / finetune-study
  Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.
  ☆82Updated last year
• LiibanMo / scikit-jax
  Your favourite classical machine learning algos on the GPU/TPU
  ☆20Updated 3 months ago
• NousResearch / StripedHyenaTrainer
  ☆61Updated last year
• CarperAI / treasure_trove
  ☆22Updated last year
• euclaise / supertrainer2000
  ☆49Updated last year
• drisspg / transformer_nuggets
  A place to store reusable transformer components of my own creation or found on the interwebs
  ☆49Updated last week
• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆30Updated 7 months ago
• luyug / magix
  Supercharge huggingface transformers with model parallelism.
  ☆76Updated 6 months ago
• hundredblocks / large-model-parallelism
  Functional local implementations of main model parallelism approaches
  ☆95Updated 2 years ago
• jfpuget / ARC-AGI-Challenge-2024
  ☆52Updated 5 months ago
• OptimalFoundation / nadir
  Nadir: Cutting-edge PyTorch optimizers for simplicity & composability! 🔥🚀💻
  ☆14Updated 10 months ago
• official-elinas / zeus-llm-trainer
  Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models
  ☆69Updated last year
• huu4ontocord / MDEL
  Multi-Domain Expert Learning
  ☆67Updated last year
• Pleias / Quest-Best-Tokens
  An introduction to LLM Sampling
  ☆77Updated 4 months ago
• krypticmouse / matryoshka-representation-learning
  PyTorch implementation for MRL
  ☆18Updated last year
• ChrisHayduk / QLoRA-for-MLM
  QLoRA for Masked Language Modeling
  ☆22Updated last year
• raphaelsty / LeNLP
  NLP with Rust for Python 🦀🐍
  ☆62Updated 10 months ago
• Muhtasham / summarization-eval
  📝 Reference-Free automatic summarization evaluation with potential hallucination detection
  ☆100Updated last year
• rasbt / nn_plus_gzip
  Gzip and nearest neighbors for text classification
  ☆56Updated last year
• eth-easl / fmengine
  Utilities for Training Very Large Models
  ☆58Updated 7 months ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆47Updated last year
• msaroufim / mynotes
  ☆17Updated 2 months ago
• Upaya07 / NeurIPS-llm-efficiency-challenge
  Code for NeurIPS LLM Efficiency Challenge
  ☆57Updated last year
• kyleliang919 / Long-context-transformers
  Exploring finetuning public checkpoints on filter 8K sequences on Pile
  ☆115Updated 2 years ago
• epfml / DenseFormer
  ☆79Updated last year
• davisyoshida / easy-lora-and-gptq
  JAX notebook showing how to LoRA + GPTQ arbitrary models
  ☆9Updated last year
• unicamp-dl / InRanker
  ☆47Updated last year
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated last year