Alternatives and similar repositories for lora_from_scratch:

Users that are interested in lora_from_scratch are comparing it to the libraries listed below

• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆85Updated 2 months ago
• euclaise / supertrainer2000
  ☆49Updated last year
• cloneofsimo / min-fsdp
  ☆78Updated 9 months ago
• rasbt / pytorch-memory-optim
  This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…
  ☆91Updated last year
• abacaj / train-with-fsdp
  ☆92Updated last year
• kyleliang919 / Long-context-transformers
  Exploring finetuning public checkpoints on filter 8K sequences on Pile
  ☆115Updated 2 years ago
• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆202Updated last year
• Pleias / Quest-Best-Tokens
  An introduction to LLM Sampling
  ☆77Updated 4 months ago
• lucidrains / llama-qrlhf
  Implementation of the Llama architecture with RLHF + Q-learning
  ☆164Updated 2 months ago
• gautierdag / bpeasy
  Fast bare-bones BPE for modern tokenizer training
  ☆154Updated 3 weeks ago
• warner-benjamin / commented-transformers
  Highly commented implementations of Transformers in PyTorch
  ☆136Updated last year
• Pleias / Various-Finetuning
  Set of scripts to finetune LLMs
  ☆37Updated last year
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆198Updated 9 months ago
• epfml / DenseFormer
  ☆79Updated last year
• idiap / sigma-gpt
  σ-GPT: A New Approach to Autoregressive Models
  ☆62Updated 8 months ago
• HazyResearch / zoology
  Understand and test language model architectures on synthetic tasks.
  ☆192Updated last month
• tensoic / Cerule
  Cerule - A Tiny Mighty Vision Model
  ☆67Updated 7 months ago
• lucidrains / grokfast-pytorch
  Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
  ☆98Updated 4 months ago
• imoneoi / multipack_sampler
  Multipack distributed sampler for fast padding-free training of LLMs
  ☆188Updated 8 months ago
• keeeeenw / MicroLlama
  Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget
  ☆146Updated last year
• catie-aq / flashT5
  A fast implementation of T5/UL2 in PyTorch using Flash Attention
  ☆103Updated last month
• dvruette / barrel-rec-pytorch
  ☆53Updated last year
• melisa-writer / short-transformers
  Prune transformer layers
  ☆68Updated 10 months ago
• akjindal53244 / Arithmo
  Small and Efficient Mathematical Reasoning LLMs
  ☆71Updated last year
• ChrisHayduk / qlora-multi-gpu
  QLoRA with Enhanced Multi GPU Support
  ☆37Updated last year
• SumanthRH / tokenization
  A comprehensive deep dive into the world of tokens
  ☆222Updated 10 months ago
• EleutherAI / concept-erasure
  Erasing concepts from neural representations with provable guarantees
  ☆227Updated 3 months ago
• fangyuan-ksgk / Tiny-GRPO
  minimal GRPO implementation from scratch
  ☆85Updated last month
• fairydreaming / farel-bench
  Testing LLM reasoning abilities with family relationship quizzes.
  ☆62Updated 2 months ago
• KaiNylund / lm-weights-encode-time
  ☆67Updated 8 months ago