Alternatives and similar repositories for lora_from_scratch

Users that are interested in lora_from_scratch are comparing it to the libraries listed below

• rasbt / dora-from-scratch
  LoRA and DoRA from Scratch Implementations
  ☆215Updated last year
• lucidrains / llama-qrlhf
  Implementation of the Llama architecture with RLHF + Q-learning
  ☆170Updated last year
• SumanthRH / tokenization
  A comprehensive deep dive into the world of tokens
  ☆226Updated last year
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆85Updated 3 weeks ago
• geronimi73 / phi2-finetune
  ☆86Updated 2 years ago
• rasbt / pytorch-memory-optim
  This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…
  ☆92Updated 2 years ago
• idiap / sigma-gpt
  σ-GPT: A New Approach to Autoregressive Models
  ☆70Updated last year
• llm-efficiency-challenge / neurips_llm_efficiency_challenge
  NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day
  ☆260Updated 2 years ago
• cloneofsimo / min-fsdp
  ☆92Updated last year
• lucidrains / CoLT5-attention
  Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
  ☆231Updated last year
• lucidrains / grokfast-pytorch
  Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
  ☆103Updated last year
• lucidrains / CALM-pytorch
  Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind
  ☆179Updated last year
• warner-benjamin / commented-transformers
  Highly commented implementations of Transformers in PyTorch
  ☆138Updated 2 years ago
• normster / llm_rules
  RuLES: a benchmark for evaluating rule-following in language models
  ☆248Updated 11 months ago
• Pleias / Various-Finetuning
  Set of scripts to finetune LLMs
  ☆38Updated last year
• gautierdag / bpeasy
  Fast bare-bones BPE for modern tokenizer training
  ☆175Updated 7 months ago
• lucidrains / simple-hierarchical-transformer
  Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
  ☆224Updated last year
• mcleish7 / arithmetic
  Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)
  ☆198Updated last year
• leehanchung / lora-instruct
  Finetune Falcon, LLaMA, MPT, and RedPajama on consumer hardware using PEFT LoRA
  ☆104Updated 8 months ago
• cloneofsimo / min-max-gpt
  Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training
  ☆132Updated last year
• wolfecameron / lora_instruction_tune
  ☆37Updated last year
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆186Updated 2 weeks ago
• NousResearch / StripedHyenaTrainer
  ☆62Updated 2 years ago
• abacaj / train-with-fsdp
  ☆94Updated 2 years ago
• jxmorris12 / cde
  code for training & evaluating Contextual Document Embedding models
  ☆202Updated 8 months ago
• catid / dora
  Implementation of DoRA
  ☆306Updated last year
• isamu-isozaki / huggingface-reading-group
  This repository's goal is to precompile all past presentations of the Huggingface reading group
  ☆48Updated last year
• srush / do-we-need-attention
  ☆167Updated 2 years ago
• center-for-humans-and-machines / transformer-heads
  Toolkit for attaching, training, saving and loading of new heads for transformer models
  ☆294Updated 11 months ago
• lucidrains / pytorch-custom-utils
  Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…
  ☆126Updated last year