Alternatives and similar repositories for LoRI:

Users that are interested in LoRI are comparing it to the libraries listed below

• wang-kee / LiNeS
  Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"
  ☆25Updated 5 months ago
• RobertCsordas / moeut
  ☆77Updated 8 months ago
• sail-sg / SkyLadder
  The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling
  ☆29Updated last month
• menhguin / minp_paper
  Code Implementation, Evaluations, Documentation, Links and Resources for Min P paper
  ☆30Updated last month
• hamishivi / tess-2
  Repository for "TESS-2: A Large-Scale, Generalist Diffusion Language Model"
  ☆33Updated 2 months ago
• frankxwang / dpo-prefix-sharing
  DPO, but faster 🚀
  ☆40Updated 4 months ago
• Infini-AI-Lab / S2FT
  ☆17Updated 3 months ago
• NJUNLP / MoE-LPR
  ☆12Updated 4 months ago
• fal-ai-community / nano-mdm
  Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
  ☆47Updated last month
• IST-DASLab / RoSA
  Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)
  ☆40Updated last year
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆44Updated 9 months ago
• OpenNLPLab / HGRN
  [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…
  ☆64Updated 11 months ago
• graphcore-research / jax-scalify
  JAX Scalify: end-to-end scaled arithmetics
  ☆16Updated 5 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆50Updated 2 years ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore
  ☆26Updated 7 months ago
• HazyResearch / prefix-linear-attention
  ☆53Updated 9 months ago
• Helw150 / levanter
  Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax
  ☆14Updated 10 months ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆34Updated 6 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆60Updated 11 months ago
• NathanGodey / qfilters
  Repository for the Q-Filters method (https://arxiv.org/pdf/2503.02812)
  ☆28Updated last month
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆54Updated 8 months ago
• lucidrains / transformer-lm-gan
  Explorations into adversarial losses on top of autoregressive loss for language modeling
  ☆35Updated last month
• ShiZhengyan / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆36Updated 10 months ago
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆47Updated 7 months ago
• liziniu / cold_start_rl
  Code for Blog Post: Can Better Cold-Start Strategies Improve RL Training for LLMs?
  ☆16Updated last month
• lucidrains / light-recurrent-unit-pytorch
  Implementation of a Light Recurrent Unit in Pytorch
  ☆47Updated 6 months ago
• princeton-pli / MeCo
  Code for preprint "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆36Updated last month
• lucidrains / self-reasoning-tokens-pytorch
  Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto
  ☆55Updated 11 months ago
• RobertCsordas / moe
  Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"
  ☆37Updated last year
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆30Updated 10 months ago