IBM / activated-loraLinks
Source code for Activated LoRA
☆20Updated 3 weeks ago
Alternatives and similar repositories for activated-lora
Users that are interested in activated-lora are comparing it to the libraries listed below
Sorting:
- Official implementation of "BERTs are Generative In-Context Learners"☆32Updated 5 months ago
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆82Updated 10 months ago
- Code for the paper "Function-Space Learning Rates"☆23Updated 3 months ago
- FMS Model Optimizer is a framework for developing reduced precision neural network models.☆20Updated 3 weeks ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆61Updated 10 months ago
- A repository for research on medium sized language models.☆78Updated last year
- train with kittens!☆62Updated 10 months ago
- An extention to the GaLore paper, to perform Natural Gradient Descent in low rank subspace☆17Updated 10 months ago
- The evaluation framework for training-free sparse attention in LLMs☆91Updated 2 months ago
- Official Implementation of "DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucination"☆24Updated 8 months ago
- ☆48Updated last year
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆33Updated last week
- JAX Scalify: end-to-end scaled arithmetics☆16Updated 10 months ago
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆36Updated last month
- Synthetic data generation and benchmark implementation for "Episodic Memories Generation and Evaluation Benchmark for Large Language Mode…☆51Updated 4 months ago
- ☆85Updated last year
- ☆49Updated 10 months ago
- Collection of autoregressive model implementation☆86Updated 4 months ago
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆85Updated last month
- [NeurIPS 2024] Goldfish Loss: Mitigating Memorization in Generative LLMs☆90Updated 9 months ago
- Using FlexAttention to compute attention with different masking patterns☆44Updated 11 months ago
- Official repository for "BLEUBERI: BLEU is a surprisingly effective reward for instruction following"☆25Updated 2 months ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated last year
- We study toy models of skill learning.☆30Updated 7 months ago
- ☆14Updated last year
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆47Updated 4 months ago
- Train, tune, and infer Bamba model☆131Updated 2 months ago
- Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…☆88Updated this week
- Unit Scaling demo and experimentation code☆16Updated last year