HanseulJo / position-couplingLinks
Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure (NeurIPS 2024) + Arithmetic Transformers Can Length-Generalize in Both Operand Length and Count (ICLR 2025)
☆11Updated 5 months ago
Alternatives and similar repositories for position-coupling
Users that are interested in position-coupling are comparing it to the libraries listed below
Sorting:
- ☆20Updated last year
- Latest Weight Averaging (NeurIPS HITY 2022)☆31Updated 2 years ago
- Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)☆81Updated 2 years ago
- Recycling diverse models☆45Updated 2 years ago
- ☆31Updated last year
- ☆19Updated 6 months ago
- [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…☆16Updated 5 months ago
- Official repository for our paper, Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Mode…☆18Updated 10 months ago
- SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)☆34Updated 10 months ago
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆83Updated 10 months ago
- Lightweight Adapting for Black-Box Large Language Models☆23Updated last year
- ☆51Updated last year
- A modern look at the relationship between sharpness and generalization [ICML 2023]☆43Updated 2 years ago
- ☆70Updated 9 months ago
- Bayesian Low-Rank Adaptation for Large Language Models☆36Updated last year
- Test-time-training on nearest neighbors for large language models☆46Updated last year
- Unofficial Implementation of Selective Attention Transformer☆17Updated 10 months ago
- ☆13Updated 2 months ago
- Source code of "Task arithmetic in the tangent space: Improved editing of pre-trained models".☆104Updated 2 years ago
- [ICML‘2024] "LoCoCo: Dropping In Convolutions for Long Context Compression", Ruisi Cai, Yuandong Tian, Zhangyang Wang, Beidi Chen☆17Updated last year
- ☆32Updated last year
- ☆28Updated 6 months ago
- A simple Jax implementation of influence functions.☆18Updated last year
- ☆29Updated 7 months ago
- This repository includes code to reproduce the tables in "Loss Landscapes are All You Need: Neural Network Generalization Can Be Explaine…☆38Updated 2 years ago
- ☆30Updated last year
- ☆74Updated 3 years ago
- [ICML2024 Spotlight] Fine-Tuning Pre-trained Large Language Models Sparsely☆23Updated last year
- Applies ROME and MEMIT on Mamba-S4 models☆14Updated last year
- ☆100Updated last year