haonan3 / AnchorContext
AnchorAttention: Improved attention for LLMs long-context training
☆206Updated 2 months ago
Alternatives and similar repositories for AnchorContext:
Users that are interested in AnchorContext are comparing it to the libraries listed below
- L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning☆148Updated last week
- [NeurIPS 2024] Code for the paper "Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models"☆138Updated 3 weeks ago
- [ICLR2025] DiffuGPT and DiffuLLaMA: Scaling Diffusion Language Models via Adaptation from Autoregressive Models☆117Updated last week
- What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective☆63Updated 3 weeks ago
- ☆83Updated 2 weeks ago
- open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality☆179Updated 7 months ago
- Function Vectors in Large Language Models (ICLR 2024)☆151Updated last week
- ☆128Updated last week
- Repo of paper "Free Process Rewards without Process Labels"☆138Updated 2 weeks ago
- Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"☆123Updated 11 months ago
- [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"☆75Updated 9 months ago
- Auto get diffusion nlp papers in Axriv. More papers Information can be found in another repository "Diffusion-LM-Papers".☆103Updated this week
- Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"☆166Updated 3 weeks ago
- A Survey on Efficient Reasoning for LLMs☆116Updated this week
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆161Updated 2 months ago
- Code accompanying the paper "Massive Activations in Large Language Models"☆150Updated last year
- Auto Interpretation Pipeline and many other functionalities for Multimodal SAE Analysis.☆124Updated 2 months ago
- PASTA: Post-hoc Attention Steering for LLMs☆113Updated 4 months ago
- A brief and partial summary of RLHF algorithms.☆127Updated 3 weeks ago
- ☆171Updated last year
- ☆166Updated last month
- Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]☆130Updated 6 months ago
- ☆260Updated last week
- State-of-the-art Parameter-Efficient MoE Fine-tuning Method☆149Updated 7 months ago
- [ICLR 2025] When Attention Sink Emerges in Language Models: An Empirical View (Spotlight)☆54Updated 5 months ago
- TokenSkip: Controllable Chain-of-Thought Compression in LLMs☆98Updated 2 weeks ago
- Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision☆119Updated 6 months ago
- [ICLR'25] Data and code for our paper "Why Does the Effective Context Length of LLMs Fall Short?"☆70Updated 4 months ago
- ☆76Updated 2 months ago
- [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models☆207Updated 3 weeks ago