dangxingyu / rnn-icrag
Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
☆24Updated 7 months ago
Related projects ⓘ
Alternatives and complementary repositories for rnn-icrag
- ☆24Updated 8 months ago
- ☆18Updated 5 months ago
- ☆45Updated 9 months ago
- ☆45Updated 4 months ago
- [ACL 2024] Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning☆30Updated 3 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆61Updated 7 months ago
- Long Context Extension and Generalization in LLMs☆39Updated 2 months ago
- ☆53Updated 3 weeks ago
- Official implementation of Bootstrapping Language Models via DPO Implicit Rewards☆39Updated 3 months ago
- Repository for Skill Set Optimization☆12Updated 3 months ago
- ☆69Updated 8 months ago
- [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623☆69Updated last month
- Stick-breaking attention☆34Updated last week
- ICML 2024 - Official Repository for EXO: Towards Efficient Exact Optimization of Language Model Alignment☆46Updated 5 months ago
- ☆21Updated last month
- HGRN2: Gated Linear RNNs with State Expansion☆49Updated 3 months ago
- Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"☆44Updated 10 months ago
- ☆15Updated 4 months ago
- Code for "Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective"☆30Updated 6 months ago
- ☆36Updated 3 months ago
- The official implementation of Self-Exploring Language Models (SELM)☆55Updated 5 months ago
- Self-Supervised Alignment with Mutual Information☆14Updated 5 months ago
- [ICML 24 NGSM workshop] Associative Recurrent Memory Transformer implementation and scripts for training and evaluating☆31Updated this week
- Advantage Leftover Lunch Reinforcement Learning (A-LoL RL): Improving Language Models with Advantage-based Offline Policy Gradients☆26Updated 2 months ago
- ☆24Updated 7 months ago
- Directional Preference Alignment☆50Updated last month
- ☆54Updated last month
- Using FlexAttention to compute attention with different masking patterns☆40Updated last month
- Conic10K: A large-scale dataset for closed-vocabulary math problem understanding. Accepted to EMNLP2023 Findings.☆23Updated 11 months ago
- Repository for NPHardEval, a quantified-dynamic benchmark of LLMs☆48Updated 7 months ago