Official code for the NeurIPS25 paper "RAT: Bridging RNN Efficiencyand Attention Accuracy in Language Modeling" (https://arxiv.org/abs/2507.04416))
☆26Dec 10, 2025Updated 8 months ago
Alternatives and similar repositories for RAT
Users that are interested in RAT are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Official codebase for "Quantile Reward Policy Optimization: Alignment with Pointwise Regression and Exact Partition Functions" (Matrenok …☆30Dec 8, 2025Updated 8 months ago
- The official code of "Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers"☆20Jul 24, 2024Updated 2 years ago
- ☆18Aug 18, 2026Updated last week
- [ICLR 2025] SDTT: a simple and effective distillation method for discrete diffusion models☆53Feb 26, 2026Updated 6 months ago
- mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models☆11Jan 19, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- FlexiTokens☆23Dec 27, 2025Updated 8 months ago
- Label shift estimation for transfer difficulty with Familiarity.☆10Feb 4, 2025Updated last year
- A method for evaluating the high-level coherence of machine-generated texts. Identifies high-level coherence issues in transformer-based …☆12Mar 18, 2023Updated 3 years ago
- Official repository for "SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space"☆29May 7, 2026Updated 3 months ago
- ☆16May 14, 2024Updated 2 years ago
- Implementation of Cascaded Head-colliding Attention (ACL'2021)☆11Sep 16, 2021Updated 4 years ago
- ☆16Jul 29, 2025Updated last year
- ☆21Dec 5, 2022Updated 3 years ago
- [ICML 2026] Scaling Beyond Masked Diffusion Language Models☆36Jul 3, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- VibeRL is a Reinforcement Learning framework built essentially through vibe coding with Kimi K2.☆18Updated this week
- Deep Data Research. Seek More, See Beyond.☆16Feb 6, 2026Updated 6 months ago
- Implementation and experiments for Partially Supervised NER via Expected Entity Ratio in TACL 2022☆14Nov 7, 2022Updated 3 years ago
- Linear Attention for Efficient Bidirectional Sequence Modeling☆18May 13, 2025Updated last year
- Official Implementation of "Variational Flow Maps: Make Some Noise for One-Step Conditional Generation"☆28Aug 7, 2026Updated 3 weeks ago
- ☆28Feb 26, 2026Updated 6 months ago
- ☆16Mar 22, 2023Updated 3 years ago
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆43Dec 29, 2025Updated 8 months ago
- HGRN2: Gated Linear RNNs with State Expansion☆59Aug 20, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Efficiently discovering algorithms via LLMs with evolutionary search and reinforcement learning.☆142Nov 18, 2025Updated 9 months ago
- [ICLR 2023] Official implementation of Transnormer in our ICLR 2023 paper - Toeplitz Neural Network for Sequence Modeling☆80Apr 24, 2024Updated 2 years ago
- continous batching and parallel acceleration for RWKV6☆23Jun 28, 2024Updated 2 years ago
- Implementation of the paper "Fine-Tuning Transformers: Vocabulary Transfer" https://arxiv.org/pdf/2112.14569.pdf☆20Dec 28, 2021Updated 4 years ago
- ☆20Apr 26, 2026Updated 4 months ago
- ☆30Mar 22, 2024Updated 2 years ago
- ☆29Jul 9, 2024Updated 2 years ago
- ☆18Oct 27, 2024Updated last year
- Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning☆30Jan 25, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Experimental playground for benchmarking language model (LM) architectures, layers, and tricks on smaller datasets. Designed for flexible…☆115Jul 17, 2026Updated last month
- German Language Understanding Evaluation Benchmark @NAACL24☆23Dec 11, 2025Updated 8 months ago
- AdaSplash: Adaptive Sparse Flash Attention (aka Flash Entmax Attention)☆47May 20, 2026Updated 3 months ago
- Code for the paper Task Agnostic Morphology Evolution.☆21May 25, 2021Updated 5 years ago
- Easily deploy my zsh and tmux configuration on new machines. Includes local and remote aliases to improve workflow.☆16Apr 23, 2026Updated 4 months ago
- (ICML 2026) IDLM: Inverse-distilled Diffusion Language Models☆16Jun 8, 2026Updated 2 months ago
- UNLP 2025 Shared Task on Detecting Social Media Manipulation☆23Aug 4, 2025Updated last year