ant-research / long-context-modelingLinks
Research work aimed at addressing the problem of modeling infinite-length context
☆46Updated last month
Alternatives and similar repositories for long-context-modeling
Users that are interested in long-context-modeling are comparing it to the libraries listed below
Sorting:
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆42Updated last month
- ☆19Updated last year
- Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"☆109Updated 4 months ago
- The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":☆44Updated last year
- Code for the preprint "Cache Me If You Can: How Many KVs Do You Need for Effective Long-Context LMs?"☆47Updated 6 months ago
- ☆120Updated last week
- Codebase for Instruction Following without Instruction Tuning☆36Updated last year
- [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection☆55Updated last year
- Long Context Extension and Generalization in LLMs☆62Updated last year
- [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623☆89Updated last year
- ☆129Updated 8 months ago
- Klear-Reasoner: Advancing Reasoning Capability via Gradient-Preserving Clipping Policy Optimization☆81Updated last month
- [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models☆60Updated last year
- ☆63Updated 8 months ago
- [ICML 2025] Teaching Language Models to Critique via Reinforcement Learning☆120Updated 9 months ago
- ☆84Updated 3 months ago
- ☆66Updated 7 months ago
- Kinetics: Rethinking Test-Time Scaling Laws☆86Updated 7 months ago
- LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification☆73Updated 6 months ago
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆52Updated last year
- instruction-following benchmark for large reasoning models☆44Updated 6 months ago
- [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens☆25Updated 2 years ago
- Code for the EMNLP24 paper "A simple and effective L2 norm based method for KV Cache compression."☆18Updated last year
- ☆135Updated 2 weeks ago
- ☆72Updated 8 months ago
- LongAttn ：Selecting Long-context Training Data via Token-level Attention☆15Updated 6 months ago
- The official repo for "AceCoder: Acing Coder RL via Automated Test-Case Synthesis" [ACL25]☆96Updated 10 months ago
- SIFT: Grounding LLM Reasoning in Contexts via Stickers☆57Updated 11 months ago
- LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models☆79Updated last year
- [ICLR2025] Codebase for "ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing", built on Megatron-LM.☆105Updated last year