"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.
☆31May 7, 2024Updated last year
Alternatives and similar repositories for Ms-PoE
Users that are interested in Ms-PoE are comparing it to the libraries listed below
Sorting:
- ☆16Jun 25, 2025Updated 8 months ago
- Google 공식 Rouge Implementation을 한국어에서 사용할 수 있도록 처리☆18Jan 3, 2024Updated 2 years ago
- Official code for the paper "Attention as a Hypernetwork"☆51Feb 24, 2026Updated last week
- Code for paper: Long cOntext aliGnment via efficient preference Optimization☆24Oct 10, 2025Updated 4 months ago
- ☆25Dec 12, 2025Updated 2 months ago
- 한국어 자연어 처리 모델 미세조정☆17Jan 26, 2021Updated 5 years ago
- ☆20Jan 16, 2024Updated 2 years ago
- KV cache compression for high-throughput LLM inference☆154Feb 5, 2025Updated last year
- Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"☆247Sep 12, 2025Updated 5 months ago
- open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality☆232Aug 2, 2024Updated last year
- 16-fold memory access reduction with nearly no loss☆108Mar 26, 2025Updated 11 months ago
- [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens☆25Nov 6, 2023Updated 2 years ago
- [ICCV 2025] Deeply Supervised Flow-Based Generative Models☆28Jun 26, 2025Updated 8 months ago
- The HELMET Benchmark☆203Updated this week
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆237Jun 15, 2025Updated 8 months ago
- ☆50Jun 16, 2025Updated 8 months ago
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Apr 17, 2024Updated last year
- ☆35Apr 8, 2025Updated 10 months ago
- Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)☆62Feb 13, 2024Updated 2 years ago
- Code and data for "Lost in the Middle: How Language Models Use Long Contexts"☆373Jan 4, 2024Updated 2 years ago
- ☆43Mar 15, 2025Updated 11 months ago
- [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning☆140Feb 25, 2026Updated last week
- LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation☆33Feb 26, 2026Updated last week
- Cascade Speculative Drafting☆33Apr 2, 2024Updated last year
- Comparing retrieval abilities from GPT4-Turbo and a RAG system on a toy example for various context lengths☆35Dec 1, 2023Updated 2 years ago
- Official Pytorch Implementation of Self-emerging Token Labeling☆35Mar 27, 2024Updated last year
- 한국어 언어 모델 학습을 위한 프로젝트(Flax, Pytorch with Huggingface Accelerate)☆32Sep 13, 2023Updated 2 years ago
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆224Dec 16, 2025Updated 2 months ago
- Repository for the Q-Filters method (https://arxiv.org/pdf/2503.02812)☆35Mar 7, 2025Updated 11 months ago
- Official implementation for 'Extending LLMs’ Context Window with 100 Samples'☆81Jan 18, 2024Updated 2 years ago
- Implementation of paper Data Engineering for Scaling Language Models to 128K Context☆486Mar 19, 2024Updated last year
- Crawl & Visualize NeurIPS 2022 Data from OpenReview☆14Nov 8, 2022Updated 3 years ago
- 기획자와 마케터를 위한 이벤트 댓글 분석 - feat. 인프런 새해 다짐 이벤트☆11Apr 22, 2020Updated 5 years ago
- Visualizing 230 years of US Census data☆12Feb 23, 2020Updated 6 years ago
- 用Paddle复现论文ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information（ACL2021）☆10Nov 15, 2021Updated 4 years ago
- ☆13Jun 17, 2025Updated 8 months ago
- ☆134May 29, 2025Updated 9 months ago
- Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"☆110Oct 11, 2025Updated 4 months ago
- ☆44Jan 21, 2025Updated last year