"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang.
☆31May 7, 2024Updated last year
Alternatives and similar repositories for Ms-PoE
Users that are interested in Ms-PoE are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆16Jun 25, 2025Updated 9 months ago
- Code for paper: Long cOntext aliGnment via efficient preference Optimization☆24Oct 10, 2025Updated 5 months ago
- To mitigate position bias in LLMs, especially in long-context scenarios, we scale only one dimension of LLMs, reducing position bias and …☆11Jun 18, 2024Updated last year
- Official code for the paper "Attention as a Hypernetwork"☆55Feb 24, 2026Updated last month
- ☆25Dec 12, 2025Updated 3 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Code and data for "Lost in the Middle: How Language Models Use Long Contexts"☆374Jan 4, 2024Updated 2 years ago
- Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"☆247Sep 12, 2025Updated 6 months ago
- open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality☆236Aug 2, 2024Updated last year
- Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)☆62Feb 13, 2024Updated 2 years ago
- ☆20Jan 16, 2024Updated 2 years ago
- The HELMET Benchmark☆204Feb 26, 2026Updated 3 weeks ago
- ☆46Mar 15, 2025Updated last year
- Comparing retrieval abilities from GPT4-Turbo and a RAG system on a toy example for various context lengths☆35Dec 1, 2023Updated 2 years ago
- The official implementation of "LightTransfer: Your Long-Context LLM is Secretly a Hybrid Model with Effortless Adaptation"☆22Apr 22, 2025Updated 11 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- 한국어 자연어 처리 모델 미세조정☆17Jan 26, 2021Updated 5 years ago
- PyTorch implementation of "Distinguishing Homophenes using Multi-Head Visual-Audio Memory" (AAAI2022)☆27Mar 9, 2024Updated 2 years ago
- The repository for papaer "Distance between Relevant Information Pieces Causes Bias in Long-Context LLMs"☆14Dec 16, 2024Updated last year
- LongMIT: Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets☆41Sep 30, 2024Updated last year
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆242Jun 15, 2025Updated 9 months ago
- [IJCV 2023] FlowNAS: Neural Architecture Search for Optical Flow Estimation☆16Feb 21, 2024Updated 2 years ago
- ☆13May 9, 2024Updated last year
- The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Mem…☆396Apr 20, 2024Updated last year
- Simplified packaging for pybind11-based C++ extensions☆13Jun 3, 2022Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- 16-fold memory access reduction with nearly no loss☆108Mar 26, 2025Updated 11 months ago
- Linux /proc data in a consistent, parsed format.☆10Mar 28, 2016Updated 9 years ago
- ☆46Apr 13, 2022Updated 3 years ago
- [ICML‘25] Official code for paper "Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training an…☆13Apr 17, 2025Updated 11 months ago
- Crawl & Visualize NeurIPS 2022 Data from OpenReview☆14Nov 8, 2022Updated 3 years ago
- BABILong is a benchmark for LLM evaluation using the needle-in-a-haystack approach.☆241Sep 2, 2025Updated 6 months ago
- Official pytorch code for "APP: Anytime Progressive Pruning" (DyNN @ ICML, 2022; CLL @ ACML, 2022, SNN @ ICML, 2022 and SlowDNN 2023)☆16Nov 22, 2022Updated 3 years ago
- LIVW-Localization: A Multi-modal Information Fused Vehicle Localization method for Complex, Large-Scale and GNSS-Denied Environments.☆14Jan 19, 2026Updated 2 months ago
- 该项目是基于Python和数据库实现的学生信息管理系统☆13Jul 21, 2021Updated 4 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Official implementation for Text Generation Beyond Discrete Token Sampling☆24Aug 11, 2025Updated 7 months ago
- GAM-Depth: Self-Supervised Indoor Depth Estimation Leveraging a Gradient-Aware Mask and Semantic Constraints☆12Apr 15, 2024Updated last year
- Code implementation for paper "On the Efficacy of Small Self-Supervised Contrastive Models without Distillation Signals".☆17Dec 15, 2021Updated 4 years ago
- Cascade Speculative Drafting☆33Apr 2, 2024Updated last year
- 类C编译器，编译原理课程设计☆11Jul 30, 2021Updated 4 years ago
- ☆15Jan 9, 2026Updated 2 months ago
- [NeurIPS '25] Multi-Token Prediction Needs Registers☆28Dec 14, 2025Updated 3 months ago