Multi-Layer Key-Value sharing experiments on Pythia models
☆35Jun 14, 2024Updated 2 years ago
Alternatives and similar repositories for pythia-mlkv
Users that are interested in pythia-mlkv are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆16Jul 23, 2024Updated 2 years ago
- ☆47Nov 25, 2024Updated last year
- ☆24Mar 7, 2025Updated last year
- Token Omission Via Attention☆130Oct 13, 2024Updated 2 years ago
- Dynamic Attention Mask (DAM) generate adaptive sparse attention masks per layer and head for Transformer models, enabling long-context in…☆14Jun 16, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Source code for SWIFT, an efficient reward model.☆21Jan 13, 2026Updated 9 months ago
- The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"☆41Oct 11, 2024Updated 2 years ago
- Code for paper: Optimizing Length Compression in Large Reasoning Models☆29Oct 20, 2025Updated last year
- Benchmark tests supporting the TiledCUDA library.☆19Nov 19, 2024Updated last year
- Crosslingual Reasoning through Test-Time Scaling☆22May 13, 2025Updated last year
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆55Oct 18, 2024Updated 2 years ago
- Testing paligemma2 finetuning on reasoning dataset☆18Dec 28, 2024Updated last year
- ☆12Aug 4, 2026Updated 3 months ago
- This is the oficial repository for "Safer-Instruct: Aligning Language Models with Automated Preference Data"☆17Feb 22, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Keyformer proposes KV Cache reduction through key tokens identification and without the need for fine-tuning☆59Mar 26, 2024Updated 2 years ago
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆156Apr 7, 2025Updated last year
- ☆31Jul 31, 2026Updated 3 months ago
- Physics-informed learning of governing equations from scarce data☆15Feb 16, 2021Updated 5 years ago
- CHERRL: A Controllable Hacking Environment for Rubric-Based Reinforcement Learning☆16Oct 15, 2026Updated 3 weeks ago
- [COLM 2024] SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models☆25Oct 5, 2024Updated 2 years ago
- DataRubrics, a structured framework for assessing the quality of both human- and model-generated datasets. Leveraging recent advances in …☆16Jun 6, 2025Updated last year
- ☆20Mar 25, 2025Updated last year
- ☆15Apr 11, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning☆155Feb 25, 2026Updated 8 months ago
- Peer-to-Peer Search Engine System☆13May 31, 2024Updated 2 years ago
- IPFS private network☆10Jul 10, 2023Updated 3 years ago
- Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging. Arxiv, 2024.☆16Oct 28, 2024Updated 2 years ago
- Dateset Reset Policy Optimization☆30Apr 12, 2024Updated 2 years ago
- Command helper for slurm system. Act as if you are on compute node.☆16Feb 1, 2025Updated last year
- A base protocol providing streams of messages☆15Aug 22, 2025Updated last year
- Safe, idiomatic, and performant Rust wrappers for FFmpeg, FFprobe, and FFplay.☆12Apr 1, 2026Updated 7 months ago
- Code and data to explore neural scaling laws of xLSTM and Transformer models.☆24Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- LLM-Merging: Building LLMs Efficiently through Merging☆208Sep 24, 2024Updated 2 years ago
- xKV: Cross-Layer SVD for KV-Cache Compression [ICML 2026]☆60Jul 7, 2026Updated 4 months ago
- ☆11Feb 1, 2026Updated 9 months ago
- ☆11Jan 13, 2026Updated 9 months ago
- FocusLLM: Scaling LLM’s Context by Parallel Decoding☆45Dec 8, 2024Updated last year
- RISC-V to x86/64 instruction emulator written in C/C++☆11Jan 10, 2022Updated 4 years ago
- ☆12Sep 1, 2023Updated 3 years ago
Are these results useful?