SakanaAI / DroPELinks
Extending the Context of Pretrained LLMs by Dropping Their Positional Embedding
☆130Updated this week
Alternatives and similar repositories for DroPE
Users that are interested in DroPE are comparing it to the libraries listed below
Sorting:
- This repo contains the source code for the paper "Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning"☆283Updated last month
- https://x.com/BlinkDL_AI/status/1884768989743882276☆28Updated 8 months ago
- https://hf.co/hexgrad/Kokoro-82M☆14Updated 10 months ago
- Official implementation of GRAPE: Group Representational Position Encoding (https://arxiv.org/abs/2512.07805)☆71Updated last week
- Lightweight package that tracks and summarizes code changes using LLMs (Large Language Models)☆34Updated 10 months ago
- RWKV-7: Surpassing GPT☆103Updated last year
- The code repository of the paper: Competition and Attraction Improve Model Fusion☆168Updated 4 months ago
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆109Updated 10 months ago
- EvaByte: Efficient Byte-level Language Models at Scale☆113Updated 8 months ago
- H-Net Dynamic Hierarchical Architecture☆80Updated 4 months ago
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆106Updated 7 months ago
- NanoGPT-speedrunning for the poor T4 enjoyers☆73Updated 8 months ago
- ☆62Updated this week
- Training teachers with reinforcement learning able to make LLMs learn how to reason for test time scaling.☆357Updated 6 months ago
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆132Updated 2 months ago
- Official JAX implementation of End-to-End Test-Time Training for Long Context☆297Updated 2 weeks ago
- ☆65Updated 9 months ago
- An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO☆29Updated this week
- Train, tune, and infer Bamba model☆137Updated 7 months ago
- Universal Reasoning Model☆113Updated 2 weeks ago
- The official github repo for "Diffusion Language Models are Super Data Learners".☆215Updated 2 months ago
- Official repository for the paper "NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks". This rep…☆60Updated last year
- The Automated LLM Speedrunning Benchmark measures how well LLM agents can reproduce previous innovations and discover new ones in languag…☆125Updated 3 months ago
- Esoteric Language Models☆107Updated last month
- ☆19Updated 10 months ago
- working implimention of deepseek MLA☆45Updated last year
- Official PyTorch implementation for Hogwild! Inference: Parallel LLM Generation with a Concurrent Attention Cache☆139Updated 5 months ago
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆56Updated 10 months ago
- 📄Small Batch Size Training for Language Models☆79Updated 3 months ago
- ☆97Updated 3 weeks ago