Jellyfish042 / uncheatable_eval
Evaluating LLMs with Dynamic Data
☆78Updated last month
Alternatives and similar repositories for uncheatable_eval:
Users that are interested in uncheatable_eval are comparing it to the libraries listed below
- Official implementation for 'Extending LLMs’ Context Window with 100 Samples'☆75Updated last year
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆121Updated 2 months ago
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆140Updated 6 months ago
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆206Updated 10 months ago
- Experiments on speculative sampling with Llama models☆125Updated last year
- ☆110Updated last week
- A repository for research on medium sized language models.☆76Updated 10 months ago
- This is the official repository for Inheritune.☆109Updated last month
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆117Updated last year
- RWKV-7: Surpassing GPT☆82Updated 4 months ago
- ☆31Updated 9 months ago
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆53Updated last month
- ☆87Updated 5 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆62Updated last year
- Official github repo for the paper "Compression Represents Intelligence Linearly" [COLM 2024]☆130Updated 6 months ago
- ☆136Updated 4 months ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Updated 7 months ago
- Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…☆148Updated 2 months ago
- Multipack distributed sampler for fast padding-free training of LLMs☆186Updated 7 months ago
- ☆17Updated 10 months ago
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆87Updated this week
- Reformatted Alignment☆115Updated 6 months ago
- My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated☆31Updated 7 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆71Updated 11 months ago
- GoldFinch and other hybrid transformer components☆45Updated 8 months ago
- Longitudinal Evaluation of LLMs via Data Compression☆32Updated 9 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"☆59Updated 5 months ago
- ☆60Updated 10 months ago
- QuIP quantization☆52Updated last year