liyucheng09 / llm-compressive
Longitudinal Evaluation of LLMs via Data Compression
☆25Updated 3 months ago
Related projects: ⓘ
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting☆60Updated 6 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆72Updated 6 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆133Updated 3 months ago
- code for Scaling Laws of RoPE-based Extrapolation☆68Updated 11 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆59Updated 9 months ago
- Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales☆30Updated last year
- ☆82Updated 5 months ago
- Official PyTorch implementation of IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact☆25Updated 3 months ago
- Low-bit optimizers for PyTorch☆109Updated 11 months ago
- Implementation of Speculative Sampling as described in "Accelerating Large Language Model Decoding with Speculative Sampling" by Deepmind☆69Updated 6 months ago
- Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"☆115Updated 2 weeks ago
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆101Updated 2 weeks ago
- ☆87Updated 4 months ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆37Updated 6 months ago
- Multi-Candidate Speculative Decoding☆27Updated 4 months ago
- ☆32Updated 3 months ago
- Code for paper "Patch-Level Training for Large Language Models"☆59Updated 2 months ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆51Updated 5 months ago
- Repository of LV-Eval Benchmark☆41Updated 3 weeks ago
- Distributed IO-aware Attention algorithm☆17Updated 3 weeks ago
- ☆99Updated last year
- A collection of instruction data and scripts for machine translation.☆20Updated 11 months ago
- An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.☆21Updated 7 months ago
- Implementations of online merging optimizers proposed by Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment☆60Updated 3 months ago
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆45Updated last year
- ☆75Updated this week
- A simple toolkit for benchmarking LLMs on mathematical reasoning tasks. 🧮✨☆72Updated 4 months ago
- ☆164Updated 4 months ago
- Official implementation of the paper "From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large L…☆34Updated 2 months ago