liyucheng09 / llm-compressive
Longitudinal Evaluation of LLMs via Data Compression
☆26Updated 5 months ago
Related projects ⓘ
Alternatives and complementary repositories for llm-compressive
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting (EMNLP 2024 main)☆76Updated last month
- An Experiment on Dynamic NTK Scaling RoPE☆61Updated 11 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆73Updated 8 months ago
- code for Scaling Laws of RoPE-based Extrapolation☆70Updated last year
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆52Updated 7 months ago
- [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection☆28Updated 3 weeks ago
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆49Updated last year
- The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.☆38Updated last month
- An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.☆22Updated 9 months ago
- We introduce ScaleQuest, a scalable, novel and cost-effective data synthesis method to unleash the reasoning capability of LLMs.☆51Updated 3 weeks ago
- Code for "Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes"☆28Updated 7 months ago
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆125Updated 2 months ago
- Implementations of online merging optimizers proposed by Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment☆66Updated 5 months ago
- Official PyTorch implementation of IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact☆32Updated 5 months ago
- Odysseus: Playground of LLM Sequence Parallelism☆57Updated 5 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆147Updated 5 months ago
- ☆88Updated last month
- Official implementation of the paper "From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large L…☆38Updated 4 months ago
- [ACL 2024] MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues☆51Updated 3 months ago
- Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales☆31Updated last year
- The official code for paper "parallel speculative decoding with adaptive draft length."☆24Updated 2 months ago
- Source code of "Reasons to Reject? Aligning Language Models with Judgments"☆56Updated 8 months ago
- Distributed IO-aware Attention algorithm☆17Updated 3 months ago
- ☆89Updated 7 months ago
- Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)☆58Updated 9 months ago
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆118Updated 4 months ago
- Suri: Multi-constraint instruction following for long-form text generation (EMNLP’24)☆17Updated last week
- Homepage for ProLong (Princeton long-context language models) and paper "How to Train Long-Context Language Models (Effectively)"☆119Updated 3 weeks ago
- Implementation of Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting☆44Updated 4 months ago
- Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"☆128Updated last month