Xnhyacinth / Awesome-LLM-Long-Context-Modeling
π° Must-read papers and blogs on LLM based Long Context Modeling π₯
β816Updated this week
Related projects: β
- β·οΈ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingβ849Updated 2 months ago
- This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitβ¦β534Updated last week
- [TMLR 2024] Efficient Large Language Models: A Surveyβ959Updated this week
- [ACL 2024] LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingβ618Updated last week
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.β608Updated last month
- SimPO: Simple Preference Optimization with a Reference-Free Rewardβ648Updated 3 weeks ago
- A curated list for Efficient Large Language Modelsβ1,119Updated this week
- Official Implementation of EAGLE-1 and EAGLE-2β749Updated 3 weeks ago
- [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningβ539Updated 6 months ago
- Awesome LLM compression research papers and tools.β1,062Updated this week
- A collection of AWESOME things about mixture-of-expertsβ920Updated last month
- Reading list of hallucination in LLMs. Check out our new survey paper: "Sirenβs Song in the AI Ocean: A Survey on Hallucination in Large β¦β901Updated 2 weeks ago
- Summarize existing representative LLMs text datasets.β824Updated 2 weeks ago
- Fast inference from large lauguage models via speculative decodingβ508Updated 3 weeks ago
- DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelsβ970Updated 8 months ago
- Must-read Papers on Knowledge Editing for Large Language Models.β829Updated 2 weeks ago
- An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & Mixtral)β2,026Updated this week
- Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]β467Updated 3 months ago
- Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"β1,041Updated 6 months ago
- Codebase for Merging Language Models (ICML 2024)β745Updated 4 months ago
- A family of open-sourced Mixture-of-Experts (MoE) Large Language Modelsβ1,353Updated 6 months ago
- β1,194Updated this week
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decodingβ1,099Updated 7 months ago
- β836Updated 2 months ago
- Aligning Large Language Models with Human: A Surveyβ671Updated last year
- Collection of training data management explorations for large language modelsβ266Updated last month
- An Awesome Collection for LLM Surveyβ289Updated last week
- Official repository for "Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing". Your efficient and high-quality sβ¦β398Updated this week
- Awesome-LLM-RAG: a curated list of advanced retrieval augmented generation (RAG) in Large Language Modelsβ871Updated last month
- A repository sharing the literatures about long-context large language models, including the methodologies and the evaluation benchmarksβ239Updated last month