thunlp / Ouroboros

Related projects: ⓘ

• DRSY / EasyKV
  Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
  ☆56Updated 7 months ago
• Infini-AI-Lab / MagicDec
  Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆55Updated this week
• Glaciohound / LM-Infinite
  Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆115Updated 2 weeks ago
• princeton-nlp / CEPE
  [ACL 2024] Long-Context Language Modeling with Parallel Encodings
  ☆133Updated 3 months ago
• infinigence / LVEval
  Repository of LV-Eval Benchmark
  ☆41Updated 2 weeks ago
• linfeng93 / BiTA
  An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.
  ☆21Updated 7 months ago
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆134Updated 2 months ago
• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆183Updated last month
• FasterDecoding / REST
  REST: Retrieval-Based Speculative Decoding, NAACL 2024
  ☆158Updated 4 months ago
• hkust-nlp / dart-math
  Official implementation for the paper *🎯DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving*
  ☆57Updated 3 weeks ago
• FasterDecoding / SnapKV
  ☆164Updated 4 months ago
• RulinShao / retrieval-scaling
  Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".
  ☆81Updated 2 weeks ago
• Equationliu / Kangaroo
  Implementation of Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting
  ☆39Updated 2 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆47Updated 2 weeks ago
• NJUNLP / MCSD
  Multi-Candidate Speculative Decoding
  ☆27Updated 4 months ago
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆65Updated 6 months ago
• tianyi-lab / Superfiltering
  [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
  ☆101Updated last week
• bigai-nlco / LooGLE
  ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models
  ☆148Updated 6 months ago
• hkust-nlp / llm-compression-intelligence
  Official github repo for the paper "Compression Represents Intelligence Linearly"
  ☆121Updated 3 months ago
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆72Updated 6 months ago
• THUDM / ReST-MCTS
  ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search
  ☆91Updated 3 months ago
• yegcjs / mixinglaws
  ☆87Updated 4 months ago
• shreyansh26 / Speculative-Sampling
  Implementation of Speculative Sampling as described in "Accelerating Large Language Model Decoding with Speculative Sampling" by Deepmind
  ☆69Updated 6 months ago
• FasterDecoding / TEAL
  ☆75Updated this week
• dilab-zju / self-speculative-decoding
  Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**
  ☆127Updated 3 months ago
• thunlp / MoEfication
  ☆104Updated last month
• TIGER-AI-Lab / LongICLBench
  Code and Data for "Long-context LLMs Struggle with Long In-context Learning"
  ☆87Updated 2 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆36Updated 8 months ago
• OpenLMLab / LongWanjuan
  Towards Systematic Measurement for Long Text Quality
  ☆27Updated 2 weeks ago
• Strivin0311 / llms-learning
  A repository sharing the literatures about large language models
  ☆19Updated 3 weeks ago