Alternatives and similar repositories for SLM_Survey:

Users that are interested in SLM_Survey are comparing it to the libraries listed below

• hetailang / SqueezeAttention
  ☆37Updated 6 months ago
• 18907305772 / FuseAI
  FuseAI Project
  ☆85Updated 2 months ago
• HanGuo97 / lq-lora
  ☆125Updated last year
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆152Updated last week
• tigerchen52 / awesome_role_of_small_models
  a curated list of the role of small models in the LLM era
  ☆100Updated 7 months ago
• hao-ai-lab / Dynasor
  Simple extension on vLLM to help you speed up reasoning model without training.
  ☆146Updated this week
• SalesforceAIResearch / GemFilter
  ☆76Updated 3 months ago
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆115Updated 4 months ago
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆44Updated this week
• thu-coai / MiniPLM
  [ICLR 2025] MiniPLM: Knowledge Distillation for Pre-Training Language Models
  ☆37Updated 5 months ago
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆87Updated 11 months ago
• hdong920 / GRIFFIN
  ☆36Updated 7 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆131Updated 10 months ago
• DAMO-NLP-SG / LongPO
  [ICLR 2025] LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
  ☆33Updated last month
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆116Updated 10 months ago
• LeanModels / DFloat11
  ☆20Updated this week
• bigcode-project / astraios
  Astraios: Parameter-Efficient Instruction Tuning Code Language Models
  ☆57Updated last year
• NVlabs / MaskLLM
  [NeurIPS 24 Spotlight] MaskLLM: Learnable Semi-structured Sparsity for Large Language Models
  ☆161Updated 3 months ago
• CASE-Lab-UMD / Unified-MoE-Compression
  The official implementation of the paper "Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques (TMLR)".
  ☆66Updated last month
• clinicalml / co-llm
  Co-LLM: Learning to Decode Collaboratively with Multiple Language Models
  ☆112Updated 11 months ago
• NuoJohnChen / JudgeLRM
  ☆22Updated last week
• HKUDS / SepLLM
  SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
  ☆70Updated 4 months ago
• IST-DASLab / QuEST
  Work in progress.
  ☆56Updated 2 weeks ago
• microsoft / LongRoPE
  LongRoPE is a novel method that can extends the context window of pre-trained LLMs to an impressive 2048k tokens.
  ☆219Updated 8 months ago
• scitix / MEAP
  Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
  ☆28Updated 2 months ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆34Updated 7 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆121Updated 3 months ago
• wuhy68 / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆142Updated 7 months ago
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆76Updated 11 months ago
• InternLM / SWE-Fixer
  ☆85Updated 2 months ago