withinmiaov / A-Survey-on-Mixture-of-Experts

Related projects: ⓘ

• GCYZSL / MoLA
  ☆109Updated last month
• EnnengYang / Awesome-Model-Merging-Methods-Theories-Applications
  Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. arXiv:2408.07666.
  ☆104Updated this week
• TUDB-Labs / MixLoRA
  State-of-the-art Parameter-Efficient MoE Fine-tuning Method
  ☆58Updated 3 weeks ago
• wutaiqiang / MoSLoRA
  ☆54Updated 2 months ago
• TsinghuaC3I / SoRA
  The source code of the EMNLP 2023 main conference paper: Sparse Low-rank Adaptation of Pre-trained Language Models.
  ☆62Updated 6 months ago
• Chaos96 / fourierft
  ☆119Updated last week
• UNITES-Lab / MC-SMoE
  [ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
  ☆63Updated 3 months ago
• mrflogs / LoRA-Pro
  Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "
  ☆49Updated last month
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆114Updated 2 months ago
• ycjing / Awesome-Model-Merging
  A curated list of Model Merging methods.
  ☆70Updated last month
• beccabai / Data-centric_multimodal_LLM
  Survey on Data-centric Large Language Models
  ☆58Updated 2 months ago
• lzhxmu / VTW
  ☆19Updated last month
• qianlima-lab / awesome-lifelong-learning-methods-for-llm
  This repository collects awesome survey, resource, and paper for Lifelong Learning for Large Language Models. (Updated Regularly)
  ☆22Updated 2 weeks ago
• Chongjie-Si / Subspace-Tuning
  A generalized framework for subspace tuning methods in parameter efficient fine-tuning.
  ☆59Updated this week
• sail-sg / sdft
  [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".
  ☆81Updated 3 weeks ago
• Dongping-Chen / MLLM-Judge
  [ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.
  ☆47Updated last month
• cmnfriend / O-LoRA
  ☆139Updated 2 months ago
• tanganke / fusion_bench
  FusionBench: A Comprehensive Benchmark of Deep Model Fusion
  ☆42Updated 2 weeks ago
• 42Shawn / LLaVA-PruMerge
  LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models
  ☆86Updated 4 months ago
• Ablustrund / LoRAMoE
  LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment
  ☆190Updated 4 months ago
• RayRuiboChen / Self-Filter
  ☆20Updated 3 months ago
• mathvision-cuhk / MATH-V
  MATH-Vision dataset and code to measure Multimodal Mathematical Reasoning capabilities.
  ☆53Updated 2 weeks ago
• liuqidong07 / MOELoRA-peft
  [SIGIR'24] The official implementation code of MOELoRA.
  ☆113Updated last month
• yushuiwx / MoLE
  ☆40Updated 5 months ago
• MM-LLMs / mm-llms.github.io
  ☆24Updated 5 months ago
• nbasyl / DoRA
  Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
  ☆120Updated 4 months ago
• Outsider565 / LoRA-GA
  ☆125Updated this week
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆56Updated this week
• FreedomIntelligence / MLLM-Bench
  MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria
  ☆49Updated last month
• OpenGVLab / MMT-Bench
  ICML'2024 | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI
  ☆84Updated 2 months ago