kamanphoebe / Look-into-MoEs

Related projects: ⓘ

• yale-nlp / refdpo
  ☆13Updated last month
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆42Updated last year
• Shwai-He / MEO
  The source code of "Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (EMNLP 2023)":
  ☆31Updated 5 months ago
• locuslab / scaling_laws_data_filtering
  ☆60Updated 5 months ago
• princeton-nlp / Edge-Pruning
  Code and data for the paper "Finding Transformer Circuits with Edge Pruning".
  ☆20Updated 2 months ago
• facebookresearch / RLCD
  Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment
  ☆63Updated last year
• sail-sg / scaling-with-vocab
  📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆52Updated 3 weeks ago
• hkust-nlp / dart-math
  Official implementation for the paper *🎯DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving*
  ☆57Updated 3 weeks ago
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models
  ☆42Updated last week
• ernie-research / Tool-Augmented-Reward-Model
  [ICLR'24 spotlight] Tool-Augmented Reward Modeling
  ☆33Updated 6 months ago
• RLHFlow / Directional-Preference-Alignment
  Directional Preference Alignment
  ☆44Updated 3 months ago
• liyucheng09 / Contamination_Detector
  Lightweight tool to identify Data Contamination in LLMs evaluation
  ☆39Updated 6 months ago
• Shwai-He / LLM-Drop
  The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".
  ☆43Updated last week
• raymin0223 / fast_robust_early_exit
  Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
  ☆51Updated 3 months ago
• whyNLP / Conic10K
  Conic10K: A large-scale dataset for closed-vocabulary math problem understanding. Accepted to EMNLP2023 Findings.
  ☆23Updated 9 months ago
• RUCAIBox / BAMBOO
  ☆31Updated 5 months ago
• DRSY / KV_Compression
  [EMNLP 2023]Context Compression for Auto-regressive Transformers with Sentinel Tokens
  ☆20Updated 10 months ago
• TianduoWang / DPO-ST
  [ACL 2024] Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning
  ☆24Updated last month
• facebookresearch / RAM
  A framework to study AI models in Reasoning, Alignment, and use of Memory (RAM).
  ☆15Updated this week
• WooSunghyeon / dropbp
  The official code for Dropping Backward Propagation (DropBP)
  ☆18Updated 3 months ago
• casmlab / NPHardEval
  Repository for NPHardEval, a quantified-dynamic benchmark of LLMs
  ☆46Updated 5 months ago
• srush / LLM-Talk
  ☆45Updated 7 months ago
• VITA-Group / Data-Efficient-Scaling
  [ICML 2023] "Data Efficient Neural Scaling Law via Model Reusing" by Peihao Wang, Rameswar Panda, Zhangyang Wang
  ☆13Updated 8 months ago
• RulinShao / retrieval-scaling
  Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".
  ☆81Updated 2 weeks ago
• googleinterns / localizing-paragraph-memorization
  ☆14Updated 6 months ago
• chujiezheng / LLM-Extrapolation
  Official repository for paper "Weak-to-Strong Extrapolation Expedites Alignment"
  ☆62Updated 3 months ago
• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆44Updated 8 months ago
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆33Updated last month
• ZongqianLi / 500xCompressor
  ☆14Updated last month
• HazyResearch / prefix-linear-attention
  ☆41Updated 2 months ago