locuslab / scaling_laws_data_filtering

Related projects: ⓘ

• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆44Updated 8 months ago
• kamanphoebe / Look-into-MoEs
  A Closer Look into Mixture-of-Experts in Large Language Models
  ☆38Updated last month
• srush / LLM-Talk
  ☆45Updated 7 months ago
• sail-sg / scaling-with-vocab
  📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆52Updated 3 weeks ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆48Updated last week
• ZitongYang / Synthetic_Continued_Pretraining
  Code implementation of synthetic continued pretraining
  ☆13Updated this week
• yegcjs / mixinglaws
  ☆87Updated 4 months ago
• facebookresearch / RLCD
  Reproduction of "RLCD Reinforcement Learning from Contrast Distillation for Language Model Alignment
  ☆63Updated last year
• hkust-nlp / llm-compression-intelligence
  Official github repo for the paper "Compression Represents Intelligence Linearly"
  ☆121Updated 3 months ago
• kyleliang919 / Online-Subspace-Descent
  This repo is based on https://github.com/jiaweizzhao/GaLore, paper coming soon
  ☆18Updated this week
• cxcscmu / MATES
  Official repository for MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models
  ☆42Updated last week
• GAIR-NLP / scaleeval
  Scalable Meta-Evaluation of LLMs as Evaluators
  ☆39Updated 7 months ago
• mrflogs / LoRA-Pro
  Official code for our paper, "LoRA-Pro: Are Low-Rank Adapters Properly Optimized? "
  ☆49Updated last month
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆42Updated last year
• RulinShao / retrieval-scaling
  Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".
  ☆81Updated 2 weeks ago
• kyegomez / Lets-Verify-Step-by-Step
  "Improving Mathematical Reasoning with Process Supervision" by OPENAI
  ☆55Updated last week
• ernie-research / Tool-Augmented-Reward-Model
  [ICLR'24 spotlight] Tool-Augmented Reward Modeling
  ☆33Updated 6 months ago
• Zyphra / Zyda_processing
  ☆22Updated 3 months ago
• IBM / SALMON
  Self-Alignment with Principle-Following Reward Models
  ☆144Updated 6 months ago
• tml-epfl / icl-alignment
  Is In-Context Learning Sufficient for Instruction Following in LLMs?
  ☆19Updated 3 months ago
• liyucheng09 / Contamination_Detector
  Lightweight tool to identify Data Contamination in LLMs evaluation
  ☆39Updated 6 months ago
• dvlab-research / Q-LLM
  This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
  ☆36Updated 2 months ago
• yale-nlp / refdpo
  ☆13Updated last month
• wwxu21 / CUT
  Source code of "Reasons to Reject? Aligning Language Models with Judgments"
  ☆54Updated 6 months ago
• yegcjs / DiffusionLLM
  Code for paper "Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning"
  ☆59Updated 7 months ago
• WHGTyen / BIG-Bench-Mistake
  A dataset of LLM-generated chain-of-thought steps annotated with mistake location.
  ☆65Updated last month
• princeton-nlp / PTP
  Improving Language Understanding from Screenshots. Paper: https://arxiv.org/abs/2402.14073
  ☆24Updated 2 months ago
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆72Updated 6 months ago
• sail-sg / regmix
  🧬 RegMix: Data Mixture as Regression for Language Model Pre-training
  ☆79Updated this week
• fxmeng / mixtral_spliter
  Converting Mixtral-8x7B to Mixtral-[1~7]x7B
  ☆20Updated 6 months ago