formll / resolving-scaling-law-discrepanciesLinks
☆19Updated 10 months ago
Alternatives and similar repositories for resolving-scaling-law-discrepancies
Users that are interested in resolving-scaling-law-discrepancies are comparing it to the libraries listed below
Sorting:
- ☆32Updated 4 months ago
- ☆25Updated 3 months ago
- The repository contains code for Adaptive Data Optimization☆24Updated 5 months ago
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Updated last year
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆32Updated 2 months ago
- Efficient Scaling laws and collaborative pretraining.☆16Updated 4 months ago
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆26Updated 7 months ago
- ☆15Updated last month
- [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…☆16Updated last month
- Is In-Context Learning Sufficient for Instruction Following in LLMs? [ICLR 2025]☆30Updated 4 months ago
- The official implementation for Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free☆38Updated 3 weeks ago
- ☆14Updated last year
- Exploration of automated dataset selection approaches at large scales.☆41Updated 3 months ago
- Self-Supervised Alignment with Mutual Information☆18Updated last year
- Offcial Repo of Paper "Eliminating Position Bias of Language Models: A Mechanistic Approach""☆14Updated 9 months ago
- Code for the paper "Data Feedback Loops: Model-driven Amplification of Dataset Biases"☆16Updated 2 years ago
- ☆19Updated 2 months ago
- Code for reproducing our paper "Low Rank Adapting Models for Sparse Autoencoder Features"☆10Updated 2 months ago
- Codebase for Instruction Following without Instruction Tuning☆34Updated 8 months ago
- Code for paper: "LASeR: Learning to Adaptively Select Reward Models with Multi-Arm Bandits"☆13Updated 8 months ago
- Code for ICLR 2025 Paper "What is Wrong with Perplexity for Long-context Language Modeling?"☆78Updated 2 weeks ago
- Official implementation of Bootstrapping Language Models via DPO Implicit Rewards☆44Updated last month
- Official code for the paper "Attention as a Hypernetwork"☆36Updated 11 months ago
- Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"☆23Updated last month
- ☆13Updated 5 months ago
- Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)☆80Updated last year
- ☆31Updated last year
- ☆18Updated 3 months ago
- PaCE: Parsimonious Concept Engineering for Large Language Models (NeurIPS 2024)☆35Updated 6 months ago
- This is the oficial repository for "Safer-Instruct: Aligning Language Models with Automated Preference Data"☆17Updated last year