liuyz0 / SuperpositionScalingLinks
Superposition Yields Robust Neural Scaling
☆44Updated last month
Alternatives and similar repositories for SuperpositionScaling
Users that are interested in SuperpositionScaling are comparing it to the libraries listed below
Sorting:
- ☆107Updated last year
- ☆33Updated last year
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆87Updated last year
- ☆18Updated 5 months ago
- Sparse and discrete interpretability tool for neural networks☆65Updated last year
- [ICLR 2025] Code for the paper "Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning"☆87Updated 10 months ago
- [ICLR 2025] Code for the paper "Implicit Search via Discrete Diffusion: A Study on Chess"☆36Updated 10 months ago
- Learning from preferences is a common paradigm for fine-tuning language models. Yet, many algorithmic design decisions come into play. Ou…☆32Updated last year
- Efficient Dictionary Learning with Switch Sparse Autoencoders (SAEs)☆25Updated last year
- Universal Neurons in GPT2 Language Models☆31Updated last year
- Complete Reinforcement Learning Toolkit for Large Language Models!☆21Updated 5 months ago
- AIRA-dojo: a framework for developing and evaluating AI research agents☆122Updated last month
- Reinforcing General Reasoning without Verifiers☆93Updated 6 months ago
- ☆23Updated 11 months ago
- Language models scale reliably with over-training and on downstream tasks☆100Updated last year
- Official PyTorch implementation and models for paper "Diffusion Beats Autoregressive in Data-Constrained Settings". We find diffusion mod…☆118Updated 2 months ago
- NeurIPS 2024 tutorial on LLM Inference☆47Updated last year
- Codebase for Context-aware Meta-learned Loss Scaling (CaMeLS). https://arxiv.org/abs/2305.15076.☆25Updated last year
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆78Updated last year
- [ICML 2025] Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction☆82Updated 7 months ago
- The repository contains code for Adaptive Data Optimization☆30Updated last year
- Official repository for our paper, Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Mode…☆20Updated last year
- ☆33Updated last year
- Using FlexAttention to compute attention with different masking patterns☆47Updated last year
- ☆91Updated last year
- Gemstones: A Model Suite for Multi-Faceted Scaling Laws (NeurIPS 2025)☆31Updated 3 months ago
- Self-Supervised Alignment with Mutual Information☆20Updated last year
- Replicating O1 inference-time scaling laws☆91Updated last year
- Code for "Reasoning to Learn from Latent Thoughts"☆124Updated 9 months ago
- Code Release for "Broken Neural Scaling Laws" (BNSL) paper☆59Updated 2 years ago