facebookresearch / Pitfalls-of-MemorizationLinks
Understanding the interplay between memorization and generalization in neural networks, featuring MAT, a learning algorithm to enhance robustness by mitigating spurious correlations.
☆40Updated 11 months ago
Alternatives and similar repositories for Pitfalls-of-Memorization
Users that are interested in Pitfalls-of-Memorization are comparing it to the libraries listed below
Sorting:
- Data for "Datamodels: Predicting Predictions with Training Data"☆97Updated 2 years ago
- ☆63Updated 3 years ago
- Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)☆81Updated 2 years ago
- LLM finetuning in resource-constrained environments.☆54Updated last year
- This repository contains the code used for the experiments in the paper "Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity…☆28Updated last month
- ☆37Updated 2 years ago
- ☆19Updated 3 years ago
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023☆138Updated last year
- A centralized place for deep thinking code and experiments☆88Updated 2 years ago
- ☆167Updated 2 years ago
- Experiments and code to generate the GINC small-scale in-context learning dataset from "An Explanation for In-context Learning as Implici…☆106Updated 2 years ago
- ☆12Updated last year
- Code Release for "Broken Neural Scaling Laws" (BNSL) paper☆59Updated 2 years ago
- ☆242Updated last year
- ☆56Updated 2 years ago
- nanoGPT-like codebase for LLM training☆113Updated last month
- Code accompanying our paper "Feature Learning in Infinite-Width Neural Networks" (https://arxiv.org/abs/2011.14522)☆63Updated 4 years ago
- This repository includes code to reproduce the tables in "Loss Landscapes are All You Need: Neural Network Generalization Can Be Explaine…☆40Updated 2 years ago
- ☆45Updated 2 years ago
- A library to create and manage configuration files, especially for machine learning projects.☆79Updated 3 years ago
- The official repository for our paper "Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks". We…☆46Updated 2 years ago
- Framework code with wandb, checkpointing, logging, configs, experimental protocols. Useful for fine-tuning models or training from scratc…☆152Updated 2 years ago
- Official code for "In Search of Robust Measures of Generalization" (NeurIPS 2020)☆28Updated 4 years ago
- ☆31Updated 2 years ago
- ☆85Updated last year
- The official repository for our paper "The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers". We s…☆67Updated 3 years ago
- Official repository for our paper, Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Mode…☆20Updated last year
- CIFAR10 ResNets implemented in JAX+Flax☆12Updated 3 years ago
- ☆76Updated last year
- Language models scale reliably with over-training and on downstream tasks☆100Updated last year