facebookresearch / MemoryMosaics
Memory Mosaics are networks of associative memories working in concert to achieve a prediction task.
☆31Updated last week
Related projects: ⓘ
- ☆48Updated 4 months ago
- ☆44Updated 11 months ago
- ☆50Updated last month
- Language models scale reliably with over-training and on downstream tasks☆91Updated 5 months ago
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆34Updated 10 months ago
- Triton Implementation of HyperAttention Algorithm☆46Updated 9 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆84Updated 4 months ago
- ☆54Updated last week
- ☆47Updated 3 months ago
- Code for reproducing our paper "Not All Language Model Features Are Linear"☆57Updated last week
- Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)☆77Updated last year
- ☆42Updated 3 months ago
- Understand and test language model architectures on synthetic tasks.☆156Updated 4 months ago
- Official implementation of Goldfish Loss: Mitigating Memorization in Generative LLMs☆68Updated 2 months ago
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆24Updated 5 months ago
- Universal Neurons in GPT2 Language Models☆25Updated 3 months ago
- Experiments for efforts to train a new and improved t5☆76Updated 5 months ago
- ☆29Updated 2 weeks ago
- ☆69Updated 4 months ago
- ☆22Updated 6 months ago
- Evaluation of neuro-symbolic engines☆29Updated last month
- Repository for NPHardEval, a quantified-dynamic benchmark of LLMs☆46Updated 5 months ago
- ☆53Updated 8 months ago
- A repository for research on medium sized language models.☆71Updated 3 months ago
- Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…☆61Updated last week
- ☆38Updated 8 months ago
- Efficient PScan implementation in PyTorch☆15Updated 8 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore, paper coming soon☆18Updated this week
- Checkpointable dataset utilities for foundation model training☆31Updated 7 months ago
- Sparse and discrete interpretability tool for neural networks☆51Updated 7 months ago