Phylliida / MambaLensLinks
Mamba support for transformer lens
☆19Updated last year
Alternatives and similar repositories for MambaLens
Users that are interested in MambaLens are comparing it to the libraries listed below
Sorting:
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆88Updated last year
- ☆53Updated last year
- Stick-breaking attention☆62Updated 7 months ago
- Universal Neurons in GPT2 Language Models☆30Updated last year
- Sparse Autoencoder Training Library☆56Updated 9 months ago
- ☆112Updated last year
- ☆74Updated last year
- Replicating O1 inference-time scaling laws☆92Updated last year
- ☆51Updated 2 years ago
- ☆12Updated last year
- A library for efficient patching and automatic circuit discovery.☆88Updated last month
- Code for reproducing our paper "Not All Language Model Features Are Linear"☆83Updated last year
- ☆34Updated 2 years ago
- [NeurIPS 2024 Spotlight] Code and data for the paper "Finding Transformer Circuits with Edge Pruning".☆64Updated 5 months ago
- Language models scale reliably with over-training and on downstream tasks☆99Updated last year
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Updated last year
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Updated last year
- Official implementation of the transformer (TF) architecture suggested in a paper entitled "Looped Transformers as Programmable Computers…☆30Updated 2 years ago
- ☆91Updated last year
- ☆35Updated last year
- ☆108Updated last year
- Code for the paper: https://arxiv.org/pdf/2309.06979.pdf☆21Updated last year
- ☆27Updated 4 months ago
- Official Pytorch Implementation of "The Curse of Depth in Large Language Models" by Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin,Yefen…☆65Updated last month
- ☆16Updated last year
- Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"☆23Updated 9 months ago
- Here we will test various linear attention designs.☆62Updated last year
- Repository for NPHardEval, a quantified-dynamic benchmark of LLMs☆63Updated last year
- ☆39Updated last year
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆42Updated last month