tanaymeh / mamba-trainLinks
A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
☆59Updated last year
Alternatives and similar repositories for mamba-train
Users that are interested in mamba-train are comparing it to the libraries listed below
Sorting:
- RWKV-7: Surpassing GPT☆96Updated 10 months ago
- Collection of autoregressive model implementation☆86Updated 5 months ago
- https://x.com/BlinkDL_AI/status/1884768989743882276☆28Updated 5 months ago
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆36Updated 2 months ago
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆161Updated 5 months ago
- working implimention of deepseek MLA☆44Updated 8 months ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆99Updated last year
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆130Updated 10 months ago
- Implementation of MambaByte in "MambaByte: Token-free Selective State Space Model" in Pytorch and Zeta☆123Updated 3 weeks ago
- Code repository for Black Mamba☆255Updated last year
- PyTorch implementation of models from the Zamba2 series.☆185Updated 8 months ago
- The simplest, fastest repository for training/finetuning medium-sized xLSTMs.☆41Updated last year
- ☆64Updated 6 months ago
- Work in progress.☆74Updated 3 months ago
- A repository for research on medium sized language models.☆78Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆128Updated last year
- A byte-level decoder architecture that matches the performance of tokenized Transformers.☆66Updated last year
- This is the code that went into our practical dive using mamba as information extraction☆55Updated last year
- Implementation of the Mamba SSM with hf_integration.☆56Updated last year
- ☆85Updated last year
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks (EMNLP'24)☆148Updated last year
- Lightweight toolkit package to train and fine-tune 1.58bit Language models☆89Updated 4 months ago
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆189Updated 3 weeks ago
- Implementation of Mind Evolution, Evolving Deeper LLM Thinking, from Deepmind☆57Updated 4 months ago
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆105Updated 6 months ago
- ☆82Updated last year
- The official repository for HyperZ⋅Z⋅W Operator Connects Slow-Fast Networks for Full Context Interaction.☆40Updated 6 months ago
- GoldFinch and other hybrid transformer components☆45Updated last year
- ☆135Updated last year
- Evaluating the Mamba architecture on the Othello game☆47Updated last year