redotvideo / mamba-chat
Mamba-Chat: A chat LLM based on the state-space model architecture 🐍
☆911Updated 8 months ago
Related projects ⓘ
Alternatives and complementary repositories for mamba-chat
- Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"☆804Updated 3 months ago
- [ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning☆615Updated 5 months ago
- A family of open-sourced Mixture-of-Experts (MoE) Large Language Models☆1,394Updated 8 months ago
- Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI☆1,338Updated 7 months ago
- ReFT: Representation Finetuning for Language Models☆1,164Updated 2 weeks ago
- Extend existing LLMs way beyond the original training length with constant memory usage, without retraining☆677Updated 7 months ago
- YaRN: Efficient Context Window Extension of Large Language Models☆1,356Updated 7 months ago
- ☆860Updated 11 months ago
- Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"☆538Updated 6 months ago
- This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?☆732Updated 3 weeks ago
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆648Updated last month
- Open weights language model from Google DeepMind, based on Griffin.☆607Updated 4 months ago
- GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection☆1,436Updated 3 weeks ago
- PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention…☆281Updated 6 months ago
- Fine-tune mistral-7B on 3090s, a100s, h100s☆702Updated last year
- ☆471Updated 3 months ago
- Large Context Attention☆642Updated 3 months ago
- Inference code for Mistral and Mixtral hacked up into original Llama implementation☆373Updated 11 months ago
- Reference implementation of Megalodon 7B model☆507Updated 7 months ago
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,150Updated last month
- Minimalistic large language model 3D-parallelism training☆1,265Updated this week
- Code for fine-tuning Platypus fam LLMs using LoRA☆623Updated 9 months ago
- Official repository for ORPO☆421Updated 5 months ago
- The official implementation of Self-Play Fine-Tuning (SPIN)☆1,048Updated 6 months ago
- A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).☆745Updated this week
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,758Updated 10 months ago
- A bagel, with everything.☆312Updated 7 months ago
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆435Updated 7 months ago