AI-Guru / helibrunnaLinks
A HuggingFace compatible Small Language Model trainer.
☆75Updated 4 months ago
Alternatives and similar repositories for helibrunna
Users that are interested in helibrunna are comparing it to the libraries listed below
Sorting:
- ☆29Updated last month
- Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.☆56Updated 2 weeks ago
- Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"☆100Updated 5 months ago
- SaLSa Optimizer implementation (No learning rates needed)☆30Updated 2 weeks ago
- ☆80Updated last year
- A State-Space Model with Rational Transfer Function Representation.☆78Updated last year
- Implementation of a Light Recurrent Unit in Pytorch☆47Updated 8 months ago
- A byte-level decoder architecture that matches the performance of tokenized Transformers.☆63Updated last year
- Code and pretrained models for the paper: "MatMamba: A Matryoshka State Space Model"☆59Updated 6 months ago
- ☆57Updated 3 weeks ago
- Explorations into adversarial losses on top of autoregressive loss for language modeling☆36Updated 3 months ago
- ☆45Updated 4 months ago
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆117Updated last year
- A tool to assist in the interpretation of learned features in sparse autoencoders (in particular the four SAE's trained by Joseph Bloom o…☆19Updated 8 months ago
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆52Updated 2 months ago
- Implementation of GateLoop Transformer in Pytorch and Jax☆90Updated 11 months ago
- Implementation of Agent Attention in Pytorch☆90Updated 10 months ago
- ☆55Updated this week
- ☆31Updated last year
- ☆55Updated last month
- Efficient encoder-decoder architecture for small language models (≤1B parameters) with cross-architecture knowledge distillation and visi…☆27Updated 4 months ago
- Scaling is a distributed training library and installable dependency designed to scale up neural networks, with a dedicated module for tr…☆62Updated 7 months ago
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆56Updated last year
- https://x.com/BlinkDL_AI/status/1884768989743882276☆28Updated last month
- The simplest, fastest repository for training/finetuning medium-sized xLSTMs.☆41Updated last year
- Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"☆119Updated 2 months ago
- Set of scripts to finetune LLMs