NousResearch / StripedHyenaTrainer
☆61Updated last year
Alternatives and similar repositories for StripedHyenaTrainer:
Users that are interested in StripedHyenaTrainer are comparing it to the libraries listed below
- gzip Predicts Data-dependent Scaling Laws☆34Updated 10 months ago
- ☆49Updated last year
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆82Updated last year
- ☆22Updated last year
- ☆27Updated 9 months ago
- ☆63Updated 7 months ago
- ☆38Updated 8 months ago
- a pipeline for using api calls to agnostically convert unstructured data into structured training data☆30Updated 7 months ago
- Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models☆69Updated last year
- [WIP] Transformer to embed Danbooru labelsets☆13Updated last year
- A MAD laboratory to improve AI architecture designs 🧪☆111Updated 4 months ago
- ☆48Updated last year
- Public Inflection Benchmarks☆68Updated last year
- Simplex Random Feature attention, in PyTorch☆74Updated last year
- an implementation of Self-Extend, to expand the context window via grouped attention☆119Updated last year
- Just large language models. Hackable, with as little abstraction as possible. Done for my own purposes, feel free to rip.☆44Updated last year
- Experiments for efforts to train a new and improved t5☆77Updated last year
- ☆92Updated last year
- ☆79Updated last year
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆39Updated 2 months ago
- ☆48Updated 5 months ago
- NanoGPT (124M) quality in 2.67B tokens☆28Updated this week
- Collection of autoregressive model implementation☆85Updated 2 months ago
- ☆21Updated 5 months ago
- inference code for mixtral-8x7b-32kseqlen☆99Updated last year
- Full finetuning of large language models without large memory requirements☆94Updated last year
- QLoRA for Masked Language Modeling☆22Updated last year
- The simplest, fastest repository for training/finetuning medium-sized xLSTMs.☆42Updated 11 months ago
- Code repository for the c-BTM paper☆106Updated last year
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆96Updated last month