kyegomez / OpenStrawberry
An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO
☆29Updated last week
Alternatives and similar repositories for OpenStrawberry:
Users that are interested in OpenStrawberry are comparing it to the libraries listed below
- https://x.com/BlinkDL_AI/status/1884768989743882276☆27Updated 2 months ago
- GoldFinch and other hybrid transformer components☆45Updated 8 months ago
- The simplest, fastest repository for training/finetuning medium-sized xLSTMs.☆42Updated 10 months ago
- Fine-tunes a student LLM using teacher feedback for improved reasoning and answer quality. Implements GRPO with teacher-provided evaluati…☆40Updated last month
- Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.☆17Updated 3 weeks ago
- entropix style sampling + GUI☆25Updated 5 months ago
- Training hybrid models for dummies.☆20Updated 2 months ago
- ☆16Updated last month
- My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated☆31Updated 8 months ago
- A repository for research on medium sized language models.☆76Updated 10 months ago
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆33Updated last year
- Train, tune, and infer Bamba model☆88Updated 2 months ago
- GoldFinch and other hybrid transformer components☆10Updated last week
- BUD-E (Buddy) is an open-source voice assistant framework that facilitates seamless interaction with AI models and APIs, enabling the cre…☆19Updated 6 months ago
- ☆48Updated 5 months ago
- RWKV-7: Surpassing GPT☆82Updated 4 months ago
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆54Updated last year
- ☆31Updated last year
- Lightweight package that tracks and summarizes code changes using LLMs (Large Language Models)☆32Updated last month
- implementation of https://arxiv.org/pdf/2312.09299☆20Updated 9 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆26Updated 6 months ago
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆39Updated 2 months ago
- Course Project for COMP4471 on RWKV☆17Updated last year
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆31Updated 10 months ago
- ☆34Updated 3 weeks ago
- ☆52Updated 3 weeks ago
- Simple Implementation of TinyGPTV in super simple Zeta lego blocks☆16Updated 5 months ago
- Implementation of Spectral State Space Models☆16Updated last year
- RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best…☆43Updated 3 weeks ago
- AgentParse is a high-performance parsing library designed to map various structured data formats (such as Pydantic models, JSON, YAML, an…☆13Updated this week