kabir2505 / tiny-mixtralLinks
☆45Updated 5 months ago
Alternatives and similar repositories for tiny-mixtral
Users that are interested in tiny-mixtral are comparing it to the libraries listed below
Sorting:
- ☆46Updated 6 months ago
- minimal GRPO implementation from scratch☆98Updated 7 months ago
- An extension of the nanoGPT repository for training small MOE models.☆202Updated 7 months ago
- in this repository, i'm going to implement increasingly complex llm inference optimizations☆68Updated 5 months ago
- NanoGPT-speedrunning for the poor T4 enjoyers☆72Updated 6 months ago
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆193Updated 4 months ago
- A collection of lightweight interpretability scripts to understand how LLMs think☆61Updated this week
- making the official triton tutorials actually comprehensible☆57Updated 2 months ago
- ☆209Updated 9 months ago
- An overview of GRPO & DeepSeek-R1 Training with Open Source GRPO Model Fine Tuning☆37Updated 5 months ago
- ☆446Updated last month
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆59Updated last year
- Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna☆58Updated last week
- A collection of tricks and tools to speed up transformer models☆182Updated this week
- Collection of autoregressive model implementation☆86Updated 6 months ago
- Low memory full parameter finetuning of LLMs☆53Updated 3 months ago
- PTX-Tutorial Written Purely By AIs (Deep Research of Openai and Claude 3.7)☆66Updated 7 months ago
- ☆222Updated 3 weeks ago
- Memory optimized Mixture of Experts☆68Updated 3 months ago
- Simple repository for training small reasoning models☆43Updated 8 months ago
- A curated list of resources for learning and exploring Triton, OpenAI's programming language for writing efficient GPU code.☆421Updated 7 months ago
- nanoGRPO is a lightweight implementation of Group Relative Policy Optimization (GRPO)☆121Updated 5 months ago
- A easy, reliable, fluid template for python packages complete with docs, testing suites, readme's, github workflows, linting and much muc…☆187Updated last week
- 🎈 A series of lightweight GPT models featuring TinyGPT Base (~51M params) and TinyGPT-MoE (~85M params). Fast, creative text generation …☆13Updated last month
- Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)☆160Updated last year
- Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.☆72Updated last week
- rl from zero pretrain, can it be done? yes.☆277Updated 3 weeks ago
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆107Updated 7 months ago
- ☆80Updated last month
- ☆64Updated 7 months ago