So, I trained a MoE based a 124M (8x12M) architecture I coded from ground up to build a small instruct model, going through the below-mentioned stages from scratch. Trained on TiyStories dataset form HuggingFace consisting of 1M tokens for a total of 14000 steps
☆14Jul 18, 2025Updated last year
Alternatives and similar repositories for SmolMixtral
Users that are interested in SmolMixtral are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- So, I trained a Llama a 130M architecture I coded from ground up to build a small instruct model from scratch. Trained on FineWeb dataset…☆17Mar 26, 2025Updated last year
- Interactive evolutionary exploration of generative design spaces with large language models☆18Jun 26, 2024Updated 2 years ago
- Notes of ADRL course taught at IISC as part of MTech AI curriculum☆15Nov 30, 2024Updated last year
- ☆10Jun 8, 2024Updated 2 years ago
- Indic-Bias is a comprehensive benchmark to evaluate the fairness of LLMs in Indian Contexts.☆18Aug 14, 2025Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- ☆51May 27, 2025Updated last year
- Open source interpretability artefacts for R1.☆186Apr 21, 2025Updated last year
- Pytorch directly integrated to the cloud all through Bench AI!☆10Dec 10, 2023Updated 2 years ago
- aesthetic tensor visualiser☆28Apr 23, 2025Updated last year
- repo of paper implementations☆20Feb 25, 2025Updated last year
- A fast grid-based open-ended MARL environment☆18Oct 6, 2026Updated 3 weeks ago
- OMNI: Open-endedness via Models of human Notions of Interestingness☆66Jan 28, 2025Updated last year
- A simple Pure Python/PyTorch performance daemon for training workloads☆15Jul 28, 2026Updated 3 months ago
- Whalegrad 🐳 is a lightweight deep learning library written in C.☆10Jan 5, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- 2023 Capstone Design☆12Nov 2, 2023Updated 2 years ago
- An repository of 2025-2026 AI Safety and Alignment programs, camps, and workshops.☆22May 18, 2025Updated last year
- Your Universal Cellular Automata☆15Aug 28, 2026Updated 2 months ago
- The fullstack clone version of Hugging Face Hub! An enterprise level of Next.js app and preparing for more..☆14Aug 3, 2023Updated 3 years ago
- Notebooks for the DLP course by Prof.Mitesh Khapra and me, offered to IITM BS Students.☆26May 21, 2025Updated last year
- Arrakis is a library to conduct, track and visualize mechanistic interpretability experiments.☆31Jul 8, 2026Updated 3 months ago
- African Medical QA Dataset☆14Sep 17, 2025Updated last year
- Learning notes and code from CS 329S: Machine Learning Systems Design series.☆24Jun 23, 2025Updated last year
- ☆16Jul 8, 2024Updated 2 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Team Unagi's repository for ICFPC 2025☆14Sep 8, 2025Updated last year
- Hiroki Sayama's artificial chemistry model -> http://bingweb.binghamton.edu/~sayama/SwarmChemistry/☆17Jul 25, 2022Updated 4 years ago
- A lightweight open-source package to fine-tune embedding models.☆22Feb 4, 2024Updated 2 years ago
- This repository demonstrates how to fine-tune the Google Gemma 2 2B model to improve its performance on Japanese instruction-following ta…☆12Aug 11, 2024Updated 2 years ago
- in this repository, i'm going to implement increasingly complex llm inference optimizations☆86May 22, 2025Updated last year
- talk to sarvam APIs directly, without code.☆12Jun 9, 2025Updated last year
- ☆16Aug 7, 2024Updated 2 years ago
- A simple Python life simulator game☆15Nov 25, 2019Updated 6 years ago
- Teach a computer to play any game.☆10Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- hybrid classical-quantum transfer learning for cardiomegaly detection on chest x-rays☆12Jul 12, 2023Updated 3 years ago
- ☆17May 15, 2025Updated last year
- ☆11May 14, 2024Updated 2 years ago
- This repository consist GPS Navigation and spacecraft attitude control matlab codes☆14Apr 30, 2020Updated 6 years ago
- Runtime repository for the SNOMED CT Entity Linking challenge on DrivenData☆14Mar 5, 2024Updated 2 years ago
- Multilayer Authenticity Identifier (MAI), a CNN model that attempts to identify synthetic AI images.☆34Jul 30, 2026Updated 2 months ago
- A curated list of awesome Artificial Life simulators, papers and resources.☆19Dec 9, 2025Updated 10 months ago