So, I trained a Llama a 130M architecture I coded from ground up to build a small instruct model from scratch. Trained on FineWeb dataset form HuggingFace consisting of 15 M texts (10BT snapshot) for a total of full 3 epochs
☆17Mar 26, 2025Updated last year
Alternatives and similar repositories for SmolLlama
Users that are interested in SmolLlama are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- So, I trained a MoE based a 124M (8x12M) architecture I coded from ground up to build a small instruct model, going through the below-men…☆14Jul 18, 2025Updated last year
- A complete framework for training Large Language Models from scratch☆24Jan 8, 2026Updated 9 months ago
- A trainable activation function incorporating sine waves☆10Jun 8, 2021Updated 5 years ago
- Notes of ADRL course taught at IISC as part of MTech AI curriculum☆15Nov 30, 2024Updated last year
- ☆12Jul 17, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Training Coder Models with Critique Reinforcement Learning [ICLR26]☆13Apr 11, 2026Updated 6 months ago
- High-performance batched Top-K selection for CPU inference. Up to 80x faster than PyTorch, optimized for LLM sampling with AVX2 SIMD.☆18Mar 20, 2026Updated 7 months ago
- This is a simple example of how to serve a DeepSeek model with Azure ML.☆10Feb 10, 2025Updated last year
- Awesome-RL-Reasoning☆17Aug 26, 2026Updated 2 months ago
- VQE on photonic quantum devices☆19Oct 1, 2026Updated last month
- https://github.com/juliakorea/translate-doc 로 옮깁니다☆10Nov 21, 2017Updated 8 years ago
- ☆12Dec 20, 2024Updated last year
- [ICML 2026] ETS: Energy-Guided Test-Time Scaling for Training-Free RL Alignment☆21May 15, 2026Updated 5 months ago
- 🔍 Code Search Tools & Experiments☆13Oct 21, 2026Updated 2 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Fast and memory-efficient exact attention☆22Sep 10, 2026Updated last month
- It shows advanced RAG and 4 design patterns of agentic workflow based on LangChain and LangGraph using Amazon Nova Pro.☆14Mar 1, 2025Updated last year
- ☆13Jan 29, 2025Updated last year
- An repository of 2025-2026 AI Safety and Alignment programs, camps, and workshops.☆22May 18, 2025Updated last year
- ☆14Mar 3, 2026Updated 8 months ago
- Arrakis is a library to conduct, track and visualize mechanistic interpretability experiments.☆31Jul 8, 2026Updated 3 months ago
- Learning notes and code from CS 329S: Machine Learning Systems Design series.☆24Jun 23, 2025Updated last year
- Notebooks for the DLP course by Prof.Mitesh Khapra and me, offered to IITM BS Students.☆26May 21, 2025Updated last year
- ☆27Jun 7, 2026Updated 5 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- [EMNLP 2026] MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Scaling of Diffusion Language Models☆30Oct 6, 2026Updated last month
- ☆24Nov 4, 2024Updated 2 years ago
- Local-First Desktop Evals, Observability & AI Memory for Your Agents and Coding Tools☆15Mar 21, 2026Updated 7 months ago
- Tutorial for DSPy☆29May 1, 2024Updated 2 years ago
- ChatBot App built using LangChain and Lightning AI☆16Mar 4, 2023Updated 3 years ago
- Real time faster whisper gradio☆24Aug 17, 2025Updated last year
- JAX port of FLUX.1 models using flax.nnx☆23Sep 28, 2024Updated 2 years ago
- This GUI aims to simplify the process of converting GGUF files to llamafile format by providing an intuitive and convenient way for users…☆14Jan 2, 2026Updated 10 months ago
- Agent CLI☆13Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆28Jul 2, 2026Updated 4 months ago
- ☆10Jan 18, 2022Updated 4 years ago
- Official implementation of the paper "Pretraining Language Models to Ponder in Continuous Space"☆28Jul 21, 2025Updated last year
- A tool to build a graph from a codebase☆14Feb 19, 2025Updated last year
- CBR (Case based reasoning) chat bot with C#.☆16Apr 30, 2019Updated 7 years ago
- Dependency call extraction tool for Python modules (function, class, import statement)☆13Aug 7, 2024Updated 2 years ago
- The best ChatGPT that $100 can buy.☆61Oct 27, 2026Updated last week