YuvrajSingh-mist / SmolLlamaLinks
So, I trained a Llama a 130M architecture I coded from ground up to build a small instruct model from scratch. Trained on FineWeb dataset form HuggingFace consisting of 15 M texts (10BT snapshot) for a total of full 3 epochs
☆15Updated 6 months ago
Alternatives and similar repositories for SmolLlama
Users that are interested in SmolLlama are comparing it to the libraries listed below
Sorting:
- ☆89Updated 5 months ago
- ☆155Updated 5 months ago
- ☆135Updated last year
- Simple examples using Argilla tools to build AI☆55Updated 10 months ago
- Entropy Based Sampling and Parallel CoT Decoding☆17Updated 11 months ago
- ☆75Updated last year
- Verifiers for LLM Reinforcement Learning☆75Updated 2 weeks ago
- Implementation of a GPT-4o like Multimodal from Scratch using Python☆71Updated 5 months ago
- ☆135Updated last month
- ☆68Updated 4 months ago
- ☆46Updated 5 months ago
- ☆44Updated 4 months ago
- rl from zero pretrain, can it be done? yes.☆269Updated this week
- ☆119Updated last year
- Train your own SOTA deductive reasoning model☆106Updated 6 months ago
- A compact LLM pretrained in 9 days by using high quality data☆327Updated 5 months ago
- MLX port for xjdr's entropix sampler (mimics jax implementation)☆62Updated 10 months ago
- chrome & firefox extension to chat with webpages: local llms☆126Updated 9 months ago
- Official PyTorch implementation for Hogwild! Inference: Parallel LLM Generation with a Concurrent Attention Cache☆124Updated last month
- Easy to use, High Performant Knowledge Distillation for LLMs☆93Updated 4 months ago
- ☆86Updated last year
- working implimention of deepseek MLA☆44Updated 8 months ago
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆105Updated 6 months ago
- One click templates for inferencing Language Models☆214Updated last month
- A repository consisting of paper/architecture replications of classic/SOTA AI/ML papers in pytorch☆365Updated 2 weeks ago
- OpenCoconut implements a latent reasoning paradigm where we generate thoughts before decoding.☆172Updated 8 months ago
- 🤗 Benchmark Large Language Models Reliably On Your Data☆395Updated 3 weeks ago
- ☆133Updated 6 months ago
- A fast, local, and secure approach for training LLMs for coding tasks using GRPO with WebAssembly and interpreter feedback.☆38Updated 5 months ago
- ⚖️ Awesome LLM Judges ⚖️☆128Updated 4 months ago