bkitano / llama-from-scratch
Llama from scratch, or How to implement a paper without crying
☆542Updated 8 months ago
Alternatives and similar repositories for llama-from-scratch:
Users that are interested in llama-from-scratch are comparing it to the libraries listed below
- What would you do with 1000 H100s...☆1,001Updated last year
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.☆772Updated this week
- Fine-tune mistral-7B on 3090s, a100s, h100s☆705Updated last year
- LLM Workshop by Sourab Mangrulkar☆363Updated 8 months ago
- nanoGPT style version of Llama 3.1☆1,316Updated 6 months ago
- LoRA and DoRA from Scratch Implementations☆196Updated 11 months ago
- Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"☆841Updated this week
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,194Updated 4 months ago
- ☆996Updated 2 weeks ago
- LLaMA 2 implemented from scratch in PyTorch☆294Updated last year
- From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)☆634Updated 3 months ago
- Puzzles for exploring transformers☆332Updated last year
- llama3.np is a pure NumPy implementation for Llama 3 model.☆973Updated 8 months ago
- Minimalistic 4D-parallelism distributed training framework for education purpose☆724Updated this week
- batched loras☆338Updated last year
- Fast & Simple repository for pre-training and fine-tuning T5-style models☆992Updated 6 months ago
- A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).☆803Updated last week
- YaRN: Efficient Context Window Extension of Large Language Models☆1,421Updated 10 months ago
- A repository for research on medium sized language models.☆491Updated last month
- Well documented, unit tested, type checked and formatted implementation of a vanilla transformer - for educational purposes.☆236Updated 10 months ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆514Updated this week
- Extend existing LLMs way beyond the original training length with constant memory usage, without retraining☆687Updated 10 months ago
- Automatically evaluate your LLMs in Google Colab☆590Updated 9 months ago
- A family of open-sourced Mixture-of-Experts (MoE) Large Language Models☆1,446Updated 11 months ago
- A comprehensive deep dive into the world of tokens☆220Updated 7 months ago
- 🤖 A PyTorch library of curated Transformer models and their composable components☆879Updated 10 months ago
- Minimalistic large language model 3D-parallelism training☆1,483Updated this week
- Puzzles for learning Triton☆1,403Updated 3 months ago
- Serving multiple LoRA finetuned LLM as one☆1,028Updated 9 months ago
- Training LLMs with QLoRA + FSDP☆1,451Updated 3 months ago