evintunador / minLlama3Links
a simplified version of Meta's Llama 3 model to be used for learning
☆42Updated last year
Alternatives and similar repositories for minLlama3
Users that are interested in minLlama3 are comparing it to the libraries listed below
Sorting:
- LLaMA 2 implemented from scratch in PyTorch☆350Updated last year
- This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT)…☆71Updated last year
- LLaMA 3 is one of the most promising open-source model after Mistral, we will recreate it's architecture in a simpler manner.☆185Updated last year
- LoRA and DoRA from Scratch Implementations☆212Updated last year
- Distributed training (multi-node) of a Transformer model☆83Updated last year
- Combining ViT and GPT-2 for image captioning. Trained on MS-COCO. The model was implemented mostly from scratch.☆43Updated last year
- Tutorial for how to build BERT from scratch☆99Updated last year
- minimal GRPO implementation from scratch☆97Updated 6 months ago
- From scratch implementation of a vision language model in pure PyTorch☆239Updated last year
- Building a 2.3M-parameter LLM from scratch with LLaMA 1 architecture.☆182Updated last year
- A compact LLM pretrained in 9 days by using high quality data☆323Updated 5 months ago
- LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch☆116Updated 2 years ago
- LLM Workshop by Sourab Mangrulkar☆395Updated last year
- Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget☆161Updated last month
- First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…☆178Updated 2 months ago
- Implementation of DoRA☆301Updated last year
- Notes and commented code for RLHF (PPO)☆107Updated last year
- Experimenting with small language models☆71Updated last year
- A bagel, with everything.☆324Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆201Updated last year
- An extension of the nanoGPT repository for training small MOE models.☆187Updated 6 months ago
- Official repository for ORPO☆464Updated last year
- Reference implementation of Mistral AI 7B v0.1 model.☆28Updated last year
- PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention…☆291Updated last year
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆48Updated last year
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆121Updated last year
- An Open Source Toolkit For LLM Distillation☆724Updated 2 months ago
- A set of scripts and notebooks on LLM finetunning and dataset creation☆110Updated 11 months ago
- nanogpt turned into a chat model☆72Updated 2 years ago
- Various installation guides for Large Language Models☆74Updated 4 months ago