evintunador / minLlama3
a simplified version of Meta's Llama 3 model to be used for learning
☆26Updated 3 months ago
Related projects: ⓘ
- LLaMA 2 implemented from scratch in PyTorch☆216Updated 11 months ago
- Notes and commented code for RLHF (PPO)☆29Updated 6 months ago
- From scratch implementation of a vision language model in pure PyTorch☆149Updated 4 months ago
- This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT)…☆47Updated 11 months ago
- [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning☆309Updated 2 weeks ago
- An Open Source Toolkit For LLM Distillation☆284Updated last month
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆45Updated 3 months ago
- Reference implementation of Mistral AI 7B v0.1 model.☆26Updated 8 months ago
- Official repository for ORPO☆409Updated 3 months ago
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆105Updated 3 months ago
- [ACL 2024] Progressive LLaMA with Block Expansion.☆464Updated 4 months ago
- LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch☆72Updated last year
- Pre-training code for Amber 7B LLM☆148Updated 4 months ago
- Starter pack for NeurIPS LLM Efficiency Challenge 2023.☆115Updated last year
- A set of scripts and notebooks on LLM finetunning and dataset creation☆89Updated last week
- Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind☆161Updated last week
- A family of compressed models obtained via pruning and knowledge distillation☆241Updated 3 weeks ago
- Explore a comprehensive collection of resources, tutorials, papers, tools, and best practices for fine-tuning Large Language Models (LLMs…☆135Updated 3 weeks ago
- Training and Fine-tuning an llm in Python and PyTorch.☆38Updated last year
- LLM Workshop by Sourab Mangrulkar☆322Updated 3 months ago
- LoRA and DoRA from Scratch Implementations☆179Updated 6 months ago
- Official Implementation of "Multi-Head RAG: Solving Multi-Aspect Problems with LLMs"☆155Updated 2 months ago
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆120Updated last week
- ☆419Updated 2 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆158Updated 2 months ago
- Simple Adaptation of BitNet☆29Updated 5 months ago
- Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget☆115Updated 5 months ago
- Prune transformer layers☆60Updated 3 months ago
- LLaMA 3 is one of the most promising open-source model after Mistral, we will recreate it's architecture in a simpler manner.☆82Updated 3 weeks ago
- Tutorial for how to build BERT from scratch☆81Updated 3 months ago