hkproj / pytorch-lora
LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
☆99Updated last year
Alternatives and similar repositories for pytorch-lora:
Users that are interested in pytorch-lora are comparing it to the libraries listed below
- LLaMA 2 implemented from scratch in PyTorch☆307Updated last year
- ☆136Updated 2 months ago
- LoRA and DoRA from Scratch Implementations☆198Updated last year
- Distributed training (multi-node) of a Transformer model☆59Updated 11 months ago
- Reference implementation of Mistral AI 7B v0.1 model.☆28Updated last year
- Notes about "Attention is all you need" video (https://www.youtube.com/watch?v=bCz4OMemCcA)☆257Updated last year
- Notes on quantization in neural networks☆77Updated last year
- From scratch implementation of a vision language model in pure PyTorch☆205Updated 10 months ago
- A Simplified PyTorch Implementation of Vision Transformer (ViT)☆169Updated 9 months ago
- Tutorial for how to build BERT from scratch☆90Updated 10 months ago
- ☆87Updated last year
- Prune transformer layers☆68Updated 9 months ago
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆54Updated 11 months ago
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆47Updated 10 months ago
- Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).☆114Updated 5 months ago
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆115Updated 9 months ago
- RAGs: Simple implementations of Retrieval Augmented Generation (RAG) Systems☆96Updated 2 months ago
- Deep learning library implemented from scratch in numpy. Mixtral, Mamba, LLaMA, GPT, ResNet, and other experiments.☆51Updated 11 months ago
- Unofficial implementation of https://arxiv.org/pdf/2407.14679☆44Updated 6 months ago
- This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…☆87Updated last year
- I will build Transformer from scratch☆61Updated 10 months ago
- Triton implementation of GPT/LLAMA☆16Updated 6 months ago
- This is the code that went into our practical dive using mamba as information extraction☆53Updated last year
- Notebooks for fine tuning pali gemma☆97Updated 2 months ago
- Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)☆163Updated last year
- Notes and commented code for RLHF (PPO)☆77Updated last year