hkproj / pytorch-lora
LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch
☆82Updated last year
Related projects ⓘ
Alternatives and complementary repositories for pytorch-lora
- LLaMA 2 implemented from scratch in PyTorch☆258Updated last year
- Reference implementation of Mistral AI 7B v0.1 model.☆27Updated 10 months ago
- LoRA and DoRA from Scratch Implementations☆188Updated 8 months ago
- A set of scripts and notebooks on LLM finetunning and dataset creation☆93Updated last month
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆47Updated 6 months ago
- Notes about "Attention is all you need" video (https://www.youtube.com/watch?v=bCz4OMemCcA)☆227Updated last year
- This is the code that went into our practical dive using mamba as information extraction☆50Updated 11 months ago
- Distributed training (multi-node) of a Transformer model☆43Updated 7 months ago
- This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…☆86Updated last year
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆108Updated 5 months ago
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆143Updated last week
- A Simplified PyTorch Implementation of Vision Transformer (ViT)☆142Updated 5 months ago
- From scratch implementation of a vision language model in pure PyTorch☆163Updated 6 months ago
- ☆46Updated last week
- LLaMA 3 is one of the most promising open-source model after Mistral, we will recreate it's architecture in a simpler manner.☆104Updated 3 months ago
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆50Updated 7 months ago
- Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling☆170Updated last week
- Set of scripts to finetune LLMs☆36Updated 7 months ago
- ML/DL Math and Method notes☆57Updated 11 months ago
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆232Updated this week
- Starter pack for NeurIPS LLM Efficiency Challenge 2023.☆118Updated last year
- ☆200Updated 4 months ago
- This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT)…☆54Updated last year
- RAGs: Simple implementations of Retrieval Augmented Generation (RAG) Systems☆83Updated 7 months ago
- ☆32Updated 5 months ago
- Conference schedule, top papers, and analysis of the data for NeurIPS 2023!☆108Updated 11 months ago
- Notes on the Mamba and the S4 model (Mamba: Linear-Time Sequence Modeling with Selective State Spaces)☆149Updated 10 months ago
- a simplified version of Google's Gemma model to be used for learning☆23Updated 8 months ago
- A family of compressed models obtained via pruning and knowledge distillation☆285Updated last week
- Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget☆132Updated 7 months ago