hkproj / pytorch-transformer-distributed
Distributed training (multi-node) of a Transformer model
☆42Updated 7 months ago
Related projects ⓘ
Alternatives and complementary repositories for pytorch-transformer-distributed
- LoRA and DoRA from Scratch Implementations☆188Updated 8 months ago
- End-to-End LLM Guide☆97Updated 4 months ago
- Complete implementation of Llama2 with/without KV cache & inference 🚀☆47Updated 5 months ago
- A set of scripts and notebooks on LLM finetunning and dataset creation☆92Updated last month
- From scratch implementation of a vision language model in pure PyTorch☆161Updated 6 months ago
- Prune transformer layers☆63Updated 5 months ago
- Reference implementation of Mistral AI 7B v0.1 model.☆27Updated 10 months ago
- Starter pack for NeurIPS LLM Efficiency Challenge 2023.☆115Updated last year
- LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch☆82Updated last year
- LLM_library is a comprehensive repository serves as a one-stop resource hands-on code, insightful summaries.☆68Updated 10 months ago
- ☆133Updated 9 months ago
- This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…☆85Updated last year
- This repository contains the code for dataset curation and finetuning of instruct variant of the Bilingual OpenHathi model. The resultin…☆23Updated 10 months ago
- ☆86Updated 9 months ago
- Code for NeurIPS LLM Efficiency Challenge☆53Updated 7 months ago
- Notes on quantization in neural networks☆57Updated 10 months ago
- Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…☆48Updated 4 months ago
- Small and Efficient Mathematical Reasoning LLMs☆71Updated 9 months ago
- ☆111Updated last month
- RAGs: Simple implementations of Retrieval Augmented Generation (RAG) Systems☆83Updated 6 months ago
- Set of scripts to finetune LLMs☆36Updated 7 months ago
- ☆116Updated 2 months ago
- experiments with inference on llama☆105Updated 5 months ago
- LLaMA 2 implemented from scratch in PyTorch☆250Updated last year
- ☆131Updated 3 months ago
- Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)☆113Updated 5 months ago
- Welcome to the LLMs Interview Prep Guide! This GitHub repository offers a curated set of interview questions and answers tailored for Dat…☆96Updated 9 months ago
- Direct Preference Optimization Implementation☆14Updated 9 months ago
- Project 2 (Building Large Language Models) for Stanford CS324: Understanding and Developing Large Language Models (Winter 2022)☆101Updated last year
- A pipeline for LLM knowledge distillation☆77Updated 3 months ago