jsbaan / transformer-from-scratch

Related projects: ⓘ

• hkproj / pytorch-llama
  LLaMA 2 implemented from scratch in PyTorch
  ☆216Updated 11 months ago
• tintn / vision-transformer-from-scratch
  A Simplified PyTorch Implementation of Vision Transformer (ViT)
  ☆123Updated 3 months ago
• srush / annotated-mamba
  Annotated version of the Mamba paper
  ☆445Updated 6 months ago
• llm-efficiency-challenge / neurips_llm_efficiency_challenge
  NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day
  ☆248Updated 10 months ago
• coaxsoft / pytorch_bert
  Tutorial for how to build BERT from scratch
  ☆81Updated 3 months ago
• srush / Transformer-Puzzles
  Puzzles for exploring transformers
  ☆293Updated last year
• gautierdag / bpeasy
  Fast bare-bones BPE for modern tokenizer training
  ☆138Updated 3 weeks ago
• linjames0 / Transformer-CUDA
  An implementation of the transformer architecture onto an Nvidia CUDA kernel
  ☆152Updated 11 months ago
• tcapelle / llm_recipes
  A set of scripts and notebooks on LLM finetunning and dataset creation
  ☆89Updated last week
• RAIVNLab / MRL
  Code repository for the paper - "Matryoshka Representation Learning"
  ☆398Updated 7 months ago
• cuda-mode / profiling-cuda-in-torch
  ☆124Updated 7 months ago
• KellerJordan / modded-nanogpt
  GPT-2 (124M) quality in 5B tokens
  ☆227Updated last week
• inseq-team / inseq
  Interpretability for sequence generation models 🐛 🔍
  ☆361Updated 3 weeks ago
• lucasdelimanogueira / PyNorch
  Recreating PyTorch from scratch (C/C++, CUDA and Python, with multi-GPU support and automatic differentiation!)
  ☆89Updated 3 months ago
• huggingface / large_language_model_training_playbook
  An open collection of implementation tips, tricks and resources for training large language models
  ☆455Updated last year
• dpressel / mint
  MinT: Minimal Transformer Library and Tutorials
  ☆247Updated 2 years ago
• rasbt / faster-pytorch-blog
  Outlining techniques for improving the training performance of your PyTorch model without compromising its accuracy
  ☆124Updated last year
• Khaliladib11 / Transformer-from-scratch
  I will build Transformer from scratch
  ☆48Updated 4 months ago
• srush / raspy
  An interactive exploration of Transformer programming.
  ☆243Updated 10 months ago
• markriedl / transformer-walkthrough
  A walkthrough of transformer architecture code
  ☆295Updated 7 months ago
• fkodom / transformer-from-scratch
  Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch
  ☆90Updated last year
• bkitano / llama-from-scratch
  Llama from scratch, or How to implement a paper without crying
  ☆499Updated 3 months ago
• srush / do-we-need-attention
  ☆160Updated last year
• kvfrans / jax-diffusion-transformer
  Implementation of Diffusion Transformer (DiT) in JAX
  ☆246Updated 3 months ago
• srush / Autodiff-Puzzles
  ☆325Updated 11 months ago
• srush / annotated-s4
  Implementation of https://srush.github.io/annotated-s4
  ☆457Updated last year
• sangmichaelxie / cs324_p2
  Project 2 (Building Large Language Models) for Stanford CS324: Understanding and Developing Large Language Models (Winter 2022)
  ☆101Updated last year
• HazyResearch / aisys-building-blocks
  Building blocks for foundation models.
  ☆347Updated 8 months ago
• CG80499 / KAN-GPT-2
  Training small GPT-2 style models using Kolmogorov-Arnold networks.
  ☆105Updated 3 months ago
• HazyResearch / m2
  Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"
  ☆530Updated 4 months ago