jsbaan / transformer-from-scratchLinks
Well documented, unit tested, type checked and formatted implementation of a vanilla transformer - for educational purposes.
☆250Updated last year
Alternatives and similar repositories for transformer-from-scratch
Users that are interested in transformer-from-scratch are comparing it to the libraries listed below
Sorting:
- Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch☆93Updated last year
- Tutorial for how to build BERT from scratch☆93Updated last year
- Annotated version of the Mamba paper☆482Updated last year
- A Simplified PyTorch Implementation of Vision Transformer (ViT)☆186Updated 11 months ago
- I will build Transformer from scratch☆69Updated last year
- LLaMA 2 implemented from scratch in PyTorch☆328Updated last year
- Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)☆151Updated 11 months ago
- Best practices & guides on how to write distributed pytorch training code