cat-state / tinyparLinks
β20Updated 2 years ago
Alternatives and similar repositories for tinypar
Users that are interested in tinypar are comparing it to the libraries listed below
Sorting:
- Large scale 4D parallelism pre-training for π€ transformers in Mixture of Experts *(still work in progress)*β87Updated last year
- some common Huggingface transformers in maximal update parametrization (Β΅P)β86Updated 3 years ago
- β50Updated last year
- β91Updated last year
- HomebrewNLP in JAX flavour for maintable TPU-Trainingβ51Updated last year
- Fast, Modern, and Low Precision PyTorch Optimizersβ116Updated last month
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limitβ62Updated 2 years ago
- Experiments for efforts to train a new and improved t5β75Updated last year
- β62Updated 3 years ago
- Exploring finetuning public checkpoints on filter 8K sequences on Pileβ115Updated 2 years ago
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT trainingβ132Updated last year
- An implementation of the Llama architecture, to instruct and delightβ21Updated 4 months ago
- β121Updated last year
- A set of Python scripts that makes your experience on TPU betterβ54Updated last month
- Experiment of using Tangent to autodiff tritonβ80Updated last year
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" givenβ¦β14Updated 2 years ago
- Simple and efficient pytorch-native transformer training and inference (batched)β78Updated last year
- Code repository for the c-BTM paperβ107Updated 2 years ago
- A place to store reusable transformer components of my own creation or found on the interwebsβ60Updated last week
- β19Updated 5 months ago
- β53Updated last year
- A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.β69Updated last year
- Experiments with generating opensource language model assistantsβ97Updated 2 years ago
- β53Updated last year
- Triton Implementation of HyperAttention Algorithmβ48Updated last year
- Automatically take good care of your preemptible TPUsβ37Updated 2 years ago
- JAX implementation of the Llama 2 modelβ216Updated last year
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023β136Updated last year
- Scaling is a distributed training library and installable dependency designed to scale up neural networks, with a dedicated module for trβ¦β64Updated 3 weeks ago
- Repository for "I am a Strange Dataset: Metalinguistic Tests for Language Models"β44Updated last year