cat-state / tinyparLinks
β20Updated 2 years ago
Alternatives and similar repositories for tinypar
Users that are interested in tinypar are comparing it to the libraries listed below
Sorting:
- Large scale 4D parallelism pre-training for π€ transformers in Mixture of Experts *(still work in progress)*β86Updated last year
- β83Updated last year
- A set of Python scripts that makes your experience on TPU betterβ54Updated last year
- β114Updated last year
- some common Huggingface transformers in maximal update parametrization (Β΅P)β82Updated 3 years ago
- Fast, Modern, and Low Precision PyTorch Optimizersβ103Updated last week
- β49Updated last year
- An implementation of the Llama architecture, to instruct and delightβ21Updated 2 months ago
- HomebrewNLP in JAX flavour for maintable TPU-Trainingβ50Updated last year
- A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.β70Updated last year
- β19Updated 2 months ago
- Exploring finetuning public checkpoints on filter 8K sequences on Pileβ116Updated 2 years ago
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT trainingβ130Updated last year
- β53Updated last year
- Language models scale reliably with over-training and on downstream tasksβ97Updated last year
- β53Updated last year
- Experiments for efforts to train a new and improved t5β76Updated last year
- Triton Implementation of HyperAttention Algorithmβ48Updated last year
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS β¦β61Updated 9 months ago
- A toolkit for scaling law research ββ50Updated 6 months ago
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limitβ63Updated 2 years ago
- Easily run PyTorch on multiple GPUs & machinesβ46Updated last month
- Experiment of using Tangent to autodiff tritonβ79Updated last year
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" givenβ¦β14Updated last year
- β61Updated 3 years ago
- Engineering the state of RNN language models (Mamba, RWKV, etc.)β32Updated last year
- Automatically take good care of your preemptible TPUsβ36Updated 2 years ago
- β21Updated 8 months ago
- β34Updated 10 months ago
- Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.β19Updated last week