AI-Hypercomputer / ray-tpuLinks
☆15Updated 5 months ago
Alternatives and similar repositories for ray-tpu
Users that are interested in ray-tpu are comparing it to the libraries listed below
Sorting:
- torchprime is a reference model implementation for PyTorch on TPU.☆39Updated last week
- Blazing fast data loading with HuggingFace Dataset and Ray Data☆16Updated last year
- ☆14Updated 5 months ago
- Experimental playground for benchmarking language model (LM) architectures, layers, and tricks on smaller datasets. Designed for flexible…☆83Updated last week
- ☆20Updated 2 years ago
- Randomized Positional Encodings Boost Length Generalization of Transformers☆82Updated last year
- ☆121Updated last year
- Official code release for "SuperBPE: Space Travel for Language Models"☆75Updated 2 weeks ago
- Fast, Modern, and Low Precision PyTorch Optimizers☆116Updated last month
- ☆62Updated 3 years ago
- ☆15Updated last year
- Two implementations of ZeRO-1 optimizer sharding in JAX☆14Updated 2 years ago
- A byte-level decoder architecture that matches the performance of tokenized Transformers.☆66Updated last year
- Machine Learning eXperiment Utilities☆46Updated 3 months ago
- ☆21Updated 7 months ago
- some common Huggingface transformers in maximal update parametrization (µP)☆86Updated 3 years ago
- A toolkit for scaling law research ⚖☆52Updated 9 months ago
- DPO, but faster 🚀☆45Updated 10 months ago
- ☆21Updated 11 months ago
- A library for unit scaling in PyTorch☆132Updated 3 months ago
- A fast implementation of T5/UL2 in PyTorch using Flash Attention☆107Updated 7 months ago
- Various transformers for FSDP research☆38Updated 2 years ago
- Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…☆52Updated 2 years ago
- MiSS is a novel PEFT method that features a low-rank structure but introduces a new update mechanism distinct from LoRA, achieving an exc…☆25Updated this week
- A place to store reusable transformer components of my own creation or found on the interwebs☆60Updated last week
- A set of Python scripts that makes your experience on TPU better☆54Updated last month
- Transformer with Mu-Parameterization, implemented in Jax/Flax. Supports FSDP on TPU pods.☆32Updated 4 months ago
- ☆190Updated this week
- Pytorch/XLA SPMD Test code in Google TPU☆23Updated last year
- ☆48Updated last year