rwitten / HighPerfLLMs2024
☆223Updated 3 months ago
Related projects ⓘ
Alternatives and complementary repositories for HighPerfLLMs2024
- Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimenta…☆456Updated last week
- ☆197Updated 3 months ago
- seqax = sequence modeling + JAX☆132Updated 3 months ago
- Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax☆516Updated this week
- JAX implementation of the Llama 2 model☆210Updated 9 months ago
- ☆177Updated this week
- JAX-Toolbox☆241Updated this week
- jax-triton contains integrations between JAX and OpenAI Triton☆340Updated last week
- ☆140Updated this week
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆479Updated 2 weeks ago
- ☆262Updated this week
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆228Updated this week
- Inference code for LLaMA models in JAX☆112Updated 5 months ago
- Accelerated First Order Parallel Associative Scan☆162Updated 2 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆190Updated 2 weeks ago
- Train very large language models in Jax.☆195Updated last year
- A simple library for scaling up JAX programs☆125Updated last week
- Cataloging released Triton kernels.☆132Updated 2 months ago
- ☆121Updated this week
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆95Updated last year
- Orbax provides common checkpointing and persistence utilities for JAX users☆296Updated this week
- Cost aware hyperparameter tuning algorithm☆119Updated 4 months ago
- This repository contains the experimental PyTorch native float8 training UX☆211Updated 3 months ago
- Puzzles for exploring transformers☆323Updated last year
- ☆296Updated this week
- JAX Synergistic Memory Inspector☆163Updated 3 months ago
- Building blocks for foundation models.☆386Updated 10 months ago
- Named Tensors for Legible Deep Learning in JAX☆153Updated this week
- A set of Python scripts that makes your experience on TPU better☆40Updated 4 months ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆163Updated this week