AI-Hypercomputer / JetStream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
☆232Updated this week
Related projects ⓘ
Alternatives and complementary repositories for JetStream
- ☆122Updated this week
- PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"☆40Updated last week
- xpk (Accelerated Processing Kit, pronounced x-p-k,) is a software tool to help Cloud developers to orchestrate training jobs on accelerat…☆81Updated this week
- Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimenta…☆457Updated last week
- Google TPU optimizations for transformers models☆75Updated this week
- ☆224Updated 4 months ago
- ☆156Updated last week
- ☆178Updated last week
- Module, Model, and Tensor Serialization/Deserialization☆188Updated last month
- ☆267Updated this week
- Testing framework for Deep Learning models (Tensorflow and PyTorch) on Google Cloud hardware accelerators (TPU and GPU)☆64Updated 2 months ago
- CUDA checkpoint and restore utility☆224Updated 7 months ago
- ☆271Updated 3 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆253Updated last month
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆193Updated this week
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆153Updated this week
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆165Updated this week
- seqax = sequence modeling + JAX☆133Updated 4 months ago
- ☆99Updated last month
- ☆119Updated this week
- This repository contains the experimental PyTorch native float8 training UX☆211Updated 3 months ago
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆257Updated this week
- A library to analyze PyTorch traces.☆307Updated this week
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆146Updated 2 weeks ago
- JAX-Toolbox☆245Updated this week
- Applied AI experiments and examples for PyTorch☆166Updated 3 weeks ago
- JAX implementation of the Llama 2 model☆210Updated 9 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆57Updated last month
- TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…☆332Updated this week
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆685Updated this week