vllm-project / tpu-inferenceLinks
TPU inference for vLLM, with unified JAX and PyTorch support.
☆223Updated this week
Alternatives and similar repositories for tpu-inference
Users that are interested in tpu-inference are comparing it to the libraries listed below
Sorting:
- ring-attention experiments☆165Updated last year
- Fast low-bit matmul kernels in Triton☆424Updated this week
- Applied AI experiments and examples for PyTorch☆314Updated 5 months ago
- Ship correct and fast LLM kernels to PyTorch☆139Updated 2 weeks ago
- ☆277Updated this week
- Accelerating MoE with IO and Tile-aware Optimizations☆563Updated 2 weeks ago
- Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.☆319Updated this week
- Cataloging released Triton kernels.☆291Updated 4 months ago
- extensible collectives library in triton☆93Updated 10 months ago
- Collection of kernels written in Triton language☆175Updated 9 months ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆219Updated this week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆279Updated 2 months ago
- PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"☆79Updated last month
- Load compute kernels from the Hub☆389Updated this week
- JAX backend for SGL☆232Updated this week
- torchcomms: a modern PyTorch communications API☆323Updated this week
- ☆102Updated last year
- PyTorch bindings for CUTLASS grouped GEMM.☆141Updated 8 months ago
- A curated collection of resources, tutorials, and best practices for learning and mastering NVIDIA CUTLASS☆249Updated 8 months ago
- Autonomous GPU Kernel Generation via Deep Agents☆223Updated this week
- This repository contains the experimental PyTorch native float8 training UX☆227Updated last year
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆205Updated last week
- A Quirky Assortment of CuTe Kernels☆772Updated last week
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆474Updated 3 weeks ago
- Boosting 4-bit inference kernels with 2:4 Sparsity☆93Updated last year
- QuTLASS: CUTLASS-Powered Quantized BLAS for Deep Learning☆165Updated 2 months ago
- Nsight Python is a Python kernel profiling interface based on NVIDIA Nsight Tools☆99Updated this week
- Perplexity GPU Kernels☆554Updated 2 months ago
- Triton-based Symmetric Memory operators and examples☆80Updated 2 weeks ago
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆732Updated this week