tenstorrent / vllmLinks
A high-throughput and memory-efficient inference and serving engine for LLMs
☆18Updated this week
Alternatives and similar repositories for vllm
Users that are interested in vllm are comparing it to the libraries listed below
Sorting:
- Tenstorrent TT-BUDA Repository☆314Updated 3 months ago
- Tenstorrent MLIR compiler☆151Updated this week
- Tenstorrent Kernel Module☆46Updated this week
- TVM for Tenstorrent ASICs☆23Updated last week
- The TT-Forge FE is a graph compiler designed to optimize and transform computational graphs for deep learning models, enhancing their per…☆46Updated this week
- Buda Compiler Backend for Tenstorrent devices☆29Updated 3 months ago
- Tenstorrent Firmware repository☆15Updated 2 weeks ago
- An experimental CPU backend for Triton☆135Updated last month
- Tenstorrent's MLIR Based Compiler. We aim to enable developers to run AI on all configurations of Tenstorrent hardware, through an open-s…☆80Updated this week
- TT-NN operator library, and TT-Metalium low level kernel programming model.☆1,002Updated this week
- ⭐️ TTNN Compiler for PyTorch 2 ⭐️ Enables running PyTorch models on Tenstorrent hardware using eager or compile path☆52Updated this week
- IREE's PyTorch Frontend, based on Torch Dynamo.☆93Updated this week
- OpenAI Triton backend for Intel® GPUs☆193Updated this week
- QuickReduce is a performant all-reduce library designed for AMD ROCm that supports inline compression.☆30Updated 4 months ago
- Evaluating Large Language Models for CUDA Code Generation ComputeEval is a framework designed to generate and evaluate CUDA code from Lar…☆53Updated last month
- Development repository for the Triton language and compiler☆125Updated this week
- Repository of model demos using TT-Buda☆62Updated 3 months ago
- AI Tensor Engine for ROCm☆232Updated this week
- Experimental projects related to TensorRT☆107Updated this week
- ☆40Updated this week
- Shared Middle-Layer for Triton Compilation☆258Updated this week
- Tenstorrent console based hardware information program☆47Updated last week
- Composable Kernel: Performance Portable Programming Model for Machine Learning Tensor Operators☆437Updated this week
- Tenstorrent Firmware Update Utility☆5Updated this week
- PyTorch emulation library for Microscaling (MX)-compatible data formats☆257Updated last month
- Backward compatible ML compute opset inspired by HLO/MHLO☆501Updated this week
- ☆223Updated last year
- ☆20Updated this week
- Repo for AI Compiler team. The intended purpose of this repo is for implementation of a PJRT device.☆18Updated this week
- BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.☆651Updated last week