TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
☆15Feb 12, 2024Updated 2 years ago
Alternatives and similar repositories for TensorRT-LLM
Users that are interested in TensorRT-LLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.☆49Oct 2, 2023Updated 2 years ago
- Specification of AluVM (algorithmic logic unit VM), its bytecode and assembly language☆13Feb 20, 2024Updated 2 years ago
- ☆14Jun 19, 2022Updated 3 years ago
- EventSource "polyfill" with custom headers☆11Feb 7, 2019Updated 7 years ago
- Inference with YOLOv5, OpenCV 4.5.4 DNN, C++, ROS and Python☆13Feb 12, 2023Updated 3 years ago
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Zeebe CLI via NPM☆11Apr 10, 2025Updated 11 months ago
- A Kafkajs wrapper for better handling parallel manual commits and back-pressure☆13Mar 28, 2019Updated 6 years ago
- ☆16Oct 4, 2023Updated 2 years ago
- This repository contains the cpp code to run YoloV8 with bytetrack tracker usinng tensorrt library☆11Mar 28, 2023Updated 2 years ago
- Repository for all the MATLAB and Simulink files for auto-tuning of PID using Q Learning for a quadrotor☆14Feb 5, 2021Updated 5 years ago
- 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.☆29Nov 17, 2025Updated 4 months ago
- Haskell bindings for libNVVM☆20Apr 1, 2014Updated 11 years ago
- Vector math and other CUDA helper functions for OptiX kernels☆10Oct 21, 2024Updated last year
- Web Debugging Utility. Fiddler Classic is the Original and Free Web Debugging Proxy Tool Exclusively for Windows. The community-trusted f…☆19May 9, 2022Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- High-speed Deep learning API Server with Libtorch (C++) and Gin (Golang)☆17Jun 8, 2019Updated 6 years ago
- ☆16May 23, 2024Updated last year
- Sigmpilot is an online demo for Sigma embedded engineering clients representing our skills in autonomous robotics, sensor fusion and simu…☆27Jan 9, 2023Updated 3 years ago
- Adaptive PID control of a flexible manipulator using DDPG algorithm.☆28Sep 4, 2025Updated 6 months ago
- ARCV2.0 updated the package with ARKit 2.0☆11Feb 24, 2019Updated 7 years ago
- yolov3 ros node using tensorrt acceleration☆14Jan 27, 2023Updated 3 years ago
- Personal finance predictions using machine learning.☆14Jan 14, 2019Updated 7 years ago
- Provides a demo of micro-ROS based on a Crazyflie.☆20Oct 18, 2021Updated 4 years ago
- Generates a QGIS qml file containing the closest match for a MapInfo style.☆19Oct 31, 2011Updated 14 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A demo to show how to share a C++ library between ROS 1 and ROS 2☆21Jan 30, 2020Updated 6 years ago
- An ONScripter plugin for JoiPlay based on OnscripterYuri☆10Jun 25, 2023Updated 2 years ago
- Sugarcoat 🍬 is a meta-framework that provides a whole lot of syntactic sugar for creating event-driven multinode systems in ROS2, using …☆36Updated this week
- Java版日レセクライアント☆12Jan 4, 2022Updated 4 years ago
- GiveCoin 2.0☆11Dec 4, 2017Updated 8 years ago
- The OpenSMILES specification☆23Jun 17, 2025Updated 9 months ago
- A Kubernetes Operator for MongoDB Atlas: https://www.mongodb.com/cloud/atlas☆14Oct 29, 2020Updated 5 years ago
- Starter Kit project with sample Amazon Echo skill created using Alexia Framework☆17Feb 21, 2017Updated 9 years ago
- GeoPlan-bench is a benchmark platform for evaluating agents in remote sensing task planning. The platform provides a complete workflow fo…☆22Dec 10, 2025Updated 3 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- ☆20Jun 30, 2021Updated 4 years ago
- Utility for modders & mod users to help resolve conflicts between mods for The Witcher 3.☆10Mar 16, 2022Updated 4 years ago
- mapboxGL echartLayer☆11Jun 16, 2022Updated 3 years ago
- ☆45Mar 18, 2026Updated last week
- Autonomous Traversal and Object Detection for Rovers☆15Mar 16, 2026Updated last week
- Tensor library for machine learning☆17Jul 13, 2023Updated 2 years ago
- An OptiX/CUDA code sample showing how to quickly build ray-tracing acceleration structures for dynamic subdivision surfaces☆24Feb 12, 2026Updated last month