TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
☆15Feb 12, 2024Updated 2 years ago
Alternatives and similar repositories for TensorRT-LLM
Users that are interested in TensorRT-LLM are comparing it to the libraries listed below
Sorting:
- An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.☆48Oct 2, 2023Updated 2 years ago
- Vecna is a Python chatbot which recommends songs and movies depending upon your feelings☆12Jun 28, 2022Updated 3 years ago
- ☆12Jan 7, 2023Updated 3 years ago
- 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.☆29Nov 17, 2025Updated 3 months ago
- Efficient-alpr-unconstrained☆31May 1, 2023Updated 2 years ago
- Autonomous Traversal and Object Detection for Rovers☆15Feb 26, 2026Updated last week
- Belief Revision based Caption Re-ranker with Visual Semantic Information. COLING 2022☆11Apr 13, 2025Updated 10 months ago
- ☆10Jul 29, 2022Updated 3 years ago
- Extract information from XBRL files in the ESEF format☆13Jan 3, 2026Updated 2 months ago
- We archive data because we are interested in the diffs. All data is from https://video-api.cartoonnetwork.com. We run the check every min…☆10Updated this week
- This project predicts wind turbine failure using numerous sensor data by applying classification based ML models that improves prediction…☆11Mar 20, 2023Updated 2 years ago
- 最稳定的ETH/ETC矿池代理，go语言编写，超高性能多线程多并发，系统占用极小。支持SSL/TCP代理，自定义抽水地址和比例。开发者费用恒定千分之一，保证用户抽水利益的最大化！可设置端口连接数，免疫一切CC攻击。一键脚本，快速上手，轻松搞定☆12Apr 3, 2022Updated 3 years ago
- Vector search with Pinecone and Openai to search through contract law textbook. If downloaded, remeber to install all dependencies. Refer…☆13Mar 30, 2023Updated 2 years ago
- Code/Report/Image Plagiarism finder☆12Apr 1, 2025Updated 11 months ago
- Self-Supervised MRI Reconstruction☆10May 25, 2021Updated 4 years ago
- This Repo contains a fully functional API ready application for delineating fields for smart farming platform☆15Jan 20, 2023Updated 3 years ago
- Deep metric learning: Triplet, Magnet and VMF loss☆11Aug 19, 2022Updated 3 years ago
- Unity share plugins for iOS and Android with sources.☆11Jul 19, 2016Updated 9 years ago
- generate video with voice narration from ppt/pdf Slides☆10Sep 4, 2023Updated 2 years ago
- The GitHub open source software repository on interpreting super-resolution CNNs for sub-pixel motion compensation in video coding☆11May 20, 2022Updated 3 years ago
- Automated Question-Answering Over Knowledge Graphs in O&M of Wind Turbines☆12Aug 16, 2022Updated 3 years ago
- Documentation and code for predictive maintenance data and assess scripts.☆11Jun 8, 2023Updated 2 years ago
- EEG-based Major Depression Disorder Recognition using Swin Transformers☆10Jun 23, 2024Updated last year
- Voila! A smart automatic pet feeder using Arduino Uno + RTC time module for scheduling + multiple sensors.☆10Jun 4, 2024Updated last year
- Virtual tour in a digital twin of Sabae city in Japan☆13Sep 9, 2021Updated 4 years ago
- Source and documentation for development of autopilot for a surface vessel☆15Jun 3, 2015Updated 10 years ago
- A simple GPT-3 interface to automate core legal writing tasks☆12Mar 8, 2023Updated 2 years ago
- 该仓库是 BUPT 智能系统实验室的法律大模型项目，基于 ChatGLM 等开源大模型进行实现。☆11Nov 28, 2023Updated 2 years ago
- Scraping LegiFrance naturalisation decrees for fun and OSINT profit☆11May 27, 2023Updated 2 years ago
- ☆14Jun 19, 2022Updated 3 years ago
- A reddit scraping and analysis bot to visualize linguistic and content trends☆12Oct 5, 2021Updated 4 years ago
- "SSPNet: An interpretable 3D-CNN for classification of schizophrenia using phase maps of resting-state complex-valued fMRI data," publish…☆10May 13, 2022Updated 3 years ago
- Generates a QGIS qml file containing the closest match for a MapInfo style.☆19Oct 31, 2011Updated 14 years ago
- WindTurbineHighSpeedBearingPrognosis-Data☆10Aug 19, 2020Updated 5 years ago
- GeoPlan-bench is a benchmark platform for evaluating agents in remote sensing task planning. The platform provides a complete workflow fo…☆20Dec 10, 2025Updated 2 months ago
- Tally Prime MCP (Model Context Protocol) Server implementation to feed Tally ERP data to popular LLM like Claude, ChatGPT supporting MCP☆19Nov 11, 2025Updated 3 months ago
- Official Pytorch Implementation for Continual Learning For On-Device Environmental Sound Classification☆14Jul 19, 2022Updated 3 years ago
- 基于go+gin+flutter 后台管理系统，支持用户管理，认证，内容管理等☆13Oct 11, 2024Updated last year
- Pytorch Implementation of the Explainable Conditional Adversarial Autoencoder using Saliency Maps and SHAP (J. of Imaging - MDPI)☆12Mar 5, 2025Updated 11 months ago