mistralai / TensorRT-LLMLinks
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
☆15Updated last year
Alternatives and similar repositories for TensorRT-LLM
Users that are interested in TensorRT-LLM are comparing it to the libraries listed below
Sorting:
- Code for blog posts from OpenCV.AI☆16Updated 2 years ago
- The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained mode…☆12Updated last year
- Simple CogVLM client script☆14Updated 2 years ago
- Deploy DL/ ML inference pipelines with minimal extra code.☆102Updated last year
- RAG-QA is a free, containerised question-answer framework that allows you to ask questions to your documents in an intuitive way☆19Updated 2 years ago
- Demo python script app to interact with llama.cpp server using whisper API, microphone and webcam devices.☆46Updated 2 years ago
- BUD-E (Buddy) is an open-source voice assistant framework that facilitates seamless interaction with AI models and APIs, enabling the cre…☆22Updated last year
- Video chat apps with computer vision filters built on top of Streamlit☆50Updated 2 years ago
- A pipeline to generate user-preferred photo-realistic avatars using stable-diffusion and bayesian-optimization.☆18Updated 8 months ago
- Finetune any model on HF in less than 30 seconds☆56Updated last week
- GGUF Quantization of any LLM.☆41Updated last year
- ☆15Updated 2 weeks ago
- ☆12Updated 8 months ago
- Interactive Textbook Demo☆53Updated 3 months ago
- Speech to Speech conversation using the OpenAI RealTime API in Python 🐍☆26Updated last year
- A ⚡️ Lightning.ai ⚡️ app demo for Voice based web search using OpenAI's Whisper and DuckDuckGo☆27Updated 3 years ago
- Using GPT-3 and Carrot (GPT-3 for computer vision) to create detailed descriptions of images.☆14Updated 3 years ago
- Segmenting a given document using recursive xy-cut algorithm.☆12Updated 7 years ago
- A Python tool to solve logic games with AI, Deep Learning and Computer Vision☆17Updated 5 years ago
- Streamlit app to Translate text to or between 50 languages with mBART-50 from Huggingface and Facebook☆25Updated 4 years ago
- Simple demo project with OpenAI's API and TTS☆15Updated 2 years ago
- Computer Vision Helping Library☆50Updated last year
- The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…☆181Updated 2 months ago
- Simple and easy stable diffusion inference with LightningModule on GPU, CPU and MPS (Possibly all devices supported by Lightning).☆17Updated 2 years ago
- Automatic defect recognition in X-ray testing using computer vision☆12Updated 7 years ago
- Passively collect images for computer vision datasets on the edge.☆35Updated 2 years ago
- Vision-based heart rate estimation using OAK-D camera☆33Updated 4 years ago
- H&M Fashion Image similarity search with Weaviate and DocArray☆43Updated last year
- 本项目是关于Yi的多模态系列模型，如Yi-VL-6B/34B等的实验与应用。☆14Updated 2 years ago
- ☆13Updated last year