This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows instead of cloud.
☆127Feb 29, 2024Updated 2 years ago
Alternatives and similar repositories for trt-llm-as-openai-windows
Users that are interested in trt-llm-as-openai-windows are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- OpenAI compatible API for TensorRT LLM triton backend☆221Aug 1, 2024Updated 2 years ago
- A developer reference project for creating Retrieval Augmented Generation (RAG) chatbots on Windows using TensorRT-LLM☆3,121Jan 21, 2026Updated 7 months ago
- ☆13Feb 18, 2024Updated 2 years ago
- Flask API for generating text embeddings using OpenAI or sentence_transformers☆14Sep 1, 2023Updated 2 years ago
- An OpenAI API compatible images server to generate or manipulate images.☆18Feb 2, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- ☆25Feb 18, 2024Updated 2 years ago
- Text-Guided Generation of Full-Body Image with Preserved Reference Face for Customized Animation☆24Jun 24, 2024Updated 2 years ago
- The Triton TensorRT-LLM Backend☆942Aug 17, 2026Updated last week
- ☆30Aug 21, 2024Updated 2 years ago
- ☆14Feb 18, 2023Updated 3 years ago
- ☆24Mar 10, 2025Updated last year
- ☆16Nov 22, 2024Updated last year
- ☆12Sep 28, 2024Updated last year
- Run Vision LLMs, TTS and STT APIs. Website and API for https://text-generator.io☆38Jul 16, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- experiments with inference on llama☆103Jun 6, 2024Updated 2 years ago
- 百度QA100万数据集☆47Nov 30, 2023Updated 2 years ago
- This is the repo with the code to conduct a comparative analysis of different audio representation models.☆11Aug 31, 2023Updated 3 years ago
- A one-stop library to standardize the inference and evaluation of all the conditional video generation models.☆50Feb 13, 2025Updated last year
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆58May 3, 2026Updated 3 months ago
- Toonification of real face images using PyTorch, Stylegan2 and Image-to-Image translation☆13Jun 14, 2022Updated 4 years ago
- The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…☆186Nov 24, 2025Updated 9 months ago
- HunyuanVideo: A Systematic Framework For Large Video Generation Model☆47Dec 14, 2024Updated last year
- ☆20Jan 17, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM☆26Feb 26, 2024Updated 2 years ago
- ☆12Feb 22, 2023Updated 3 years ago
- Render human motion sequences with Blender, with or without overlaying on a video☆17Jul 23, 2026Updated last month
- Safely push a Cog model version by making sure it works and is backwards-compatible with previous versions.☆17Dec 4, 2025Updated 8 months ago
- An end to end ML project. Using MLflow for experiment tracking and model registry. Prefect for workflow orchestration. S3 for artifacts s…☆12Sep 11, 2022Updated 3 years ago
- Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification☆11Aug 12, 2023Updated 3 years ago
- Annotation builder to use segmentation in Mask_RCNN, even if your annotations are rectangular instead of polygon.☆15Feb 16, 2022Updated 4 years ago
- [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models☆23Mar 15, 2024Updated 2 years ago
- A script for MIDI/AUDIO routing in Max for Live devices.☆12Dec 27, 2024Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Experiment with NVIDIA Triton and Whisper☆15Apr 29, 2024Updated 2 years ago
- This is the code for reproducing the TABBIE baseline in our paper: "Retrieval-Based Transformer for Table Augmentation"☆12Sep 17, 2025Updated 11 months ago
- LLM-Training-API: Including Embeddings & ReRankers, mergekit, LaserRMT☆27Feb 18, 2024Updated 2 years ago
- This repository contains the experimental PyTorch native float8 training UX☆226Aug 1, 2024Updated 2 years ago
- Probabilistic LLM evaluations. [CogSci2023; ACL2023]☆73Jul 27, 2024Updated 2 years ago
- ☆32Dec 3, 2024Updated last year
- For converting LLM datasets from one format into another.☆22Nov 12, 2025Updated 9 months ago