This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows instead of cloud.
☆128Feb 29, 2024Updated 2 years ago
Alternatives and similar repositories for trt-llm-as-openai-windows
Users that are interested in trt-llm-as-openai-windows are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- OpenAI compatible API for TensorRT LLM triton backend☆221Aug 1, 2024Updated last year
- A developer reference project for creating Retrieval Augmented Generation (RAG) chatbots on Windows using TensorRT-LLM☆3,128Jan 21, 2026Updated 5 months ago
- ☆13Feb 18, 2024Updated 2 years ago
- The hackpack is a collection of resources to get started, and continue, your understanding of artificial intelligence.☆10Nov 1, 2018Updated 7 years ago
- Flask API for generating text embeddings using OpenAI or sentence_transformers☆14Sep 1, 2023Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- An OpenAI API compatible images server to generate or manipulate images.☆18Feb 2, 2025Updated last year
- ☆25Feb 18, 2024Updated 2 years ago
- GenAI Examples☆16Dec 13, 2024Updated last year
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆58Jul 13, 2026Updated last week
- Text-Guided Generation of Full-Body Image with Preserved Reference Face for Customized Animation☆24Jun 24, 2024Updated 2 years ago
- Research Rovers AI Research Assistants for NASA☆13May 17, 2024Updated 2 years ago
- ☆14Nov 22, 2024Updated last year
- The Triton TensorRT-LLM Backend☆939Updated this week
- Star Trek Game☆14Feb 16, 2021Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Wyoming protocol server that calls an external program to play audio☆19Apr 28, 2024Updated 2 years ago
- ☆24Mar 10, 2025Updated last year
- ☆11Sep 28, 2024Updated last year
- Run Vision LLMs, TTS and STT APIs. Website and API for https://text-generator.io☆38Updated this week
- ☆12May 23, 2024Updated 2 years ago
- A simple node to download repos from HF specify a repo ID or File create a folder where you want to download the files then rename the fo…☆25Jul 14, 2025Updated last year
- experiments with inference on llama☆103Jun 6, 2024Updated 2 years ago
- Offline version of the NanoPy Editor. No server needed.☆14Mar 7, 2025Updated last year
- FlashInfer: Kernel Library for LLM Serving (Windows build & kernels)☆15Jul 1, 2026Updated 2 weeks ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A one-stop library to standardize the inference and evaluation of all the conditional video generation models.☆50Feb 13, 2025Updated last year
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆58May 3, 2026Updated 2 months ago
- The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…☆185Nov 24, 2025Updated 7 months ago
- HunyuanVideo: A Systematic Framework For Large Video Generation Model☆47Dec 14, 2024Updated last year
- ☆19Jan 17, 2025Updated last year
- This is the official code for paper: [PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs]☆37Aug 31, 2024Updated last year
- 大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM☆27Feb 26, 2024Updated 2 years ago
- ComfyUI Dia text to speech☆14May 29, 2025Updated last year
- Capturing and Logging OpenVR☆12Mar 9, 2024Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification☆11Aug 12, 2023Updated 2 years ago
- [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models☆23Mar 15, 2024Updated 2 years ago
- An open-source framework designed to extend and maintain the ESCO taxonomy☆18Mar 12, 2026Updated 4 months ago
- ☆350Updated this week
- [ICML'24] Creative Text-to-Audio Generation via Synthesizer Programming☆41Sep 26, 2024Updated last year
- LLM-Training-API: Including Embeddings & ReRankers, mergekit, LaserRMT☆27Feb 18, 2024Updated 2 years ago
- Official Repository for "LLMs as Visual Explainers: Advancing Image Classification with Evolving Visual Descriptions"☆15Apr 20, 2025Updated last year