This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows instead of cloud.
☆127Feb 29, 2024Updated 2 years ago
Alternatives and similar repositories for trt-llm-as-openai-windows
Users that are interested in trt-llm-as-openai-windows are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- OpenAI compatible API for TensorRT LLM triton backend☆221Aug 1, 2024Updated 2 years ago
- All-in-one Full-Featured Python/Flet/Flutter Application to make the most of all the latest Open-Source AI Art Generators in an intuitive…☆16May 30, 2025Updated last year
- Flask API for generating text embeddings using OpenAI or sentence_transformers☆14Sep 1, 2023Updated 3 years ago
- An OpenAI API compatible images server to generate or manipulate images.☆18Feb 2, 2025Updated last year
- ☆24Feb 18, 2024Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆59Updated this week
- Text-Guided Generation of Full-Body Image with Preserved Reference Face for Customized Animation☆24Jun 24, 2024Updated 2 years ago
- TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizat…☆14,780Updated this week
- Wyoming protocol server that calls an external program to play audio☆19Apr 28, 2024Updated 2 years ago
- ☆14Feb 18, 2023Updated 3 years ago
- ☆16Nov 22, 2024Updated last year
- ☆12Sep 28, 2024Updated 2 years ago
- Run Vision LLMs, TTS and STT APIs. Website and API for https://text-generator.io☆38Jul 16, 2026Updated 2 months ago
- Code for "Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation"☆13Jul 10, 2020Updated 6 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- experiments with inference on llama☆103Jun 6, 2024Updated 2 years ago
- Offline version of the NanoPy Editor. No server needed.☆14Mar 7, 2025Updated last year
- A one-stop library to standardize the inference and evaluation of all the conditional video generation models.☆50Feb 13, 2025Updated last year
- Toonification of real face images using PyTorch, Stylegan2 and Image-to-Image translation☆14Jun 14, 2022Updated 4 years ago
- The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…☆187Nov 24, 2025Updated 10 months ago
- ☆20Jan 17, 2025Updated last year
- An example of using multimodal LLMs to processpide feed from camera and get image description☆14Mar 11, 2024Updated 2 years ago
- 大模型部署实战：TensorRT-LLM, Triton Inference Server, vLLM☆26Feb 26, 2024Updated 2 years ago
- ☆12Feb 22, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Safely push a Cog model version by making sure it works and is backwards-compatible with previous versions.☆17Dec 4, 2025Updated 10 months ago
- An end to end ML project. Using MLflow for experiment tracking and model registry. Prefect for workflow orchestration. S3 for artifacts s…☆12Sep 11, 2022Updated 4 years ago
- Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification☆11Aug 12, 2023Updated 3 years ago
- Annotation builder to use segmentation in Mask_RCNN, even if your annotations are rectangular instead of polygon.☆15Feb 16, 2022Updated 4 years ago
- Facebook AI Research Sequence-to-Sequence Toolkit written in Python.☆12Oct 10, 2020Updated 5 years ago
- ☆50Jun 7, 2025Updated last year
- ☆26Mar 6, 2024Updated 2 years ago
- ☆361Updated this week
- LLM-Training-API: Including Embeddings & ReRankers, mergekit, LaserRMT☆28Feb 18, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- NMT with ssp☆11Oct 28, 2021Updated 4 years ago
- Probabilistic LLM evaluations. [CogSci2023; ACL2023]☆73Jul 27, 2024Updated 2 years ago
- Shader library for HeavyM, ISF format, mostly from Shadertoy.com☆22Dec 22, 2021Updated 4 years ago
- Expanded KR-BERT by adding more training data☆13Apr 23, 2021Updated 5 years ago
- ☆32Dec 3, 2024Updated last year
- For converting LLM datasets from one format into another.☆22Nov 12, 2025Updated 10 months ago
- Source code for MWO2KG and Echidna: Constructing and Exploring Knowledge Graphs from Maintenance Data☆11Feb 13, 2023Updated 3 years ago