TensorRT-LLM server with Structured Outputs (JSON) built with Rust
☆72Apr 25, 2025Updated last year
Alternatives and similar repositories for llgtrt
Users that are interested in llgtrt are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Super-fast Structured Outputs☆843Updated this week
- ☆26Jul 26, 2026Updated 3 weeks ago
- Lazily-evaluated, Streaming, State Management for Effect☆14May 24, 2025Updated last year
- OpenAI compatible API for TensorRT LLM triton backend☆221Aug 1, 2024Updated 2 years ago
- Graph model execution API for Candle☆18Jul 27, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Rust crate for some audio utilities☆32Jun 17, 2026Updated 2 months ago
- A Keras like abstraction layer on top of the Rust ML framework candle☆23Jun 16, 2024Updated 2 years ago
- Controllable Language Model Interactions in TypeScript☆10May 17, 2024Updated 2 years ago
- llm_utils: Basic LLM tools, best practices, and minimal abstraction.☆47Feb 18, 2025Updated last year
- A Rust library for working with the TEN VAD (Voice Activity Detection) ONNX model.☆18Apr 3, 2026Updated 4 months ago
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆714Updated this week
- A libcluster strategy for Digital Ocean Droplets☆12May 11, 2023Updated 3 years ago
- Cldr-based fork of the most excellent "trans" library☆10May 11, 2026Updated 3 months ago
- A collection of LLM token samplers in Rust☆18Nov 9, 2023Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Rust implementation of Huggingface transformers pipelines using onnxruntime backend with bindings to C# and C.☆47Apr 1, 2023Updated 3 years ago
- A highly customizable, full scale web backend for web-rwkv, built on axum with websocket protocol.☆30Apr 15, 2024Updated 2 years ago
- ORB Keypoint Detection with Rust☆15Dec 22, 2022Updated 3 years ago
- An iTransformer implementation in Rust☆22Jan 10, 2026Updated 7 months ago
- Using TensorRT and Triton Server to build BERT model as a service☆13Jan 10, 2022Updated 4 years ago
- PyTorch-equivalent deep learning framework in pure Rust — 22 crates, CUDA GPU, vision, LLMs, ONNX, autograd, distributed training☆30Jun 28, 2026Updated last month
- The repository is designed to help you build intent classification for user queries, and also generate tags for AI chat responses.☆12Mar 29, 2024Updated 2 years ago
- Tantivy directory implementation backed by object_store☆42Jan 22, 2024Updated 2 years ago
- A flat, index-based Red-Black Tree with no heap allocations. Ideal memory-constrained environments.☆17Jul 6, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- The Triton TensorRT-LLM Backend☆941Updated this week
- Manage branches for your Jira tickets☆18Updated this week
- Testing library/runner for load and integration testing using intelligent bots☆20Apr 8, 2024Updated 2 years ago
- experiment☆12Jan 1, 2023Updated 3 years ago
- Pure-Rust SMB2/3 client. Faster than native macOS SMB on all operations. No C dependencies.☆28Aug 8, 2026Updated last week
- Fast, fixed-size, floating-point decimal types.☆21May 16, 2026Updated 3 months ago
- rswarm is a Rust library crafted to streamline AI agent interactions, particularly when working with OpenAI’s API.☆14Updated this week
- Retrieval-Augmented Generation with pgvector as vector database☆14Jan 23, 2024Updated 2 years ago
- Улучшенный морфологический анализатор для русского языка с DAWG-оптимизацией☆39Nov 8, 2025Updated 9 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- The Rust library for video generation models based on Candle (HF)☆23Jul 24, 2026Updated 3 weeks ago
- Async SSH tunnel through a jump host.☆13Jun 27, 2023Updated 3 years ago
- A clone-able base site to help you get started quickly with Bartholomew☆17Jan 2, 2024Updated 2 years ago
- Your dive log for deep work☆11Aug 16, 2023Updated 3 years ago
- Learn how to create and deploy an ESP high availability system using Kafka as the message broker.☆11Feb 20, 2020Updated 6 years ago
- ☆25Jan 29, 2026Updated 6 months ago
- Faster structured generation☆306Aug 5, 2026Updated 2 weeks ago