TensorRT-LLM server with Structured Outputs (JSON) built with Rust
☆71Apr 25, 2025Updated last year
Alternatives and similar repositories for llgtrt
Users that are interested in llgtrt are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Super-fast Structured Outputs☆820Jun 25, 2026Updated last month
- OpenAI compatible API for TensorRT LLM triton backend☆221Aug 1, 2024Updated last year
- Rust crate for some audio utilities☆32Jun 17, 2026Updated last month
- Sampling techniques for Candle.☆21Apr 3, 2024Updated 2 years ago
- A Keras like abstraction layer on top of the Rust ML framework candle☆23Jun 16, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Controllable Language Model Interactions in TypeScript☆10May 17, 2024Updated 2 years ago
- llm_utils: Basic LLM tools, best practices, and minimal abstraction.☆47Feb 18, 2025Updated last year
- A Rust crate for subscribing and retrieving the current active space on Mac OS X☆21Oct 26, 2024Updated last year
- A Rust library for working with the TEN VAD (Voice Activity Detection) ONNX model.☆18Apr 3, 2026Updated 3 months ago
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆702Updated this week
- 💫 Open-Source AI coding agent that integrates with Jupyter Notebooks☆22Jan 22, 2026Updated 6 months ago
- Functional configuration management with Typescript☆20Apr 7, 2025Updated last year
- A collection of LLM token samplers in Rust☆18Nov 9, 2023Updated 2 years ago
- Not everyone can code, but everyone can learn. This Project is an AI powered DSA/Competitive Programming Helper with an inbuilt editor to…☆14Jun 3, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Rust implementation of Huggingface transformers pipelines using onnxruntime backend with bindings to C# and C.☆47Apr 1, 2023Updated 3 years ago
- Machine Learning package allowing faster cross-validation☆18Aug 26, 2019Updated 6 years ago
- ORB Keypoint Detection with Rust☆15Dec 22, 2022Updated 3 years ago
- A TUI to view traces, spans, metrics, and inspect your Effect applications☆16Feb 28, 2026Updated 5 months ago
- An iTransformer implementation in Rust☆22Jan 10, 2026Updated 6 months ago
- @effect/schema compiler and pipeable API for OpenAPI☆16Jun 23, 2024Updated 2 years ago
- Using TensorRT and Triton Server to build BERT model as a service☆13Jan 10, 2022Updated 4 years ago
- PyTorch-equivalent deep learning framework in pure Rust — 22 crates, CUDA GPU, vision, LLMs, ONNX, autograd, distributed training☆30Jun 28, 2026Updated last month
- The repository is designed to help you build intent classification for user queries, and also generate tags for AI chat responses.☆12Mar 29, 2024Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- AI gateway and observability server written in Rust. Designed to help optimize multi-agent workflows.☆67Jul 2, 2024Updated 2 years ago
- In this section, I share the Meta-Heuristic algorithm codes that I wrote myself☆13Apr 6, 2023Updated 3 years ago
- Rust bindings to the Vectorscan high-performance regex library☆24Mar 13, 2026Updated 4 months ago
- Use `outlines` generators with Haystack.☆14Updated this week
- Run unit tests in a @vue/cli project with Karma☆10Jan 4, 2023Updated 3 years ago
- Source code for paper "Breaking Security-Critical Voice Authentication".☆13Jul 10, 2023Updated 3 years ago
- Proof of concept for running moshi/hibiki using webrtc☆21Feb 28, 2025Updated last year
- ocr，pdf转docx，pdf to docx☆23Nov 4, 2022Updated 3 years ago
- Rust bindings to the https://github.com/coqui-ai TTS library☆21Mar 27, 2023Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Pure-Rust SMB2/3 client. Faster than native macOS SMB on all operations. No C dependencies.☆27Jul 20, 2026Updated last week
- rswarm is a Rust library crafted to streamline AI agent interactions, particularly when working with OpenAI’s API.☆15Jul 20, 2026Updated last week
- Fast, fixed-size, floating-point decimal types.☆21May 16, 2026Updated 2 months ago
- Retrieval-Augmented Generation with pgvector as vector database☆14Jan 23, 2024Updated 2 years ago
- Улучшенный морфологический анализатор для русского языка с DAWG-оптимизацией☆37Nov 8, 2025Updated 8 months ago
- RusTTS is an unofficial Coqui TTS implementation.☆21Aug 12, 2022Updated 3 years ago
- Async SSH tunnel through a jump host.☆13Jun 27, 2023Updated 3 years ago