TensorRT-LLM server with Structured Outputs (JSON) built with Rust
☆73Apr 25, 2025Updated last year
Alternatives and similar repositories for llgtrt
Users that are interested in llgtrt are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Super-fast Structured Outputs☆907Updated this week
- OpenAI compatible API for TensorRT LLM triton backend☆221Aug 1, 2024Updated 2 years ago
- Rust crate for some audio utilities☆32Jun 17, 2026Updated 4 months ago
- Use `outlines` generators with Haystack.☆14Updated this week
- Exposes a serialized machine learning model through a HTTP API.☆13Jun 11, 2023Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- rust port of silaro-vad python package☆14Apr 4, 2025Updated last year
- llm_utils: Basic LLM tools, best practices, and minimal abstraction.☆47Feb 18, 2025Updated last year
- fused MoE kernel in Candle backend☆14Feb 3, 2026Updated 9 months ago
- A Rust crate for subscribing and retrieving the current active space on Mac OS X☆21Oct 26, 2024Updated 2 years ago
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆740Oct 18, 2026Updated 2 weeks ago
- A collection of LLM token samplers in Rust☆18Nov 9, 2023Updated 2 years ago
- Generates llama.cpp compatible grammars from pydantic objects☆11Apr 2, 2024Updated 2 years ago
- A Rust compiler backend and runtime that compiles Rust kernels to many NPU/GPU accelerators via MLIR — vendor-agnostic (Ascend, Trainium,…☆21Updated this week
- Rust implementation of Huggingface transformers pipelines using onnxruntime backend with bindings to C# and C.☆47Apr 1, 2023Updated 3 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- ☆12Dec 6, 2018Updated 7 years ago
- Run unit tests in a @vue/cli project with Karma☆10Jan 4, 2023Updated 3 years ago
- Speexdsp bindings and pure-rust implementation☆26Feb 2, 2026Updated 9 months ago
- SWE-Serve: an agentic benchmark of 53 production inference-engineering tasks derived from merged SGLang pull requests, run with Harbor.☆23Oct 9, 2026Updated 3 weeks ago
- Goprox: Simplify Google searches with automatic proxy handling and user-agent selection.☆11Apr 12, 2024Updated 2 years ago
- The repository is designed to help you build intent classification for user queries, and also generate tags for AI chat responses.☆12Mar 29, 2024Updated 2 years ago
- Python SDK for Warrant☆12Jun 3, 2024Updated 2 years ago
- rswarm is a Rust library crafted to streamline AI agent interactions, particularly when working with OpenAI’s API.☆14Updated this week
- AI gateway and observability server written in Rust. Designed to help optimize multi-agent workflows.☆68Jul 2, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Rust bindings to the https://github.com/coqui-ai TTS library☆21Mar 27, 2023Updated 3 years ago
- detect timestamp of all scene changes in video☆10Nov 23, 2021Updated 4 years ago
- Rust bindings to frameworks made by Apple☆12Feb 13, 2023Updated 3 years ago
- The Triton TensorRT-LLM Backend☆944Updated this week
- ocr，pdf转docx，pdf to docx☆21Nov 4, 2022Updated 4 years ago
- TensorFlow Lite Elixir bindings with optional EdgeTPU support.☆24Oct 14, 2026Updated 3 weeks ago
- ☆13Mar 19, 2024Updated 2 years ago
- Ask shortgpt for instant and concise answers☆13May 7, 2023Updated 3 years ago
- In this section, I share the Meta-Heuristic algorithm codes that I wrote myself☆13Apr 6, 2023Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆10Oct 16, 2026Updated 3 weeks ago
- The EventHubClient gem is a simple wrapper of EventHub HTTP APIs.☆14Jun 3, 2014Updated 12 years ago
- Your dive log for deep work☆11Aug 16, 2023Updated 3 years ago
- 🤖 Complete reproduction of 'AlphaGo Moment for Model Architecture Discovery' using MLX-LM instead of GPT-4. Autonomous neural architectu…☆30Jul 27, 2025Updated last year
- The subjective gate for Impatient Programming: no PR merges while a reviewer would ask for the change to be simplified☆10Oct 5, 2026Updated last month
- ☆14Oct 20, 2026Updated 2 weeks ago
- Discover and manage contracts, multisigs and their access control policies on various blockchains.☆11Aug 1, 2025Updated last year