guidance-ai / llgtrtLinks
TensorRT-LLM server with Structured Outputs (JSON) built with Rust
☆59Updated 6 months ago
Alternatives and similar repositories for llgtrt
Users that are interested in llgtrt are comparing it to the libraries listed below
Sorting:
- High-performance MinHash implementation in Rust with Python bindings for efficient similarity estimation and deduplication of large datas…☆210Updated 3 weeks ago
- Faster structured generation☆254Updated last week
- Official Rust Implementation of Model2Vec☆138Updated 3 weeks ago
- This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback☆107Updated 7 months ago
- A high-performance constrained decoding engine based on context free grammar in Rust☆55Updated 5 months ago
- Super-fast Structured Outputs☆561Updated last week
- ☆139Updated last year
- Inference engine for GLiNER models, in Rust☆74Updated this week
- ☆134Updated last year
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆496Updated this week
- The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes☆236Updated 2 months ago
- Fast serverless LLM inference, in Rust.☆94Updated 7 months ago
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆97Updated 3 months ago
- Formatron empowers everyone to control the format of language models' output with minimal overhead.☆226Updated 4 months ago
- webgpu autograd library☆33Updated 5 months ago
- OpenAI compatible API for serving LLAMA-2 model☆218Updated 2 years ago
- Inference Llama 2 in one file of zero-dependency, zero-unsafe Rust☆39Updated 2 years ago
- implement llava using candle☆15Updated last year
- Unofficial Rust bindings to Apple's mlx framework☆202Updated 2 weeks ago
- Guaranteed Structured Output from any Language Model via Hierarchical State Machines☆145Updated 2 weeks ago
- Performance centered DSPy rewrite to(not port) Rust☆160Updated this week
- ☆13Updated 2 months ago
- InferX: Inference as a Service Platform☆136Updated this week
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆139Updated last year
- xet client tech, used in huggingface_hub☆302Updated this week
- Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.☆62Updated 2 years ago
- Rust crates for XetHub☆70Updated last year
- Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly a…☆78Updated 4 months ago
- Inference server benchmarking tool☆118Updated 3 weeks ago
- ☆197Updated last year