guidance-ai / llgtrtLinks
TensorRT-LLM server with Structured Outputs (JSON) built with Rust
☆55Updated last month
Alternatives and similar repositories for llgtrt
Users that are interested in llgtrt are comparing it to the libraries listed below
Sorting:
- A high-performance constrained decoding engine based on context free grammar in Rust☆53Updated last month
- High-performance MinHash implementation in Rust with Python bindings for efficient similarity estimation and deduplication of large datas…☆177Updated this week
- Super-fast Structured Outputs☆305Updated this week
- This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback☆94Updated 3 months ago
- Proxy server for triton gRPC server that inferences embedding model in Rust☆21Updated 10 months ago
- implement llava using candle☆15Updated last year
- Inference engine for GLiNER models, in Rust☆59Updated 2 months ago
- ☆30Updated 7 months ago
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆85Updated last week
- Inference Llama 2 in one file of zero-dependency, zero-unsafe Rust☆38Updated last year
- Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.☆62Updated last year
- ☆130Updated last year
- Faster structured generation☆224Updated last month
- Structured outputs for LLMs☆47Updated 11 months ago
- xet client tech, used in huggingface_hub☆117Updated this week
- Tensor library for Zig☆11Updated 7 months ago
- Unofficial Rust bindings to Apple's mlx framework☆158Updated last week
- Official Rust Implementation of Model2Vec☆118Updated 3 weeks ago
- ☆221Updated this week
- IBM development fork of https://github.com/huggingface/text-generation-inference☆60Updated last month
- ☆66Updated last year
- The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…☆137Updated last month
- ModernBERT model optimized for Apple Neural Engine.☆26Updated 5 months ago
- Unofficial python bindings for the rust llm library. 🐍❤️🦀☆75Updated last year
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆137Updated 10 months ago
- ☆13Updated last year
- Proof of concept for running moshi/hibiki using webrtc☆19Updated 3 months ago
- ☆19Updated 8 months ago
- Aana SDK is a powerful framework for building AI enabled multimodal applications.☆47Updated this week
- The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes☆203Updated 4 months ago