guidance-ai / llgtrt
TensorRT-LLM server with Structured Outputs (JSON) built with Rust
☆13Updated last week
Related projects ⓘ
Alternatives and complementary repositories for llgtrt
- Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.☆55Updated last year
- High-performance MinHash implementation in Rust with Python bindings for efficient similarity estimation and deduplication of large datas…☆65Updated 3 months ago
- Rust implementation of Surya☆52Updated last month
- Heelix is an open-source chat client written in Rust. Collecting data as you work via accessibility API and vision into a local database,…☆12Updated last week
- A single-binary, GPU-accelerated LLM server (HTTP and WebSocket API) written in Rust☆80Updated 10 months ago
- "Just hoof it!" - A spotlight like interface to Ollama☆56Updated 7 months ago
- Library for doing RAG☆42Updated last week
- Unofficial Rust bindings to Apple's mlx framework☆68Updated this week
- Easily create LLM automation/agent workflows☆55Updated 9 months ago
- Structured outputs for LLMs☆31Updated 4 months ago
- A Fish Speech implementation in Rust, with Candle.rs☆45Updated this week
- 8-bit floating point types for Rust☆39Updated last month
- Inference Llama 2 in one file of zero-dependency, zero-unsafe Rust☆37Updated last year
- AI gateway and observability server written in Rust. Designed to help optimize multi-agent workflows.☆46Updated 4 months ago
- Light WebUI for lm.rs☆22Updated last month
- HTTP proxy for on-demand model loading with llama.cpp (or other OpenAI compatible backends)☆41Updated this week
- The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes☆133Updated 3 weeks ago
- git-like rag pipeline☆32Updated this week
- The fastest CLI tool for prompting LLMs. Including support for prompting several LLMs at once!☆62Updated 2 months ago
- Ask shortgpt for instant and concise answers☆13Updated last year
- Rust bindings for OpenNMT/CTranslate2☆22Updated this week
- A python package for serving LLM on OpenAI-compatible API endpoints with prompt caching using MLX.☆55Updated last week
- Run Generative AI models directly on your hardware☆22Updated 3 months ago
- Neural search for web-sites, docs, articles - online!☆128Updated 3 weeks ago
- ☆10Updated last year
- A simple and clear way of hosting llama.cpp as a private HTTP API using Rust☆26Updated 5 months ago
- LLama implementations benchmarking framework☆12Updated last year
- Rust library to access openai API☆15Updated 11 months ago
- Rust bindings to https://github.com/k2-fsa/sherpa-onnx☆45Updated this week
- ☆15Updated 8 months ago