AIAnytime / LLM-Inference-API-in-RustLinks
LLM Inference API in Rust. It also has a streamlit app that requests the running API in Rust.
☆20Updated 2 years ago
Alternatives and similar repositories for LLM-Inference-API-in-Rust
Users that are interested in LLM-Inference-API-in-Rust are comparing it to the libraries listed below
Sorting:
- ⚡️Lightning fast in-memory VectorDB written in rust🦀☆26Updated 7 months ago
- Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.☆62Updated 2 years ago
- Rust implementation of Surya☆60Updated 7 months ago
- Ask shortgpt for instant and concise answers☆12Updated 2 years ago
- Light WebUI for lm.rs☆24Updated 11 months ago
- An educational Rust project for exporting and running inference on Qwen3 LLM family☆29Updated 2 months ago
- Ingest any document type and query☆13Updated 2 years ago
- A single-binary, GPU-accelerated LLM server (HTTP and WebSocket API) written in Rust☆79Updated last year
- auto-rust is an experimental project that automatically generate Rust code with LLM (Large Language Models) during compilation, utilizing…☆42Updated 10 months ago
- The easiest & fastest way to run customized and fine-tuned LLMs locally or on the edge☆20Updated 6 months ago
- 🤖📝 A markdown editor powered by AI (Ollama)☆64Updated 11 months ago
- On-device LLM Inference using Mediapipe LLM Inference API.☆22Updated last year
- Run AI models anywhere. https://muna.ai/explore☆68Updated this week
- OpenAI compatible API for serving LLAMA-2 model☆218Updated last year
- 🦀 A Pure Rust Framework For Building AGI (WIP).☆106Updated last week
- Built for demanding AI workflows, this gateway offers low-latency, provider-agnostic access, ensuring your AI applications run smoothly a…☆78Updated 4 months ago
- AI Assistant☆20Updated 5 months ago
- Model Context Protocol (MCP) CLI server template for Rust☆81Updated 5 months ago
- An LLM interface (chat bot) implemented in pure Rust using HuggingFace/Candle over Axum Websockets, an SQLite Database, and a Leptos (Was…☆137Updated last year
- A simple, CUDA or CPU powered, library for creating vector embeddings using Candle and models from Hugging Face☆41Updated last year
- ☆80Updated last year
- An openAI CLI built in rust☆10Updated 2 years ago
- ☆13Updated last month
- Turn natual language into commands. Your CLI tasks, now as easy as a conversation. Run it 100% offline, or use OpenAI's models.☆62Updated last year
- Unofficial python bindings for the rust llm library. 🐍❤️🦀☆76Updated 2 years ago
- Data extraction with LLM on CPU☆68Updated last year
- Rivet plugin for integration with Ollama, the tool for running LLMs locally easily☆42Updated 4 months ago
- An LLM-powered, autonomous coding assistant. Also offers an MCP and ACP mode.☆101Updated last week
- Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.☆117Updated last year
- VSCode Copilot for Groq fans!☆41Updated 3 months ago