Alternatives and similar repositories for LLM-Inference-API-in-Rust

Users that are interested in LLM-Inference-API-in-Rust are comparing it to the libraries listed below

• KevKibe / memvectordb
  ⚡️Lightning fast in-memory VectorDB written in rust🦀
  ☆26Updated 7 months ago
• spyglass-search / memex
  Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc.
  ☆62Updated 2 years ago
• smithakolan / AssemblyAI-Applications
  Here is a collection of cool applications that I've built with AssemblyAI
  ☆36Updated last year
• PlugOvr-ai / PlugOvr
  AI Assistant
  ☆20Updated 6 months ago
• ayaansh-roy / voice_assistant_llm
  ☆68Updated last year
• ShelbyJenkins / candle_embed
  A simple, CUDA or CPU powered, library for creating vector embeddings using Candle and models from Hugging Face
  ☆46Updated last year
• pixelspark / poly
  A single-binary, GPU-accelerated LLM server (HTTP and WebSocket API) written in Rust
  ☆79Updated last year
• rupeshs / shortgpt
  Ask shortgpt for instant and concise answers
  ☆12Updated 2 years ago
• samuel-vitorino / lm.rs-webui
  Light WebUI for lm.rs
  ☆24Updated last year
• jimexist / surya-rs
  Rust implementation of Surya
  ☆62Updated 8 months ago
• AIAnytime / On-device-LLM-Inference-using-Mediapipe
  On-device LLM Inference using Mediapipe LLM Inference API.
  ☆22Updated last year
• minskylab / auto-rust
  auto-rust is an experimental project that automatically generate Rust code with LLM (Large Language Models) during compilation, utilizing…
  ☆42Updated 11 months ago
• unclecode / groqopilot
  VSCode Copilot for Groq fans!
  ☆42Updated 3 months ago
• rust10x / rust-ai-buddy
  ☆40Updated last year
• AIAnytime / Function-Calling-Mistral-7B
  Function Calling Mistral 7B. Learn how to make functions call for open source LLMs.
  ☆48Updated last year
• Intellicode / writer
  🤖📝 A markdown editor powered by AI (Ollama)
  ☆64Updated last year
• juliooa / secondbrain
  Multi-platform desktop app to download and run Large Language Models(LLM) locally in your computer.
  ☆289Updated 2 years ago
• startino / aitino
  Aitino is a platform that allows for the creation of crews of AI Agents to help you automate tasks and solve complex problems.
  ☆89Updated 7 months ago
• sudarshan-koirala / rag-chat-with-pdf
  Simple Chainlit UI for running llms locally using Ollama and LangChain
  ☆120Updated last year
• AmineDiro / cria
  OpenAI compatible API for serving LLAMA-2 model
  ☆218Updated 2 years ago
• haseeb-heaven / langchain-coder
  Web Application that can generate code and fix bugs and run using various LLM's (GPT,Gemini,PALM)
  ☆126Updated last year
• jeremychone / rust-agentic
  Support library for building Agentic MCP and Agent2Agent-based services.
  ☆28Updated 4 months ago
• IncredibleDevHQ / agent-panel
  AI gateway and observability server written in Rust. Designed to help optimize multi-agent workflows.
  ☆64Updated last year
• definitive-io / llama3-function-calling
  ☆59Updated last year
• mcp-ectors / mcp-ectors
  The MCP enterprise actors-based server or mcp-ectors for short
  ☆31Updated 5 months ago
• AllAboutAI-YT / chatTTS
  ChatTTS + Ollama
  ☆83Updated last year
• katanaml / llm-rag-invoice-cpu
  Data extraction with LLM on CPU
  ☆68Updated last year
• john-adeojo / crew_ai_tutorial
  A quick Crew AI tutorial
  ☆23Updated last year
• zflow-dev / zflow
  Flow Based Low/No Code Workflow and Robotic Process Automation Library
  ☆91Updated 2 months ago
• danielclough / fireside-chat
  An LLM interface (chat bot) implemented in pure Rust using HuggingFace/Candle over Axum Websockets, an SQLite Database, and a Leptos (Was…
  ☆137Updated last year