A local LLM server built for concurrent work. Drop-in replacement for Ollama and/or OpenAI and Ollama APIs on one port. Requests that share a prompt reuse each other's KV cache instead of each prefilling it. Rust, wrapping llama.cpp.
☆188Aug 22, 2026Updated 3 weeks ago
Alternatives and similar repositories for fox
Users that are interested in fox are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Profile-guided GPU kernel optimizer for AMD/RDNA3. Auto-tunes llama.cpp MMVQ kernels per model shape. 2x decode speedup on 7900 XTX.☆69Sep 5, 2026Updated 2 weeks ago
- Generate Agent Skills from MCP server tools. Connects via Streamable HTTP, discovers tools, and outputs a skill with schema docs and a ca…☆16Feb 15, 2026Updated 7 months ago
- A minimal, async-first AI agent framework in Rust with multi-provider support (OpenRouter, OpenAI, Anthropic, Ollama), JSON schema-based …☆15Apr 27, 2026Updated 4 months ago
- vLLM fork with Marlin W4A8 SM121 patches + TMA module☆15Mar 23, 2026Updated 5 months ago
- ☆24Dec 29, 2025Updated 8 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Open-source LLM observability and evaluation platform☆16Jul 13, 2026Updated 2 months ago
- SmarterRouter: An intelligent LLM gateway and VRAM-aware router for Ollama, llama.cpp, and OpenAI. Features semantic caching, model profi…☆152May 10, 2026Updated 4 months ago
- A powerful CLI tool to manage, test, and optimize RAG pipelines. Streamline your Retrieval-Augmented Generation workflows from terminal.☆23Jan 12, 2026Updated 8 months ago
- Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware☆521Updated this week
- Your AI colleague, in the apps you already use.☆14Updated this week
- The go to place for official fluvio connectors☆22Apr 26, 2023Updated 3 years ago
- AI accelerators, edge inference devices, compilers, runtimes, benchmarks, and research for building and evaluating machine-learning syste…☆18Jun 16, 2026Updated 3 months ago
- Full-stack OpenTelemetry observability for Apache Spark☆18Updated this week
- The Rust library for video generation models based on Candle (HF)☆26Sep 2, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Python tools for analyzing Claude Code conversation history☆27May 21, 2026Updated 4 months ago
- ☆26Oct 3, 2025Updated 11 months ago
- 🦀 Claude Code toolkit for self improving skills, performance and mcp-essentials☆36Apr 29, 2026Updated 4 months ago
- SwiftLet is a lightweight Python framework for running open-source Large Language Models (LLMs) locally using safetensors☆29Aug 6, 2025Updated last year
- Build real software step-by-step with Claude, Codex, OpenCode, Gemini, Cursor, and other agents☆16Mar 29, 2026Updated 5 months ago
- Mooch - Your Interview Pal☆17Updated this week
- Pure Rust Inference Engine☆700Updated this week
- llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.☆275Updated this week
- A minimal implementation of ZKPs of Ethereum block execution using Reth. Supports both Ethereum and OP Stack.☆13Jul 2, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆22May 12, 2023Updated 3 years ago
- A very fancy counter☆11Nov 19, 2024Updated last year
- Start at the bodies bearing the weight. Track where the weight goes. Build solutions that remove the weight and transfer it to something …☆17Updated this week
- ☆14Sep 7, 2020Updated 6 years ago
- On-device documentation search for agents☆54Mar 27, 2026Updated 5 months ago
- Tools for generating to and from wit format☆12Sep 6, 2024Updated 2 years ago
- Frona is a personal AI assistant. You create autonomous agents, give them tools, and talk to them through a chat interface. Agents act on…☆203Updated this week
- Thermal pocket printer - BLE protocol reverse engineering, Python CLI, and web GUI☆21Sep 4, 2026Updated 2 weeks ago
- ☆13Aug 7, 2023Updated 3 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Mixed-vendor GPU inference cluster manager with speculative decoding☆35Jul 2, 2026Updated 2 months ago
- Camera control UI 3d Minimax☆82Updated this week
- Rust-libp2p transport for {t,n}-threshold signature schemes☆33Jul 8, 2022Updated 4 years ago
- Desktop application to download Udemy courses with attachments.☆16Nov 8, 2025Updated 10 months ago
- ☆14Feb 17, 2024Updated 2 years ago
- browserifiable lotion light client☆15May 11, 2019Updated 7 years ago
- Open-source research workspace for connecting sources, evidence, notes and ideas. Built for academic research with Zotero integration. Av…☆257Updated this week