30 tok/s for 20B MoE on 8 GB VRAM. Flat throughput to 32K context. Native MXFP4 + GGUF Q4_K/Q5_K/Q6_K via ggml CUDA kernels — zero dequant. Expert offloading for models that don't fit in GPU memory.
☆23Apr 7, 2026Updated 7 months ago
Alternatives and similar repositories for tinyserve
Users that are interested in tinyserve are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Orquestr Pro - AI Assistant Desktop App☆19Mar 5, 2026Updated 8 months ago
- Teach GLM-5.2 to see art: minimal frozen-backbone vision-projector training (method by Harry Partridge)☆14Jul 18, 2026Updated 3 months ago
- Transforms raw text into interconnected knowledge graphs and autonomously generates insights through advanced pattern detection. It combi…☆11Jul 29, 2026Updated 3 months ago
- Exact SGLang runtime for GLM-5.3-Flash-NVFP4 on 4x RTX PRO 6000 Blackwell (SM120): flashinfer_cutlass MoE, DSA+KDA attention, NEXTN MTP5,…☆17Aug 27, 2026Updated 2 months ago
- Go program to launch and configure LXD containers and attached disk devices☆12Nov 22, 2023Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A linux gaming system manager☆12Sep 4, 2026Updated 2 months ago
- KV cache visualizations☆12Oct 7, 2026Updated last month
- A cross-platform printer AI MCP server that supports Windows, macOS, and Linux systems, providing printer management, status queries, and…☆14Mar 10, 2026Updated 8 months ago
- CLI + MCP for agent context: a task layer over TickTick, Notion, GitHub, Obsidian & more (RRF retrieval, provenance, reviewed writes, und…☆17Updated this week
- an MCP for querying the EVM☆12Apr 1, 2025Updated last year
- ☆15Aug 2, 2026Updated 3 months ago
- ☆13Oct 28, 2025Updated last year
- Send files or text instantly and securely. SafeBeam generates a one-time OTP valid for 10 minutes only. No sign-up, no device limits — wo…☆14May 15, 2026Updated 5 months ago
- Python library for Telize JSON IP and GeoIP REST API☆12Nov 26, 2015Updated 10 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A dumb launcher to spawn swaylock and ensure it runs no matter what☆12Apr 30, 2022Updated 4 years ago
- Cross-platform pure C+GPU inference for Llama 2-family, Mistral Nemo, GPT-2, Gemma-3n, Qwen 3.5+, GPT-NeoX, StableLM, Spark-X2.6, GLM-5.3…☆16Updated this week
- ☆12Mar 17, 2015Updated 11 years ago
- ☆15Feb 10, 2026Updated 9 months ago
- Romance Meter Extension for SillyTavern☆16Aug 4, 2025Updated last year
- Qwen3.6 27B × DFlash — 30-35 tok/s on NVIDIA DGX Spark (GB10) - LLama.Cpp☆17May 2, 2026Updated 6 months ago
- PolarEngine: vLLM plugin for PolarQuant quantized LLM inference — 75% FP16 speed at 2.3x less VRAM☆36Apr 13, 2026Updated 7 months ago
- Bluetooth Low Energy (BLE) MCP server for Claude Code and other MCP-compatible agents.☆19Mar 30, 2026Updated 7 months ago
- High-fidelity transcript memory for Claude Code — index every session, recall byte-identical, search, prune, compose. Local-first, stdlib…☆19Aug 10, 2026Updated 3 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Huggingface but for inference engines.☆14Updated this week
- Production-grade Skill orchestration, SOP enforcement, and long-context runtime control for OpenCode☆18Mar 29, 2026Updated 7 months ago
- Django Productivity Tools☆13May 13, 2011Updated 15 years ago
- Frequency-based KV cache pruning for llama.cpp — 25% cache reduction, improved PPL at long context. GPU compaction kernel for HIP/ROCm.☆16Apr 18, 2026Updated 6 months ago
- Effortlessly deploy a hardware-passthrough (VFIO) setup for Virtual Machines (VMs) on a Linux desktop. Includes quality-of-life features …☆18Apr 3, 2025Updated last year
- Qwen 3.8 Next Flash toolbox utilizing pwilkin's strix-halo llama.cpp fork☆13Sep 16, 2026Updated last month
- ☆15Oct 20, 2024Updated 2 years ago
- Generalized app for two-step verification.☆12Feb 28, 2024Updated 2 years ago
- Pure PyTorch implementation of HISA (Hierarchical Indexed Sparse Attention) as a drop-in plugin for HuggingFace models☆16Apr 7, 2026Updated 7 months ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- [Unmaintained] A select component heavily inspired by Harvest's Chosen☆18Nov 1, 2013Updated 13 years ago
- Opensource DDR4 Controller☆22Sep 23, 2026Updated last month
- This repo implements a generic version of the REAM algorithm described in “REAM: Compressing Mixture-of-Experts LLMs” and makes it easy t…☆11Aug 29, 2026Updated 2 months ago
- My Incus playground☆22Updated this week
- GPT with metaweights: weights that don't actually exist☆26Oct 4, 2026Updated last month
- A Gradio Web UI for running local LLM on Intel GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max) using IPEX-LLM.☆16Updated this week
- AI skill for @chenglou/pretext — exact text metrics without DOM reflows☆15Updated this week
Are these results useful?