30 tok/s for 20B MoE on 8 GB VRAM. Flat throughput to 32K context. Native MXFP4 + GGUF Q4_K/Q5_K/Q6_K via ggml CUDA kernels — zero dequant. Expert offloading for models that don't fit in GPU memory.
☆22Apr 7, 2026Updated 5 months ago
Alternatives and similar repositories for tinyserve
Users that are interested in tinyserve are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆15Feb 10, 2026Updated 7 months ago
- A dumb launcher to spawn swaylock and ensure it runs no matter what☆12Apr 30, 2022Updated 4 years ago
- ☆14Oct 20, 2024Updated last year
- My wezterm config☆17Feb 18, 2023Updated 3 years ago
- PolarEngine: vLLM plugin for PolarQuant quantized LLM inference — 75% FP16 speed at 2.3x less VRAM☆36Apr 13, 2026Updated 5 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A collection of LittleSnitch rule sets to block outbound connections to a given country.☆16Jun 13, 2023Updated 3 years ago
- Effortlessly deploy a hardware-passthrough (VFIO) setup for Virtual Machines (VMs) on a Linux desktop. Includes quality-of-life features …☆18Apr 3, 2025Updated last year
- ☆23May 8, 2026Updated 4 months ago
- Frequency-based KV cache pruning for llama.cpp — 25% cache reduction, improved PPL at long context. GPU compaction kernel for HIP/ROCm.☆16Apr 18, 2026Updated 5 months ago
- Production-grade Skill orchestration, SOP enforcement, and long-context runtime control for OpenCode☆18Mar 29, 2026Updated 6 months ago
- An alternative to Arc Control by Intel®.☆18Jun 16, 2024Updated 2 years ago
- GUI for LLaDA Diffusion LLM with Quantization for low end GPU and CPU options.☆24Mar 7, 2025Updated last year
- ☆40Mar 17, 2025Updated last year
- ☆20Jul 21, 2026Updated 2 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- My Incus playground☆22Sep 25, 2026Updated last week
- Telegram bot which can work with both openAI and LocalAI modes, it also uses UncensoredGPT models like Wizard-Uncensored. It can be launc…☆21Mar 14, 2025Updated last year
- Generate random nonsense phrases from words in a file☆21Dec 27, 2025Updated 9 months ago
- World's first TurboQuant KV cache compression for llama.cpp on AMD ROCm (RX 9070 / gfx1201)☆21Apr 28, 2026Updated 5 months ago
- Tools for debugging in Django☆23Nov 19, 2009Updated 16 years ago
- Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization☆475Aug 17, 2026Updated last month
- Local drive deep search.☆35Jun 4, 2025Updated last year
- EWDK C++ toolchain for bazel☆12Aug 22, 2026Updated last month
- A faster lolcat.☆27Nov 6, 2019Updated 6 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- LLM Chat is an open-source serverless alternative to ChatGPT.☆36Sep 13, 2024Updated 2 years ago
- A command-line tool that combines multiple AI models (Claude, OpenAI, Gemini) to generate comprehensive architectural plans☆72May 22, 2025Updated last year
- A set of scripts to build a docker environment that easily runs QEMU/Libvirt, and includes looking-glass + scream for sound. Supports VFI…☆30Nov 6, 2022Updated 3 years ago
- L'application pour bloquer un paquet, snipping, analyser le réseau☆11Dec 23, 2016Updated 9 years ago
- ☆12May 17, 2020Updated 6 years ago
- Self-Loading Registration Free COM Functions☆11Nov 12, 2019Updated 6 years ago
- Automated Deployment of AMD ROCm 7.1.1 and PyTorch 2.11.0 (nightly@ROCm7.1) AI Stack for Ubuntu 22.04/24.04 on CDNA 3, RDNA4 and RDNA3. T…☆25May 5, 2026Updated 4 months ago
- A lightweight adjustment tool for smoothing token probabilities in the Qwen models to encourage balanced multilingual generation.☆109Jul 9, 2025Updated last year
- 4 distros, one USB, ZFS on root. Debian, Fedora, RHEL and Arch. Offline install, boot environments, WireGuard, eBPF. Free.☆39Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆11Jan 8, 2022Updated 4 years ago
- Windows file system driver which allows to block access to files at run-time (C/C++, C#, WDK, SDK)☆12Jan 1, 2023Updated 3 years ago
- Panda - is a set of utilities used to research how PsExec encrypts its traffic.☆13Apr 20, 2021Updated 5 years ago
- 一个开放源码的Windows视频播放器☆10May 9, 2021Updated 5 years ago
- PhysPatch performs physical memory scanning and patching of the entire Windows Kernel using DMA☆13Nov 10, 2024Updated last year
- Can convert any binary data into an Obj with vertices, useful for reverse-engineering.☆11Mar 27, 2026Updated 6 months ago
- A REALLY Danger Windows Driver, Turn Any threads Ring0!☆14Jul 20, 2026Updated 2 months ago