Sushi engine runs very high quality quant model in Apple Silicon, using hybrid affine/exl3 with special expert tuning techniques.
☆270Oct 17, 2026Updated this week
Alternatives and similar repositories for sushi
Users that are interested in sushi are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Conformance and capability test suite for LLM inference engines — surface coverage, spec conformance, model capability, and an MTP/perfor…☆49Oct 5, 2026Updated last week
- Convert and Run Exl3 format LLM quant on Apple Silicon.☆58Aug 24, 2026Updated last month
- DeepSeek 4 Flash local inference engine for Metal☆72Oct 4, 2026Updated last week
- ☆27Updated this week
- A local inference engine for Apple silicon, built around the model.☆1,504Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Local computer use on Apple silicon: on-device typed decisions with laya and form filling with CUA-S1-FORMS, driven through the macOS Acc…☆520Updated this week
- qwen38-flash-next recipe for tp=2 GB10☆127Oct 6, 2026Updated last week
- Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell☆68Updated this week
- Metal Cuda Direct Memory Access☆196Updated this week
- LLM Inference Engine for Metal and CUDA.☆1,359Updated this week
- Apple Silicon GPU DVFS reverse engineering: voltage corners, CLPC frequency control, Metal benchmarks☆62Aug 24, 2026Updated last month
- Speakrail - a low-latency fully-local voice assistant that runs on a single RTX 4090☆187Oct 5, 2026Updated last week
- Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gem…☆719Oct 5, 2026Updated last week
- GGUF to MLX Converter; LM studio advanced tips; Simple text generation using Apple's MLX framework;Enhanced MLX implementation with trans…☆44Oct 8, 2026Updated last week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallel…☆118Aug 13, 2026Updated 2 months ago
- Optimized inference for the DGX Spark and spark clusters☆120Updated this week
- RCO: Optimization of Non-Decomposable Objectives under Exact Budget Constraints for LLM Quantization and Pruning☆53Jun 13, 2026Updated 4 months ago
- Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quan…☆135Updated this week
- Swift CLI that helps AI coding agents inspect runtime UI in Apple platform apps.☆105Updated this week
- One llama.cpp flag unlocks +33-39% decode speed for Qwen3.8-27B on consumer GPUs. The MTP head already ships inside your GGUF. Recipe, pa…☆363Sep 26, 2026Updated 3 weeks ago
- lithos-metal: high-performance LLM inference kernels and serving for Apple silicon.☆272Updated this week
- Codeberg fork of StillDeadcode / vllm-radiance which is a custom vllm tuned for R9700 GPU's☆122Oct 8, 2026Updated last week
- ☆141Sep 21, 2026Updated 3 weeks ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Library built for generating semantic layer graphs for query routing, query generation and data discovery☆147Updated this week
- ☆73Sep 14, 2026Updated last month
- EmbeddingGemma2 resources: local search apps, decision models, inference frameworks, tutorials, and datasets. English / 简体中文.☆43Updated this week
- Portable optimized ROCm and Qwen3.8-27B setup for AMD Strix Halo☆83Sep 16, 2026Updated last month
- GLM-5.3-Flash (NVFP4) at TP=4 across 4x ASUS GX10 / GB10 - vLLM + mentat instead of Ray, 524k context, DFlash2. Minimal working recipe.☆87Updated this week
- Real-time collaborative architectural CAD in your browser. Based on Kafka, gRPC, Redis, and OpenCASCADE.☆12Jan 7, 2023Updated 3 years ago
- keys-Wasserman Slate AI Studio powered by TensorFold: an AI film studio for Mac and NVIDIA DGX Spark. Plan a film like a crew (Slate work…☆137Updated this week
- automates the entire process of creating a bootable OpenCore hackintosh USB. No manual config.plist editing, no hunting down kexts, no ma…☆302Updated this week
- ☆123Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- AI 原生 3D 导演工作台与场景创作工具 / AI-native 3D direction and scene creation studio☆236Jul 30, 2026Updated 2 months ago
- Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with c…☆1,966Updated this week
- CUDA Blackwell performance fork of antirez/ds4 (DeepSeek-V4-Flash): ~4x prefill, ~1.5x decode, continuous batching, weight server, DSpark☆77Sep 4, 2026Updated last month
- Gumbel-Softmax post-training quantization for LLMs (1–3 bit scalar, INT/GGUF-compatible).☆98Jul 11, 2026Updated 3 months ago
- a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-…☆451Updated this week
- Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable☆1,024Updated this week
- CUDA compiler and runtime for Apple Silicon☆204Updated this week