V100 / SM70-focused vLLM engineering fork for modern LLM inference.
☆1,431Oct 28, 2026Updated this week
Alternatives and similar repositories for 1Cat-vLLM
Users that are interested in 1Cat-vLLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Implementation of FlashAttention-2 for Nvidia Tesla V100 / Titan V☆216Oct 14, 2026Updated 2 weeks ago
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆254Sep 14, 2026Updated last month
- Hand-written NVFP4 W4A16 CUDA kernels for Volta☆178Aug 19, 2026Updated 2 months ago
- forked from vllm-project/flash-attention☆70May 9, 2026Updated 5 months ago
- llama.cpp optimized for NVIDIA Tesla V100 (SM70): Qwen3.8-Flash-Next with MTP (prefill 3402 tok/s at 200K context on 6x V100-16GB; also r…☆64Oct 23, 2026Updated last week
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- SGLang for cool dudes using V100☆82Oct 5, 2026Updated 3 weeks ago
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆26Jun 20, 2026Updated 4 months ago
- Strata port for Tesla V100 (sm_70): Qwen3.8-Flash-Next with 262K context and MTP☆113Updated this week
- NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). …☆52Sep 15, 2026Updated last month
- This project is specifically developed for V100, based on lmdeploy 0.12.1, and supports mainstream open-source models from Q4 2025 to Q1 …☆22Mar 18, 2026Updated 7 months ago
- A 198t/s and 353t/s qwen 3.8 27b inferencing engine for 2*v100 and 4*v100☆84Oct 24, 2026Updated last week
- 1CatV2 with TileLANG written FA-v100 and many goodies☆20Jul 15, 2026Updated 3 months ago
- Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.☆23Jul 10, 2026Updated 3 months ago
- vLLM fork for Tesla V100☆15May 31, 2026Updated 5 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/dis…☆20Updated this week
- Block-FP8 (W8A16) on NVIDIA V100 / Volta (sm_70) for vLLM — the FP8 path upstream drops on Volta, tuned for sparse MoE.☆14Jul 10, 2026Updated 3 months ago
- Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, Mini…☆34Oct 7, 2026Updated 3 weeks ago
- Auto-built Docker image of 1Cat-vLLM for NVIDIA Tesla V100 (sm_70/Volta) with CUDA 12.8 — rebuilt daily when upstream releases☆12Jul 22, 2026Updated 3 months ago
- High-performance Duo-V100-GPU inference for selected model checkpoints and GPUs.☆35Oct 14, 2026Updated 2 weeks ago
- Fork of llama.cpp Nvida Volta V100 (tensor parallelism 4 x v100 GPU)☆12Updated this week
- 1Cat-vLLM on Intel Gaudi☆44Oct 12, 2026Updated 2 weeks ago
- The definitive vLLM runtime for dual RTX 2080 Ti 22GB + NVLink, delivering Qwen 27B local inference with maximum 200+ tok/s single-reques…☆1,194Oct 1, 2026Updated 3 weeks ago
- The set-and-forget LLM engine for Pascal and Volta: PXQ codec + kernels, auto-tuned per card. Ready-to-run PXQ models in MODELS.md; bench…☆102Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A ComfyUI custom node enabling **Flash Attention 1** on legacy NVIDIA GPUs (Tesla V100, T4) that lack Compute Capability 8.0+ required by…☆33Feb 9, 2026Updated 8 months ago
- A tool to unlobotomize your NVIDIA card!☆818Updated this week
- CUDA engine that splits every layer of a production quality MoE LLM across three PCIe V100 cards delivering 382 tok/s on a used $1,800 bo…☆43Updated this week
- Local LLM Inference Speed Test Tool☆247Oct 21, 2026Updated last week
- Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on loca…☆36,894Updated this week
- High-performance single-GPU inference for selected model checkpoints and GPUs.☆3,223Updated this week
- FlashAttention brought back to Tesla V100 — a deep llama.cpp fork: SM 7.0 D256 kernels, SplitKV3, q4_0 KV cache, DFlash2 speculative deco…☆19Updated this week
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆474Sep 22, 2026Updated last month
- Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own …☆2,317Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Unlock P2P comms between consumer NVIDIA GPUs☆684Updated this week
- Qwen3.8-27B in native NVFP4/FP8 on 2x PCIe Tesla V100-32GB (SM70): the PCIe runbook for v100-skinny + 1Cat-vLLM, with the 3 fixes that ma…☆15Sep 13, 2026Updated last month
- ML software (llama.cpp, ComfyUI, Strata, Maya, ...) builds for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60☆364Updated this week
- vLLM for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60☆436Feb 20, 2026Updated 8 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆3,354Updated this week
- DeepSeek-V4-Flash-0731 on 4x CMP 170HX (sm_80): 98 tok/s decode, ~5300 tok/s prefill. DSpark speculative decoding under pipeline parallel…☆118Aug 13, 2026Updated 2 months ago
- KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.☆1,212Updated this week