1Cat-vLLM on Intel Gaudi
☆47Oct 12, 2026Updated last month
Alternatives and similar repositories for 1Cat-vLLM-Gaudi
Users that are interested in 1Cat-vLLM-Gaudi are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Tested recipes, runtime fixes, kernels and fleet practices for serving LLMs with vLLM on Intel Gaudi☆11Oct 25, 2026Updated 2 weeks ago
- 使用nvidia oam转接卡将intel gaudi2转接到pcie并在ubuntu下正常驱动和使用的指南☆31Nov 4, 2026Updated last week
- DeepSeek-V4.1-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 297 tok/s for one user, up to 868 tok/s at eight, 262K context, one-command c…☆28Nov 2, 2026Updated last week
- marlin_v100 是一个从 vLLM 主树中提取出来的最小 Marlin 独立开发工作区，聚焦于 Marlin dense 与 Marlin MoE 的源码开发、最小构建和轻量验证。它保留了核心 CUDA/C++ 实现、最小 Python 薄封装、生成器测试与主树回写…☆26Jul 2, 2026Updated 4 months ago
- vLLM for NVIDIA CMP 170HX (Ampere sm_80): fast, repeatable model serving on one card or several☆28Nov 2, 2026Updated last week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- vllm混合推理扩展插件，支持多NUMA混合推理，单卡推理Qwen3-Next模型可达1000+ prefill☆34Nov 7, 2025Updated last year
- This project is specifically developed for V100, based on lmdeploy 0.12.1, and supports mainstream open-source models from Q4 2025 to Q1 …☆22Mar 18, 2026Updated 7 months ago
- Run AMD AITER's hand-written ASM kernels on CDNA2 / gfx90a (MI210, MI250). 242 of 1,422 kernels translated, with the tests and benchmarks…☆11Sep 9, 2026Updated 2 months ago
- The world-first Pearl miner for Ascend NPU.☆10Jul 20, 2026Updated 3 months ago
- ☆11Oct 18, 2026Updated 3 weeks ago
- V100 / SM70-focused vLLM engineering fork for modern LLM inference.☆1,494Updated this week
- A fork of jasl/vllm (codex/ds4-sm120-min-enable) with CPU-GPU hybrid inference support for DeepSeek-V4 on SM120+.☆32Sep 10, 2026Updated 2 months ago
- Container driven LLM control for VLLM & llama models☆14Updated this week
- GLM-5.3-Flash on 4× NVIDIA CMP 170HX with vLLM: up to 482.6 tok/s single-user, up to 943.7 tok/s at 8 users, 262K context, TP4 or PP4, on…☆90Oct 26, 2026Updated 2 weeks ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Stable, high-throughput inference server for multi-GPU NVIDIA V100 (SM70): Qwen3.8 Flash-Next with AWQ / NVFP4, multimodal input, CPU/dis…☆52Updated this week
- SGLang for cool dudes using V100☆86Updated this week
- Agentic-coding oriented inference engine for N x V100-SXM2, based on SGLang: GLM-5.3-Flash, Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash, Mini…☆37Oct 7, 2026Updated last month
- Run DeepSeek-V4.1-Flash / DeepSeek-V4-Flash and GLM-5.3-Flash on SM89 (Ada / RTX 4090) and SM120 (RTX PRO 6000) with vLLM☆206Updated this week
- vLLM fork for Tesla V100 (SM70) — extends 1CatAI's AWQ support and adds GGUF support☆27Jun 20, 2026Updated 4 months ago
- NInfer on Tesla V100 (sm_70): Qwen3.8-27B deployment, build/convert scripts, systemd service, and cross-engine benchmark (vs llama.cpp). …☆55Sep 15, 2026Updated last month
- A tool to unlobotomize your NVIDIA card!☆841Oct 28, 2026Updated 2 weeks ago
- Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parall…☆168Updated this week
- vLLM fork for Tesla V100☆15May 31, 2026Updated 5 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- AMD MI210 (gfx90a/CDNA2) LLM inference optimization — TurboQuant, KIVI, per-layer KV types, FlashAttention, MoE expert caching☆17Aug 25, 2026Updated 2 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆38Nov 4, 2026Updated last week
- Implementation of FlashAttention-2 for Nvidia Tesla V100 / Titan V☆218Nov 5, 2026Updated last week
- CPU-GPU hybrid inference for DeepSeek-V4 on NVIDIA SM80+ (A100/RTX 4090 etc.), forked from yhfgyyf/vllm-deepseek-v4-sm89.☆61Sep 16, 2026Updated last month
- .NET wrapper for PJSUA2☆11Oct 20, 2026Updated 3 weeks ago
- MicroPython firmware for the GrowMax automation board - a complete IoT solution for precision agriculture and greenhouse management. Feat…☆27Jun 28, 2025Updated last year
- Hand-written NVFP4 W4A16 CUDA kernels for Volta☆185Aug 19, 2026Updated 2 months ago
- ☆65Aug 6, 2026Updated 3 months ago
- Container toolbox that runs llama.cpp with AMD ROCm and NVIDIA CUDA in one build, for inference on two GPUs☆22Oct 16, 2026Updated 3 weeks ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- llama.cpp optimized for NVIDIA Tesla V100 (SM70). Qwen3.8-Flash-Next (MoE) with MTP on 1/2/3/6x V100-16GB at 262K context (6 GPUs: prefil…☆78Updated this week
- AI Fitness Coach for Claude Code — Garmin / Coros / WHOOP / Apple Watch, Obsidian memory, bilingual (EN/CN) | 支持佳明·高驰·WHOOP·Apple Watch 的…☆34Oct 26, 2026Updated 2 weeks ago
- Pick & Place Challenge: Get a Unitree G1 humanoid to pick up a cylinder and place it on another table. MuJoCo simulation with pre-trained…☆27Apr 7, 2026Updated 7 months ago
- 6800% faster "os" module replacement. A drop-in replacement for Python's standard 'OS' module. Fully-rewritten, optimized, and speeded-up…☆18Feb 20, 2023Updated 3 years ago
- Obleth is a high performance, fairshare AI gateway designed to be placed between the load balancer and an upstream provider like openai, …☆27Updated this week
- ☆11Dec 27, 2023Updated 2 years ago
- LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features …☆474Nov 6, 2026Updated last week
Are these results useful?