Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.
☆197Oct 30, 2026Updated this week
Alternatives and similar repositories for modelexpress
Users that are interested in modelexpress are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling☆285Updated this week
- NVIDIA Inference Xfer Library (NIXL)☆1,350Updated this week
- ☆27Updated this week
- llm-d-batch-gateway is a standalone, backend-agnostic, OpenAI-compatible batch API and processing engine☆28Updated this week
- ☆401Updated this week
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆66Updated this week
- A workload for deploying LLM inference services on Kubernetes☆313Updated this week
- Offline optimization of your disaggregated Dynamo graph☆461Updated this week
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆838Updated this week
- ☆95Updated this week
- Scheduler-Plugins maintains a list of kubernetes scheduler plugins used in InftyAI community.☆11Jun 16, 2025Updated last year
- NVSentinel detects and remediates GPU faults on Kubernetes nodes☆421Updated this week
- A Datacenter Scale Distributed Inference Serving Framework☆8,416Updated this week
- The main purpose of runtime copilot is to assist with node runtime management tasks such as configuring registries, upgrading versions, i…☆13May 16, 2023Updated 3 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Benchmark SGLang on SLURM☆25Apr 20, 2026Updated 6 months ago
- AISimulate predicts LLM serving behavior and searches for strong deployment configurations offline, without bringing up a GPU serving clu…☆114Updated this week
- htop-like TUI for real-time RDMA network monitoring.☆125Updated this week
- ☆13Feb 11, 2025Updated last year
- LeaderWorkerSet & DisaggregatedSet: Kubernetes APIs for Distributed and Disaggregated AI Serving☆898Updated this week
- Kubernetes controllers for fast model actuation using vLLM sleep/wake and launcher-based model swapping☆28Updated this week
- Snapshot gets GPU pods ready in seconds instead of minutes — by restoring a fully initialized GPU worker instead of starting one from scr…☆76Updated this week
- 🧯 Kubernetes coverage for fault awareness and recovery, works for any LLMOps, MLOps, AI workloads.☆35Updated this week
- Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…☆522Updated this week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale☆1,625Updated this week
- Tooling for optimized, validated, and reproducible GPU-accelerated AI runtime in Kubernetes☆450Updated this week
- KV cache store for distributed LLM inference☆449Sep 16, 2026Updated last month
- GenAI inference performance benchmarking tool☆267Updated this week
- JobSet: a k8s native API for distributed ML training and HPC workloads☆347Updated this week
- Validation and management tools for NVIDIA AI Cloud environments.☆34Oct 25, 2026Updated last week
- llm-d Router: The intelligent entry point for inference requests☆440Updated this week
- A toolkit for discovering cluster network topology.☆193Updated this week
- Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini &…☆695Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- High-performance safetensors model loader☆181Oct 17, 2026Updated 2 weeks ago
- State of the art inference performance with modern accelerators on Kubernetes☆5,119Updated this week
- High-performance RDMA for Python and PyTorch (ibverbs, efa, NVMe-oF)☆14Sep 2, 2026Updated last month
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆27Apr 24, 2025Updated last year
- Gateway API Inference Extension☆799Oct 19, 2026Updated last week
- Open Source AI Accelerator Research Platform Standard / 开源推理研究平台☆1,956Updated this week
- ☆12May 22, 2023Updated 3 years ago