InferX: Inference as a Service Platform
☆230Jul 23, 2026Updated this week
Alternatives and similar repositories for inferx
Users that are interested in inferx are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆180Aug 10, 2025Updated 11 months ago
- A hybrid router that uses Spot GPU instances to reduce costs and Serverless GPUs for making Cold Starts faster.☆27Mar 13, 2026Updated 4 months ago
- ☆18Jul 1, 2025Updated last year
- ☆1,585Updated this week
- ☆214Sep 7, 2025Updated 10 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- TUI for LM Studio☆17Oct 19, 2025Updated 9 months ago
- Single-file, pure CUDA C implementation for running inference on Qwen3 0.6B GGUF. No Dependencies.☆24Nov 26, 2025Updated 8 months ago
- OllaDeck is a purple technology stack for Generative AI (text modality) cybersecurity. It provides a comprehensive set of tools for both …☆17Sep 21, 2024Updated last year
- A high performance batching router optimises max throughput for text inference workload☆16Sep 6, 2023Updated 2 years ago
- ⛔ DEPRECATED -- use flash-head instead (pip install flash-head)☆29Apr 10, 2026Updated 3 months ago
- Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extensio…☆22May 5, 2026Updated 2 months ago
- Sparse Inferencing for transformer based LLMs☆220Mar 25, 2026Updated 4 months ago
- Make Qwen3 Think like Gemini 2.5 Pro | Open webui function☆25May 10, 2025Updated last year
- ☆17Mar 20, 2026Updated 4 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Practical and advanced guide to LLMOps. It provides a solid understanding of large language models’ general concepts, deployment techniqu…☆82Aug 16, 2024Updated last year
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆94Updated this week
- Yet another frontend for LLM, written using .NET and WinUI 3☆11Sep 14, 2025Updated 10 months ago
- ☆24Dec 29, 2025Updated 7 months ago
- 🔍📃 LLM-powered PDF Table Extractor☆19Jun 26, 2025Updated last year
- A simple but instructive implementation of DP, TP, FSDP, FSDP+TP using pytorch distributed primitives☆19Apr 12, 2026Updated 3 months ago
- Manifold is an experimental platform for enabling long horizon workflow automation using teams of AI assistants.☆498Updated this week
- Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond☆1,119Updated this week
- The OTP-faithful actor runtime for Rust: supervision, lifecycle, and crash semantics traced to Erlang/OTP, running as zero-ceremony Tokio…☆28Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Pytorch routines for (Ker)nel (Mac)hines☆12Oct 10, 2025Updated 9 months ago
- PegaFlow is a high-performance KV cache offloading solution for vLLM v1 on single-node multi-GPU setups.☆25Jan 7, 2026Updated 6 months ago
- A simple, "Ollama-like" tool for managing and running GGUF language models from your terminal.☆25Jan 2, 2026Updated 6 months ago
- An agent for CUDA compute-communication kernel co-design☆35May 7, 2026Updated 2 months ago
- Self-hosted LLM chatbot arena, with yourself as the only judge☆41Feb 6, 2024Updated 2 years ago
- A repository to store helpful information and emerging insights in regard to LLMs☆21Oct 27, 2023Updated 2 years ago
- Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.☆3,149Jul 20, 2026Updated last week
- Anthropic's Contextual Retrieval implementation with visual chunk comparison. Preview context enrichment before/after embedding.☆30Sep 25, 2025Updated 10 months ago
- KV cache store for distributed LLM inference☆423Nov 13, 2025Updated 8 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆32Jul 19, 2026Updated last week
- Achieve state of the art inference performance with modern accelerators on Kubernetes☆3,898Updated this week
- Code for ICML 2025 paper | Joint Localization and Activation Editing for Low-Resource Fine-Tuning☆28Jun 18, 2025Updated last year
- NVIDIA Inference Xfer Library (NIXL)☆1,155Updated this week
- ☆13Feb 18, 2024Updated 2 years ago
- Vibe Coded Project Management System☆21Apr 19, 2025Updated last year
- Generate audio datasets for training Text-To-Speech models, through smart audio splitting with silence detection, and transcription using…☆30May 27, 2023Updated 3 years ago