InferX: Inference as a Service Platform
☆238Aug 16, 2026Updated this week
Alternatives and similar repositories for inferx
Users that are interested in inferx are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆179Aug 10, 2025Updated last year
- A hybrid router that uses Spot GPU instances to reduce costs and Serverless GPUs for making Cold Starts faster.☆27Mar 13, 2026Updated 5 months ago
- ☆18Jul 1, 2025Updated last year
- ☆1,598Updated this week
- ☆214Sep 7, 2025Updated 11 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- TUI for LM Studio☆17Oct 19, 2025Updated 9 months ago
- Single-file, pure CUDA C implementation for running inference on Qwen3 0.6B GGUF. No Dependencies.☆24Nov 26, 2025Updated 8 months ago
- OllaDeck is a purple technology stack for Generative AI (text modality) cybersecurity. It provides a comprehensive set of tools for both …☆17Sep 21, 2024Updated last year
- A high performance batching router optimises max throughput for text inference workload☆16Sep 6, 2023Updated 2 years ago
- Mistral Vibe rewritten in Rust by Devstral 2☆21Dec 23, 2025Updated 7 months ago
- ⛔ DEPRECATED -- use flash-head instead (pip install flash-head)☆29Apr 10, 2026Updated 4 months ago
- Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extensio…☆24May 5, 2026Updated 3 months ago
- Sparse Inferencing for transformer based LLMs☆220Mar 25, 2026Updated 4 months ago
- ☆17Mar 20, 2026Updated 4 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Practical and advanced guide to LLMOps. It provides a solid understanding of large language models’ general concepts, deployment techniqu…☆83Aug 16, 2024Updated 2 years ago
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆125Updated this week
- Yet another frontend for LLM, written using .NET and WinUI 3☆11Sep 14, 2025Updated 11 months ago
- 🔍📃 LLM-powered PDF Table Extractor☆19Jun 26, 2025Updated last year
- A simple but instructive implementation of DP, TP, FSDP, FSDP+TP using pytorch distributed primitives☆21Apr 12, 2026Updated 4 months ago
- These are performance benchmarks we did to prepare for our own privacy-preserving and NDA-compliant in-house AI coding assistant. If by a…☆32Apr 2, 2025Updated last year
- Manifold is an experimental platform for enabling long horizon workflow automation using teams of AI assistants.☆496Aug 8, 2026Updated last week
- The OTP-faithful actor runtime for Rust: supervision, lifecycle, and crash semantics traced to Erlang/OTP, running as zero-ceremony Tokio…☆28Updated this week
- Pytorch routines for (Ker)nel (Mac)hines☆11Oct 10, 2025Updated 10 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- PegaFlow is a high-performance KV cache offloading solution for vLLM v1 on single-node multi-GPU setups.☆25Jan 7, 2026Updated 7 months ago
- A simple, "Ollama-like" tool for managing and running GGUF language models from your terminal.☆25Jan 2, 2026Updated 7 months ago
- An agent for CUDA compute-communication kernel co-design☆36May 7, 2026Updated 3 months ago
- Self-hosted LLM chatbot arena, with yourself as the only judge☆41Feb 6, 2024Updated 2 years ago
- Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.☆3,174Updated this week
- Anthropic's Contextual Retrieval implementation with visual chunk comparison. Preview context enrichment before/after embedding.☆30Sep 25, 2025Updated 10 months ago
- KV cache store for distributed LLM inference☆430Nov 13, 2025Updated 9 months ago
- ☆34Aug 9, 2026Updated last week
- Achieve state of the art inference performance with modern accelerators on Kubernetes☆4,049Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- NVIDIA Inference Xfer Library (NIXL)☆1,195Updated this week
- Generate audio datasets for training Text-To-Speech models, through smart audio splitting with silence detection, and transcription using…☆30May 27, 2023Updated 3 years ago
- Vibe Coded Project Management System☆21Apr 19, 2025Updated last year
- Scrollable infinity terminal app☆15Aug 8, 2026Updated last week
- Docker/podman container for llama.cpp/vllm/exllamav{2,3} orchestrated using llama-swap☆18Jun 17, 2026Updated 2 months ago
- ☆16Jun 18, 2024Updated 2 years ago
- Model Context Protocol for Research tasks☆16Apr 1, 2025Updated last year