InferX: Inference as a Service Platform
☆246Sep 16, 2026Updated this week
Alternatives and similar repositories for inferx
Users that are interested in inferx are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆179Aug 10, 2025Updated last year
- A hybrid router that uses Spot GPU instances to reduce costs and Serverless GPUs for making Cold Starts faster.☆25Mar 13, 2026Updated 6 months ago
- ☆18Jul 1, 2025Updated last year
- Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and i…☆156Updated this week
- ☆19Feb 23, 2026Updated 6 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Why observe computer if computer can observe for you☆1,616Updated this week
- ☆214Sep 7, 2025Updated last year
- TUI for LM Studio☆17Oct 19, 2025Updated 11 months ago
- Single-file, pure CUDA C implementation for running inference on Qwen3 0.6B GGUF. No Dependencies.☆25Nov 26, 2025Updated 9 months ago
- OllaDeck is a purple technology stack for Generative AI (text modality) cybersecurity. It provides a comprehensive set of tools for both …☆17Sep 21, 2024Updated last year
- A high performance batching router optimises max throughput for text inference workload☆16Sep 6, 2023Updated 3 years ago
- Mistral Vibe rewritten in Rust by Devstral 2☆21Dec 23, 2025Updated 8 months ago
- ⛔ DEPRECATED -- use flash-head instead (pip install flash-head)☆29Apr 10, 2026Updated 5 months ago
- Memory-bounded compressed sparse attention via streaming top-k. Triton kernels for the DeepSeek-V4 lightning indexer. 32x regime extensio…☆26May 5, 2026Updated 4 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Sparse Inferencing for transformer based LLMs☆222Mar 25, 2026Updated 5 months ago
- Make Qwen3 Think like Gemini 2.5 Pro | Open webui function☆25May 10, 2025Updated last year
- ☆17Mar 20, 2026Updated 5 months ago
- Practical and advanced guide to LLMOps. It provides a solid understanding of large language models’ general concepts, deployment techniqu…☆85Aug 16, 2024Updated 2 years ago
- Yet another frontend for LLM, written using .NET and WinUI 3☆11Sep 14, 2025Updated last year
- ☆24Dec 29, 2025Updated 8 months ago
- 🔍📃 LLM-powered PDF Table Extractor☆19Jun 26, 2025Updated last year
- A simple but instructive implementation of DP, TP, FSDP, FSDP+TP using pytorch distributed primitives☆24Apr 12, 2026Updated 5 months ago
- Manifold is an experimental platform for enabling long horizon workflow automation using teams of AI assistants.☆497Sep 10, 2026Updated last week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- These are performance benchmarks we did to prepare for our own privacy-preserving and NDA-compliant in-house AI coding assistant. If by a…☆32Apr 2, 2025Updated last year
- PegaFlow is a high-performance KV cache offloading solution for vLLM v1 on single-node multi-GPU setups.☆25Jan 7, 2026Updated 8 months ago
- A simple, "Ollama-like" tool for managing and running GGUF language models from your terminal.☆25Jan 2, 2026Updated 8 months ago
- Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond☆1,421Updated this week
- Self-hosted LLM chatbot arena, with yourself as the only judge☆41Feb 6, 2024Updated 2 years ago
- A repository to store helpful information and emerging insights in regard to LLMs☆21Oct 27, 2023Updated 2 years ago
- Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.☆3,217Updated this week
- Anthropic's Contextual Retrieval implementation with visual chunk comparison. Preview context enrichment before/after embedding.☆31Sep 25, 2025Updated 11 months ago
- KV cache store for distributed LLM inference☆439Updated this week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆35Aug 9, 2026Updated last month
- Achieve state of the art inference performance with modern accelerators on Kubernetes☆4,594Updated this week
- Code for ICML 2025 paper | Joint Localization and Activation Editing for Low-Resource Fine-Tuning☆28Jun 18, 2025Updated last year
- ☆13Feb 18, 2024Updated 2 years ago
- NVIDIA Inference Xfer Library (NIXL)☆1,263Updated this week
- Docker/podman container for llama.cpp/vllm/exllamav{2,3} orchestrated using llama-swap☆18Jun 17, 2026Updated 3 months ago
- ☆17Jun 18, 2024Updated 2 years ago