A distributed inference framework for large language models that routes requests to workers, manages KV cache lifecycle, handles failures gracefully, and applies backpressure so memory — not compute — is the bottleneck.
☆27Oct 6, 2026Updated 2 weeks ago
Alternatives and similar repositories for distributed-inference-engine
Users that are interested in distributed-inference-engine are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Deployment & experimentation of LLM inference at production scale (P/D disaggregation, KV offloading, parallelism, and autoscaling).☆62Sep 26, 2026Updated 3 weeks ago
- Agent Harness prototype with main components and deployment on Railway☆26Aug 21, 2026Updated last month
- Bash script for checking whether affected versions from the `keyv` and `cacheable` npm supply-chain compromise on August 4, 2026☆20Aug 5, 2026Updated 2 months ago
- An open file format for plans and plan schemas using Markdown☆12Jul 6, 2024Updated 2 years ago
- AI Proctoring System☆10May 27, 2021Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Reverse-engineer any video (YouTube, TikTok, Reels, X...) into a reusable blueprint for making similar videos. A Claude Code skill.☆27Oct 11, 2026Updated last week
- Connect all of your agents to everything they need☆30Jun 1, 2026Updated 4 months ago
- Pick a YC batch, get founders and likely emails, write personalised cold emails from one template. Free, stdlib Python + one HTML file, d…☆167Oct 9, 2026Updated last week
- Water's off-limits bruh☆35Updated this week
- ☆12Apr 1, 2025Updated last year
- ☆11Sep 7, 2026Updated last month
- ☆49Sep 25, 2026Updated 3 weeks ago
- Comprehensive Hugging Face Hub Library for rust☆24Sep 23, 2026Updated 3 weeks ago
- ☆32Aug 13, 2026Updated 2 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Claude Code skill: turn your last AI coding session into a credibility-checked, quiz-placed reading path☆17Aug 1, 2026Updated 2 months ago
- Astra Motion Recreation — Remotion. Editable animation source, bundled assets, reproducible renders, and a comparison with the original b…☆29Sep 8, 2026Updated last month
- An AI based optimizer and debugger for python☆12Jul 28, 2024Updated 2 years ago
- Automated scanner for Jio Gemini (Google AI) activation links using exposed Firebase Realtime Database panels and Jio's public APIs.☆49Sep 30, 2026Updated 2 weeks ago
- ☆33Sep 23, 2026Updated 3 weeks ago
- everything you need to know about inference engineering☆43Sep 29, 2026Updated 3 weeks ago
- ☆24Sep 2, 2026Updated last month
- Example apps demonstrating the WRLD Android Java API to display stunning interactive 3D maps. https://www.wrld3d.com/developers/☆13Nov 3, 2022Updated 3 years ago
- ☆16Dec 23, 2024Updated last year
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Generate a YC startup name before someone else does.☆51Aug 30, 2026Updated last month
- Building a Computer From Scratch with verilog☆11Feb 6, 2026Updated 8 months ago
- NodeMuncher is a powerful and user-friendly CLI tool designed to help developers keep their projects clean and clutter-free.☆11Jul 31, 2024Updated 2 years ago
- Demo web server app that shows how BERT model trained on SQuAD dataset deals with the machine comprehension task.☆10Dec 8, 2022Updated 3 years ago
- A skill graph for design engineering: animation, transitions, taste, typography, color, components, anti-patterns.☆34Updated this week
- Car Racer Game in Vanilla JavaScript☆14Mar 31, 2023Updated 3 years ago
- Organize ideas and collaborate using Markdown, a lightweight language for text formatting.☆10May 29, 2024Updated 2 years ago
- Visual DMG editor with animated GIF backgrounds, custom text, and export for Electron, Tauri, and native macOS apps☆27Sep 15, 2026Updated last month
- Repository that explains how the AutoML pipeline can be used for the Richter's Predictor competition on DrivenData☆10May 2, 2019Updated 7 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆57Jul 16, 2026Updated 3 months ago
- LLMWiki Template☆47Aug 9, 2026Updated 2 months ago
- ☆120Oct 7, 2026Updated last week
- Backend Engineering, Low-Level Design, System Design, Networking, Security, SQL, and Distributed Systems explained through visual diagram…☆113Sep 3, 2026Updated last month
- Repo contains lecture notes, code examples, and interactive demos for various topics in AI and programming demonstrated in class.☆11Aug 30, 2024Updated 2 years ago
- 100 days of LLM inference engineering — daily posts, experiments, and visualizations☆1,865Apr 30, 2026Updated 5 months ago
- ☆15Feb 15, 2025Updated last year