Reproducible llama.cpp CPU inference profiling and a deterministic LLM serving simulator with continuous batching, KV cache, prefix caching, and workload-driven latency analysis.
☆22Jul 19, 2026Updated 2 months ago
Alternatives and similar repositories for llm-inference-optimization-lab
Users that are interested in llm-inference-optimization-lab are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- The official implementations of Agent-as-a-Router: Agentic Model Routing for Coding Tasks.☆1,045Sep 20, 2026Updated 2 weeks ago
- The agent that remembers and learns.☆1,941Jul 18, 2026Updated 2 months ago
- A Chrome browser plugin that quickly brings web content into Figma, suitable for reference analysis, partial redesign, and interface reus…☆55Sep 1, 2026Updated last month
- 有情绪的 AI 桌面伴侣 | 隐私优先 · 语音唤醒 · 协助办公 · MCP 无限扩展 | An AI desktop companion with emotions — privacy-first, voice wake, office assistance, and …☆664Aug 12, 2026Updated last month
- Official Implementation of OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation☆197Jul 5, 2025Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- bnb four meme | bnb four meme | bnb four meme | bnb four meme | bnb four meme | bnb four meme | bnb four meme | bnb four meme | bnb four …☆159Jul 14, 2026Updated 2 months ago
- dat release☆150Aug 6, 2026Updated 2 months ago
- Your AI usage, quota, and balance — as a desk pet. 🐾 A local companion service that normalizes every AI provider into one LAN feed, plus…☆105Jun 30, 2026Updated 3 months ago
- Self-Supervised Deep Learning for Fisheye Image Rectification (pytorch implementation).☆333Aug 13, 2025Updated last year
- Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention☆2,580Sep 28, 2026Updated last week
- Open-source Telegram server implementation / MTProto server in Go for Telegram Desktop, Android, Web, bots, calls, media, and self-hosted…☆125Updated this week
- hyperliquid copy hyperliquid copy hyperliquid copy hyperliquid copy hyperliquid copy hyperliquid copy hyperliquid copy hyperliquid copy h…☆160Sep 18, 2026Updated 3 weeks ago
- Official DIDA Hotel Booking MCP Server. 14-year travel tech data stack, 2M+ hotels at wholesale rates, 40+ LLM compatible. Free unlimited…☆46Sep 8, 2026Updated last month
- Full-stack open-source interactive long-horizon world model.☆793Sep 30, 2026Updated last week
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- 把 Codex / Petdex 宠物包变成不依赖 Codex 的独立桌宠应用。☆46Jul 13, 2026Updated 2 months ago
- A full-stack monorepo project built with NestJS (backend) and React (admin dashboard) 🚀 一个基于 NestJS 后端与 React 后台管理系统的全栈 Monorepo 项目☆317Sep 30, 2026Updated last week
- An open-source, PyTorch-like runtime for dynamic multi-agent and multi-session workflows.☆961Sep 28, 2026Updated last week
- polymarket polymarket polymarket polymarket polymarket polymarket polymarket polymarket polymarket polymarket polymarket polymarket polym…☆133Updated this week
- 从零到工作流，掌握 AstronClaw（云端）与 Loomy（桌面）的 AI 助手实战教程 | A complete tutorial to master AstronClaw (cloud AI) & Loomy (desktop AI)☆354Oct 3, 2026Updated last week
- The released agentic artifacts for reproduction of position paper: Why Game Development Matters for Scaling World Models☆155Aug 29, 2026Updated last month
- Open-source, local-first desktop AI research workbench for scientific computing with Python/R, MCP bioinformatics tools, SSH/WSL/GPU runt…☆1,023Updated this week
- An Open-Source Algorithm Library of Point Cloud Quality Assessment based on Deep Learning☆184Jun 20, 2026Updated 3 months ago
- Rank-gap statistics for exploring high-dimensional overlap☆359Jan 6, 2026Updated 9 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Compile your documents into a local, linked, LLM-maintained wiki.☆26Jun 9, 2026Updated 4 months ago
- Local-first CLI for agent skills, browser automation, knowledge bases, external CLIs, and terminal AI chat.☆255Sep 9, 2026Updated last month
- ☆1,243Jul 7, 2026Updated 3 months ago
- HTTP, HTTPS, Websocket debugging proxy☆279Sep 2, 2026Updated last month
- "Aurogen🍊: The Multi-Agent Evolution of OpenClaw."☆546Mar 21, 2026Updated 6 months ago
- Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation☆62Jun 7, 2026Updated 4 months ago
- A domain-specific language (DSL) based on Triton but providing higher-level abstractions.☆359Updated this week
- Codex skill for design-system-first frontend UI implementation☆106Jun 27, 2026Updated 3 months ago
- Repo for "Q-Eval-100K: Evaluating Visual Quality and Alignment Level for Text-to-Vision Content"☆63Jun 9, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆68Jul 11, 2026Updated 2 months ago
- BRFv4 - HTML5/Javascript - examples project. Reference implementation for all other platform example packages.☆313Aug 13, 2025Updated last year
- ☆131Jul 13, 2026Updated 2 months ago
- solana trading bot | solana trading bot | solana trading bot | solana trading bot | solana trading bot | solana trading bot | solana trad…☆166Updated this week
- a cross-platform screen casting & control software supporting group control and automated tasks(一款跨平台的移动端屏幕投屏、控制、群控和自动化脚本工具).☆423Jul 27, 2026Updated 2 months ago
- An Open-Source Algorithm Library of Just Noticeable Difference☆229Jun 20, 2026Updated 3 months ago
- Open-source cyber threat intelligence workstation for OSINT triage, Threat Actor Profiles, AI-assisted analysis, and hunt-query generatio…☆46Jul 28, 2026Updated 2 months ago