LLM inference engine from scratch — paged KV cache, continuous batching, chunked prefill, prefix caching, speculative decoding, CUDA graph, tensor parallelism, OpenAI-compatible serving
☆236Apr 24, 2026Updated 6 months ago
Alternatives and similar repositories for mini-infer
Users that are interested in mini-infer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A comprehensive library management application developed using C++ and SQLite, designed to provide an efficient, lightweight solution for…☆174Mar 19, 2026Updated 7 months ago
- 【ICML 2026 🔥】Knowledge injection method based on knowledge-oriented controls, achieving precision adaptation and powerful retention.☆201Apr 8, 2026Updated 6 months ago
- Controllable, Reproducible, Evaluable Agent Platform☆251Aug 17, 2026Updated 2 months ago
- 【ICLR 2026 🔥】This work introduces MMEVOKE benchmark to reveal challenges in knowledge injection and explores potential solutions.☆216Jun 11, 2025Updated last year
- A lifecycle guard skill.☆179Aug 14, 2026Updated 2 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Universal EAS local builder with configurable Kotlin versions and auto-fixes.☆101Aug 6, 2025Updated last year
- 基于数字人与(微调)大模型的劳动仲裁辅助平台，支持辅助生成仲裁文书与法律咨询。☆137Updated this week
- Coze MCP and Skill Management for OpenClaw☆96Mar 11, 2026Updated 7 months ago
- 社交平台表情包收集☆98Sep 28, 2026Updated last month
- ☆91Mar 23, 2026Updated 7 months ago
- CUDA编程练习项目-Hands-on CUDA kernels and performance optimization, covering GEMM, FlashAttention, Tensor Cores, CUTLASS, quantization, KV cac…☆127May 11, 2026Updated 5 months ago
- Terminal-first AI assistant for software engineering tasks (inspired by Claude Code v2.0.67)☆192Jul 24, 2026Updated 3 months ago
- ☆136Jun 11, 2026Updated 4 months ago
- Muxify is a VSCode extension that allows you to visually manage tmux sessions, windows, and panes directly from the sidebar - no need to …☆80Feb 1, 2026Updated 9 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- utils☆298Apr 6, 2026Updated 6 months ago
- 计算机毕业设计、机器学习毕业设计、深度学习毕业设计、原创AI项目【源码+论文】☆154Mar 29, 2026Updated 7 months ago
- 一个双分支高效排序库☆56Updated this week
- 【ACL 2026 🔥】Temporal Awareness Evaluation, Comprehensive Benchmarking, and Multi-Dimensional Analysis!☆174Oct 30, 2025Updated last year
- AI 小说推文自动化 - 小说一键转短视频（有声书+AI配图），适用于抖音/小红书☆280Aug 18, 2026Updated 2 months ago
- Data and Codes for Our Paper "PEOD: A Pixel-Aligned Event-RGB Benchmark for Object Detection under Challenging Conditions"☆165Aug 25, 2026Updated 2 months ago
- Go-based AI assistant with auto-summary sessions and BM25+RAG memory system☆117Apr 12, 2026Updated 6 months ago
- ☆262Aug 6, 2026Updated 2 months ago
- [ICLR'26] Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling☆95Jan 29, 2026Updated 9 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- **My Automatic Mouse Clicker, Scroller, Keyboard Clicker** is an opensource _personal macro automation tool_ you run on your Windows devi…☆58Feb 27, 2026Updated 8 months ago
- M-Cube (M³) — Multi-thinking, Multimodal, Multi-verification Patent Drafting Assistant☆210May 23, 2026Updated 5 months ago
- MathLens 是一个专注于数学题目视频讲解的 Agent Skill。你只需粘贴一道数学题（图片或文字），它就能自动完成从题目分析、可视化讲解、配音脚本到 Manim 动画视频的全流程制作。单条视频1-10 分钟，成本 0.2-1 元以内。☆359Mar 10, 2026Updated 7 months ago
- 小可の聚集地，由 Next.js、TypeScript、MDX 和 TailwindCSS 构建。My blog built with Next.js, TypeScript, MDX, and TailwindCSS.☆210Sep 22, 2026Updated last month
- Kakobuy Spreadsheet features 3,000+ trending products from Weidian, Taobao, and 1688, with affordable new arrivals added daily. Exp…☆184Mar 21, 2026Updated 7 months ago
- Resource collection of medical agent for clinical dialogue and health☆412Feb 12, 2026Updated 8 months ago
- Plug-and-play nanobot, can be used on Windows 10 (included) or later systems immediately.☆171Mar 14, 2026Updated 7 months ago
- 这是基于多语言所搭建的基于web，工作流，新型算法组件的智能体☆139Oct 11, 2026Updated 3 weeks ago
- 下节啥课：一个无广美观有提醒的课表软件☆64Mar 26, 2026Updated 7 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- python可视化编程IDE(Python Visual Studio Code)☆198Aug 9, 2026Updated 2 months ago
- CNCF openGemini Python Client Library☆130Oct 20, 2025Updated last year
- Marina to OceanBase is Geneva to LanceDB. It supports scalable feature engineering using Python UDFs.☆123Apr 7, 2026Updated 6 months ago
- Golem — The self-evolving GeoAI Agent for the geospatial industry — natural-language GDAL/PostGIS workflows, learned pipeline reuse, fabr…☆200Jul 7, 2026Updated 3 months ago
- [ICLR 2026] "Does FLUX Already Know How to Perform Physically Plausible Image Composition?" (Official Implementation)☆214Mar 26, 2026Updated 7 months ago
- CS 21-26☆59Mar 15, 2026Updated 7 months ago
- A Django mbti test system with user registration, login, logout, and password change functionality☆310Sep 17, 2026Updated last month