bentoml / llm-inference-handbookLinks
Everything you need to know about LLM inference
☆232Updated this week
Alternatives and similar repositories for llm-inference-handbook
Users that are interested in llm-inference-handbook are comparing it to the libraries listed below
Sorting:
- High-Performance Implementation of OpenAI's TikToken.☆451Updated 2 months ago
- Securely run AI-generated code in stateful sandboxes that run forever.☆219Updated 5 months ago
- Content addressable storage with excellent search☆348Updated this week
- RAG Logger is an open-source logging tool designed specifically for Retrieval-Augmented Generation (RAG) applications. It serves as a lig…☆225Updated 8 months ago
- Git Based Memory Storage for Conversational AI Agent☆608Updated last week
- See Through Your Models☆400Updated 2 months ago
- Pixelagent — Multimodal stateful agents☆217Updated 3 months ago
- Parallel thinking for LLMs. Confidence‑gated, strategy‑driven, offline ‑friendly☆242Updated 3 weeks ago
- A Python toolkit for chain-of-thought prompting 🐍☆172Updated 3 weeks ago
- LLM plugin for pulling content from Hacker News☆117Updated 4 months ago
- This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient …☆219Updated 9 months ago
- Your filesystem as a vector database☆471Updated 4 months ago
- ☆196Updated 4 months ago
- Your toolkit for autonomous, evolving agent ecosystems. Create, execute, govern, and evolve agents that learn from experience, collaborat…☆443Updated last month
- A comprehensive Model Context Protocol (MCP) server implementing the latest specification.☆333Updated 2 months ago
- Build data processing and data analysis pipelines that leverage the power of LLMs 🧠☆203Updated this week
- Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-…☆115Updated 7 months ago
- Applying the ideas of Deepseek R1 to computer use☆216Updated 7 months ago
- CleverBee - The Open Source Deep Researcher Tool☆307Updated 3 months ago
- Physical AI Assistant that illuminates your life☆162Updated last month
- ☆151Updated 2 months ago
- Heirarchical Navigable Small Worlds☆101Updated last month
- Animating R1's thoughts.☆384Updated 7 months ago
- EnrichMCP is a python framework for building data driven MCP servers☆611Updated 2 weeks ago
- Build Secure and Compliant AI agents and MCP Servers. YC W23☆151Updated 3 months ago
- Visual inference exploration & experimentation playground☆95Updated 9 months ago
- ☆280Updated 3 weeks ago
- Run and explore Llama models locally with minimal dependencies on CPU☆190Updated 11 months ago
- VSCode extension that demonstrates the use of large language models (LLMs) for active debugging of programs☆353Updated 7 months ago
- A GTK graphical interface for chatting with large language models (LLMs)☆80Updated 2 weeks ago