Alternatives and similar repositories for llm-inference-in-production

Users that are interested in llm-inference-in-production are comparing it to the libraries listed below

• Brandon-c-tech / RAG-logger
  RAG Logger is an open-source logging tool designed specifically for Retrieval-Augmented Generation (RAG) applications. It serves as a lig…
  ☆222Updated 6 months ago
• arc53 / llm-price-compass
  This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient …
  ☆221Updated 6 months ago
• simonw / llm-hacker-news
  LLM plugin for pulling content from Hacker News
  ☆114Updated 2 months ago
• taylorai / aiq
  ai for jq
  ☆243Updated 9 months ago
• RedPlanetHQ / core
  Your personal plug and play memory layer for LLMs
  ☆336Updated this week
• M4THYOU / TokenDagger
  High-Performance Implementation of OpenAI's TikToken.
  ☆432Updated last week
• janito-dev / janito
  Large Language Model Thin Client agent
  ☆104Updated this week
• SureScaleAI / cleverbee
  CleverBee - The Open Source Deep Researcher Tool
  ☆302Updated last month
• banagale / FileKitty
  GUI for selecting text files for concatenation and submission to LLMs
  ☆176Updated this week
• Z-Gort / Reservoirs-Lab
  ☆278Updated last month
• ggozad / haiku.rag
  Retrieval Augmented Generation based on SQLite
  ☆253Updated this week
• jamsocket / forevervm
  Securely run AI-generated code in stateful sandboxes that run forever.
  ☆205Updated 2 months ago
• agentsea / r1-computer-use
  Applying the ideas of Deepseek R1 to computer use
  ☆214Updated 5 months ago
• aerugo / prelude
  A very simple tool to build LLM prompts from your code repositories.
  ☆153Updated 4 months ago
• merliot / hub
  Merliot Device Hub
  ☆144Updated last month
• slashml / amd_inference
  Docker-based inference engine for AMD GPUs
  ☆231Updated 9 months ago
• klara-research / klarity
  See Through Your Models
  ☆398Updated this week
• andyk / recursive_llm
  Implement recursion using English as the programming language and an LLM as the runtime.
  ☆237Updated 2 years ago
• dipampaul17 / KVSplit
  Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit …
  ☆356Updated last month
• janwilmake / efficient-recorder
  Attempt to create an Open Source Privacy Focused Rewind.ai Alternative for data capture
  ☆215Updated 5 months ago
• tearflake / flake-ui
  fractal-structure inspired, parent-children orbiting, zooming-elements based interactive graph visualization user interface
  ☆131Updated 4 months ago
• dhealy05 / frames_of_mind
  Animating R1's thoughts.
  ☆383Updated 4 months ago
• QDScholium / ScholiumAI
  Your AI research assistant
  ☆79Updated 3 months ago
• mcpgod / cli
  Fine-grained control over model context protocol (MCP) clients, servers, and tools. Context is God.
  ☆111Updated last month
• matiasmolinas / evolving-agents
  Your toolkit for autonomous, evolving agent ecosystems. Create, execute, govern, and evolve agents that learn from experience, collaborat…
  ☆439Updated last week
• stanford-mast / blast
  Browser-LLM Auto-Scaling Technology
  ☆528Updated this week
• Foreseerr / TScale
  ☆196Updated 2 months ago
• anordin95 / run-llama-locally
  Run and explore Llama models locally with minimal dependencies on CPU
  ☆191Updated 9 months ago
• fiddlecube / compliant-llm
  Build Secure and Compliant AI agents and MCP Servers. YC W23
  ☆143Updated last month
• hemanth / paws-on-mcp
  A comprehensive Model Context Protocol (MCP) server implementing the latest specification.
  ☆326Updated 3 weeks ago