A collection of all available inference solutions for the LLMs
☆94Mar 1, 2025Updated last year
Alternatives and similar repositories for llm-inference-solutions
Users that are interested in llm-inference-solutions are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆51May 31, 2024Updated 2 years ago
- ☆15Jul 10, 2026Updated 4 months ago
- Kubernetes 中的 gRPC 负载均衡☆13Dec 15, 2021Updated 4 years ago
- A Text2Cypher ReAct agent built with LangGraph and the Neo4j Cypher MCP server.☆25Nov 14, 2025Updated last year
- Tiny Calculator with support of +, -, *, /, ^, sin, cos, tan...☆10Apr 2, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- A sleek, customizable interface for managing LLMs with responsive design and easy agent personalization.☆19Aug 30, 2024Updated 2 years ago
- a phone to app implementation using Asterisk and Node.js☆15Oct 27, 2014Updated 12 years ago
- ☆14Feb 7, 2024Updated 2 years ago
- 🌍 LLOT - Local LLM Ollama Translator | Privacy-first translation with 40+ languages | Self-hosted☆14Updated this week
- 5X faster 60% less memory QLoRA finetuning☆21May 28, 2024Updated 2 years ago
- A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.☆192Mar 23, 2026Updated 7 months ago
- ☆10Apr 12, 2019Updated 7 years ago
- ☆12Feb 25, 2023Updated 3 years ago
- LCM OpenVINO model converter☆25Mar 27, 2024Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Inference Llama 2 with a model compiled to native code by TorchInductor☆14Feb 8, 2024Updated 2 years ago
- A model serving framework for various research and production scenarios. Seamlessly built upon the PyTorch and HuggingFace ecosystem.☆23Oct 11, 2024Updated 2 years ago
- The PSC particle-in-cell code☆11May 2, 2016Updated 10 years ago
- 🤖 AI-powered CLI for file reorganization. Runs fully locally — no data leaves your machine.☆20Jul 2, 2025Updated last year
- A benchmark and playground for Completely Fair Scheduling in Go☆11Feb 12, 2022Updated 4 years ago
- 🎓Automatically Update CV Papers Daily using Github Actions (Update Every 12th hours)☆12May 17, 2026Updated 6 months ago
- Dataset Resplitting for Generalization in KGQA. See also https://github.com/semantic-systems/KGQA-datasets☆17Jun 29, 2022Updated 4 years ago
- LaTex support to Marked☆10Updated this week
- Agent application/benchmark/workload traces should be placed here.☆20Apr 13, 2026Updated 7 months ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Screen Operator - Android app that operates the screen with vision LLMs☆16Updated this week
- Chain-of-thought 방식을 활용하여 llama2를 fine-tuning☆10Nov 18, 2023Updated 2 years ago
- lightsocks client implements by golang☆13Sep 11, 2015Updated 11 years ago
- A micro LLM multi-agent system for data analysis☆17Apr 27, 2025Updated last year
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆3,835May 28, 2026Updated 5 months ago
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆253Mar 15, 2024Updated 2 years ago
- A REPL-oriented multi-line Readline, without full-screen UI.☆12May 9, 2026Updated 6 months ago
- Shell scripts and desktop configurations of my daily life☆17Oct 30, 2026Updated 2 weeks ago
- This repo showcases a project that transforms ML model training into a simplified, production-ready Kedro Dockerized Pipeline. It emphasi…☆10Feb 12, 2025Updated last year
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Unity 3D / C# / Visual Studio - Original house visit development - 'Archviz' - version 2.0 - (Download links for Windows and MacOSX)☆10Aug 29, 2023Updated 3 years ago
- Offline-first, desktop AI assistant tailored for educators, enabling them to generate questions directly from source materials.☆24Aug 2, 2025Updated last year
- ☆10Sep 19, 2019Updated 7 years ago
- Cray-LM unified training and inference stack.☆22Jan 30, 2025Updated last year
- ALCF benchmarks for coupled simulation and AI workflows☆17Dec 11, 2025Updated 11 months ago
- llama INT4 cuda inference with AWQ☆55Jan 20, 2025Updated last year
- Nexusflow function call, tool use, and agent benchmarks.☆28Dec 13, 2024Updated last year
Are these results useful?