A collection of all available inference solutions for the LLMs
☆95Mar 1, 2025Updated last year
Alternatives and similar repositories for llm-inference-solutions
Users that are interested in llm-inference-solutions are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A sleek, customizable interface for managing LLMs with responsive design and easy agent personalization.☆19Aug 30, 2024Updated last year
- ☆51May 31, 2024Updated 2 years ago
- A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.☆190Mar 23, 2026Updated 4 months ago
- This Guidance demonstrates how to deploy a machine learning inference architecture on Amazon Elastic Kubernetes Service (Amazon EKS). It …☆45May 29, 2025Updated last year
- A Text2Cypher ReAct agent built with LangGraph and the Neo4j Cypher MCP server.☆24Nov 14, 2025Updated 8 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Tiny Calculator with support of +, -, *, /, ^, sin, cos, tan...☆10Apr 2, 2024Updated 2 years ago
- ☆14Feb 7, 2024Updated 2 years ago
- Notes and Examples to get started Parallel Computing with CUDA.☆13Nov 1, 2019Updated 6 years ago
- Inference Llama 2 with a model compiled to native code by TorchInductor☆14Feb 8, 2024Updated 2 years ago
- A model serving framework for various research and production scenarios. Seamlessly built upon the PyTorch and HuggingFace ecosystem.☆23Oct 11, 2024Updated last year
- 5X faster 60% less memory QLoRA finetuning☆21May 28, 2024Updated 2 years ago
- a phone to app implementation using Asterisk and Node.js☆15Oct 27, 2014Updated 11 years ago
- Kubernetes 中自动替换墙外镜像的下载地址☆24May 11, 2020Updated 6 years ago
- Probably one of the lightest native RAG + Agent apps out there，experience the power of Agent-powered models and Agent-driven knowledge ba…☆33May 30, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- These are papers that I read and reviewed related to NLP, CV, and Deep Learning 😉 You can check paper links and my reviews 😊☆13Jan 3, 2024Updated 2 years ago
- Retrieval-augmented Image Captioning☆13Feb 16, 2023Updated 3 years ago
- ☆59Aug 19, 2025Updated 11 months ago
- ☆12Apr 4, 2024Updated 2 years ago
- NixOps VirtualBox backend [maintainer=@AmineChikhaoui]☆24Aug 10, 2023Updated 3 years ago
- Simple GUI to load a PDF/Docx/txt file and have LM Studio Answer based off of it.☆14Jul 31, 2024Updated 2 years ago
- [NeurIPS 2024] Artemis: Towards Referential Understanding in Complex Videos☆27Apr 8, 2025Updated last year
- This repository contains some examples of using borb in google colab. These examples enable you to try out the features of borb without i…☆13Sep 4, 2022Updated 3 years ago
- A benchmark and playground for Completely Fair Scheduling in Go☆11Feb 12, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆274Aug 6, 2025Updated last year
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆3,823May 28, 2026Updated 2 months ago
- A simple no-install web UI for Ollama and OAI-Compatible APIs!☆31Jan 30, 2025Updated last year
- Time series forecasting with DuckDB and Evidence☆43Nov 1, 2024Updated last year
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆251Mar 15, 2024Updated 2 years ago
- llama INT4 cuda inference with AWQ☆54Jan 20, 2025Updated last year
- Faker Extension☆15Jan 29, 2016Updated 10 years ago
- ALCF benchmarks for coupled simulation and AI workflows☆17Dec 11, 2025Updated 8 months ago
- Chain-of-thought 방식을 활용하여 llama2를 fine-tuning☆10Nov 18, 2023Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Content based Recommendation☆14Jun 23, 2021Updated 5 years ago
- ☆17Jun 9, 2024Updated 2 years ago
- Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.☆867Jan 15, 2024Updated 2 years ago
- An Open Source Playground with Agent Datasets and APIs for building and testing your own Autonomous Web Agents☆206Dec 4, 2023Updated 2 years ago
- A curated list of my GitHub stars!☆25Updated this week
- Faster X-ray Bragg Peak Analysis with Deep Learning☆16Apr 12, 2022Updated 4 years ago
- ☆20Feb 23, 2026Updated 5 months ago