Tutorial for LLM developers about engine design, service deployment, evaluation/benchmark, etc. Provide a C/S style optimized LLM inference engine.
☆19Sep 5, 2023Updated 3 years ago
Alternatives and similar repositories for LLM-Inference-Deployment-Tutorial
Users that are interested in LLM-Inference-Deployment-Tutorial are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- 🚀 Automatically convert unstructured data into a high-quality 'textbook' format, optimized for fine-tuning Large Language Models (LLMs)☆26Oct 15, 2023Updated 3 years ago
- （已完结）斯坦福大学机器学习项目MiniTorch解读——《MiniTorch 学习全攻略》 的随书源码。☆10Jul 16, 2022Updated 4 years ago
- Website for Stanford SysML Seminar☆17Oct 27, 2025Updated last year
- ☆30Jun 13, 2023Updated 3 years ago
- A Overview of Efficiently Serving Foundation Models across Edge Devices☆15Jan 17, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and L…☆23Apr 12, 2024Updated 2 years ago
- ☆12Dec 30, 2025Updated 9 months ago
- Chat with your Database! Natural language to SQL with a friendly UI. LangChain+Streamlit+SQL Agents with SQLAlchemy wrap-up (BigQuery/MyS…☆26Dec 7, 2023Updated 2 years ago
- Codes of the paper Deformable Butterfly: A Highly Structured and Sparse Linear Transform.☆16Nov 1, 2021Updated 4 years ago
- TaskWeaver Plugins☆12Jan 28, 2024Updated 2 years ago
- 🧑🚀 Professional translation and reading of English academic papers in PDF format.☆12Nov 2, 2023Updated 2 years ago
- ☆31Jul 31, 2026Updated 2 months ago
- ☆19Sep 15, 2022Updated 4 years ago
- ATC23 AE☆45May 11, 2023Updated 3 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Code showing how to use a model based on the ML model base class.☆10Sep 30, 2022Updated 4 years ago
- Scripts for reading, extracting, and organizing data from either HTML or PDF documents and prepare them to be converted into embeddings f…☆13Aug 26, 2024Updated 2 years ago
- ☆16Jan 14, 2025Updated last year
- Kafka Sink connector for Qdrant. Stream vector data into a Qdrant collection.☆13Updated this week
- This repository will take you through creating a FastAPI StableDiffusion app (including Dockerfile) all the way to adding a new feature u…☆38Nov 9, 2022Updated 3 years ago
- Welcome to our SaaS AI platform repository! 🚀 Our SaaS AI platform is designed to empower businesses with cutting-edge Artificial Intel…☆17Jul 22, 2023Updated 3 years ago
- Welcoming contributions from practitioners building AI/data systems - share your real-world problems, document where current tools fail, …☆14Sep 4, 2025Updated last year
- A compute framework for building Search, RAG, Recommendations and Analytics over complex (structured+unstructured) data, with ultra-modal…☆12Sep 16, 2024Updated 2 years ago
- Google Cloud Platform (GCP) CLI and utils☆14May 6, 2023Updated 3 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- A TypeScript Library for Linear Algebra and Machine Learning☆10Dec 10, 2022Updated 3 years ago
- ☆10Nov 26, 2025Updated 11 months ago
- An example project using a tensorflow model and Movidius NCS to do object recognition RPI camera module stream☆12Jun 27, 2018Updated 8 years ago
- This project scrapes text from Telugu books(Novels)☆11Aug 3, 2021Updated 5 years ago
- This is code depository for my upcoming session. Will update details post the session☆41Jan 29, 2023Updated 3 years ago
- pytorch code examples for measuring the performance of collective communication calls in AI workloads☆21Sep 18, 2025Updated last year
- Implementations of transformer models in pytorch☆14Jun 2, 2020Updated 6 years ago
- HyDE based RAG using NVIDIA NIM.☆16Mar 20, 2024Updated 2 years ago
- ☆14Oct 22, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A simple example to showcase machine learning model deployment with an API☆10Mar 7, 2022Updated 4 years ago
- Contains all the "handout" materials for my Python for Entrepreneurs course. This includes notes and the final version of the website cod…☆12Jul 31, 2018Updated 8 years ago
- This is a tutorial I made on how to deploy a HuggingFace/LangChain pipeline on the newly released Falcon 7B LLM by TII☆10Jul 2, 2023Updated 3 years ago
- ☆16Sep 10, 2024Updated 2 years ago
- UI for Pandas AI, the Python library that makes dataframes conversational.☆14Jun 6, 2023Updated 3 years ago
- ☆10Oct 22, 2026Updated last week
- Simple version of a Vanna streamlit app☆17Mar 7, 2024Updated 2 years ago