Tutorial for LLM developers about engine design, service deployment, evaluation/benchmark, etc. Provide a C/S style optimized LLM inference engine.
☆19Sep 5, 2023Updated 2 years ago
Alternatives and similar repositories for LLM-Inference-Deployment-Tutorial
Users that are interested in LLM-Inference-Deployment-Tutorial are comparing it to the libraries listed below
Sorting:
- ☆16Sep 4, 2023Updated 2 years ago
- Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and L…☆17Apr 12, 2024Updated last year
- Website for Stanford SysML Seminar☆17Oct 27, 2025Updated 4 months ago
- Chat with your Database! Natural language to SQL with a friendly UI. LangChain+Streamlit+SQL Agents with SQLAlchemy wrap-up (BigQuery/MyS…☆24Dec 7, 2023Updated 2 years ago
- Code associated with the paper **Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees**.☆28Apr 25, 2023Updated 2 years ago
- ☆31Jun 13, 2023Updated 2 years ago
- Super Mario is a legendary game we all cherish! In this project, we will deploy Super Mario on Amazon EKS (Elastic Kubernetes Service) us…☆11Feb 3, 2026Updated 3 weeks ago
- This repository will take you through creating a FastAPI StableDiffusion app (including Dockerfile) all the way to adding a new feature u…☆38Nov 9, 2022Updated 3 years ago
- Multiprocessing in python☆10Aug 20, 2021Updated 4 years ago
- This is the official repository for the paper "Words That Unite The World: A Unified Framework for Deciphering Global Central Bank Commun…☆17Oct 19, 2025Updated 4 months ago
- ☆16May 26, 2025Updated 9 months ago
- This is code depository for my upcoming session. Will update details post the session☆40Jan 29, 2023Updated 3 years ago
- ☆12Jun 17, 2023Updated 2 years ago
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Apr 28, 2023Updated 2 years ago
- Free tool to copy CSVs from https://chartink.com/☆15Sep 7, 2025Updated 5 months ago
- Code accompanying the NeurIPS 2019 paper AutoAssist: A Framework to Accelerate Training of Deep Neural Networks.☆14Oct 3, 2022Updated 3 years ago
- It gives you a step by step approach to predict binary data using linear regression.☆11Feb 28, 2021Updated 5 years ago
- ☆11Apr 8, 2024Updated last year
- netbeacon - monitoring your network capture, NIDS or network analysis process☆19Oct 26, 2013Updated 12 years ago
- Google Cloud Platform (GCP) CLI and utils☆14May 6, 2023Updated 2 years ago
- ☆10Aug 12, 2024Updated last year
- Integrating with Spotify API and extracting Data. Deploying code on AWS Lambda for Data Extraction. Adding trigger to run the extraction …☆11Jul 5, 2023Updated 2 years ago
- Example repo to kickstart integration with mlflow recipes.☆46Aug 1, 2025Updated 6 months ago
- ☆11Oct 19, 2018Updated 7 years ago
- ☆15Jul 18, 2023Updated 2 years ago
- Machine Learning for Internet of Things☆12Jul 24, 2019Updated 6 years ago
- ☆13Jan 21, 2022Updated 4 years ago
- Python script to automate AMI backup of AWS EC2 instances☆13May 31, 2014Updated 11 years ago
- Chaitin-Briggs register-allocation algorithm (LLVM back-end)☆12Jan 6, 2016Updated 10 years ago
- ☆20Jul 29, 2024Updated last year
- ☆11Oct 21, 2023Updated 2 years ago
- ☆12May 18, 2024Updated last year
- Code showing how to use a model based on the ML model base class.☆10Sep 30, 2022Updated 3 years ago
- ☆10Jun 4, 2021Updated 4 years ago
- Distributed, Replicated, Protocol-generic Key-value Store in Async Rust For SMR Protocols Research☆17Feb 10, 2026Updated 2 weeks ago
- Saturn Cloud starter code for LLM Zoomcamp☆11Jul 1, 2024Updated last year
- A tool for cross-checking Verilog compilers☆14Apr 16, 2025Updated 10 months ago
- ☆21Apr 28, 2025Updated 10 months ago
- ☆11Apr 3, 2023Updated 2 years ago