Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.
☆22Aug 11, 2026Updated last month
Alternatives and similar repositories for Efficient-LLM-Inference-Serving-Systems
Users that are interested in Efficient-LLM-Inference-Serving-Systems are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- CastleHill: Separable Causal Diffusion / Varitaion Flow Maps for LTX-2 long-form video generation☆15May 19, 2026Updated 4 months ago
- ☆11Feb 22, 2025Updated last year
- Full End-to-End examples showing how to use First-gen Gaudi and Gaudi2 in common use cases☆13Dec 2, 2024Updated last year
- CLIP is an open source, multimodal computer vision model and it's awesome!☆17Dec 16, 2024Updated last year
- ☆15Apr 14, 2025Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- VidKV: Plug-and-Play 1.x-Bit KV Cache Quantization for Video Large Language Models☆25Mar 26, 2025Updated last year
- Iterate fast on your RAG pipelines☆24Jun 21, 2025Updated last year
- Source code for SWIFT, an efficient reward model.☆21Jan 13, 2026Updated 8 months ago
- Repository for ACM India Summer School on Generative AI for Text☆13Jul 11, 2024Updated 2 years ago
- Replication package for evaluation of code generation metrics☆17Nov 24, 2025Updated 10 months ago
- Training HuggingFace models using fastai☆11Jul 22, 2021Updated 5 years ago
- ☆15Aug 26, 2023Updated 3 years ago
- Example of using the SAINT architecture in fastai☆12Aug 25, 2021Updated 5 years ago
- World Models That Know When They Don't Know: Controllable Video Generation with Calibrated Uncertainty☆26Sep 1, 2026Updated last month
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Use the Google Cloud Speech API to transcribe audio files from a podcast.☆21May 17, 2017Updated 9 years ago
- Triton implementation of GPT/LLAMA☆21Aug 28, 2024Updated 2 years ago
- IET's Hostel Management App☆25Feb 20, 2024Updated 2 years ago
- Tricks used in deep learning. Including papers read recently.☆16Jul 10, 2018Updated 8 years ago
- Implementation of IODINE model☆10Jun 7, 2019Updated 7 years ago
- ☆10Sep 26, 2019Updated 7 years ago
- Utilizing nbdev in Google Colaboratory☆14Apr 12, 2023Updated 3 years ago
- A curated list of resources dedicated to Code-mixed Natural Language Processing (NLP).☆18Sep 27, 2026Updated last week
- Mobile Viewer for W&B, built on top of Flutter.☆41Mar 2, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- (NeurIPS-2019 MicroNet Challenge - 3rd Winner) Open source code for "SIPA: A simple framework for efficient networks"☆19Dec 18, 2022Updated 3 years ago
- Use AI to preview how garments look on you directly on product pages from Amazon and Coupang. Upload your photo, click "Try On," and see …☆22Apr 13, 2025Updated last year
- DREN:Deep Rotation Equivirant Network☆16Mar 24, 2019Updated 7 years ago
- ☆11Apr 27, 2019Updated 7 years ago
- Brain-Box is a web application that allows students to organize and manage their study materials, including subjects, chapters, notes, an…☆27Feb 20, 2024Updated 2 years ago
- Get images from MangaPanda, MangaReader or MangaFox and save it for offline reading.☆11Jul 5, 2016Updated 10 years ago
- Code for "Simulated Multiple Reference Training Improves Low-Resource Machine Translation"☆15Dec 1, 2020Updated 5 years ago
- Implemention of the GRASP world model planner for dino_wm environments☆36Apr 6, 2026Updated 6 months ago
- treemind interprets tree models☆41Apr 9, 2026Updated 6 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Implementation of <Orthogonal Model Merging>☆37Aug 5, 2026Updated 2 months ago
- ☆37May 11, 2024Updated 2 years ago
- CNN ensemble for prostate cancer Gleason grading☆20Jan 28, 2026Updated 8 months ago
- ☆40Jun 19, 2024Updated 2 years ago
- 🔍 GemInsights: Unleash Gemini AI on your data! 🚀 Analyze dataframes for valuable insights, replacing traditional data analysis. 📊 A c…☆20Aug 25, 2026Updated last month
- ☆33Sep 30, 2026Updated last week
- Python code for Bayesian Conditional Cointegration☆18May 28, 2017Updated 9 years ago