Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.
☆21Aug 11, 2026Updated last month
Alternatives and similar repositories for Efficient-LLM-Inference-Serving-Systems
Users that are interested in Efficient-LLM-Inference-Serving-Systems are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Nvim plugin for reviewing PRs with side by side diffs, inline comments, and persistent review state.☆18Apr 3, 2026Updated 5 months ago
- Deriving steepest descent convergence bounds and hyperparameter scaling laws in machine learning optimization from first principles, form…☆17Apr 11, 2026Updated 5 months ago
- ☆11Feb 22, 2025Updated last year
- CLIP is an open source, multimodal computer vision model and it's awesome!☆17Dec 16, 2024Updated last year
- Repository for NLP project. Name to be changed when we decide on a project☆17Apr 19, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆15Apr 14, 2025Updated last year
- Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and L…☆23Apr 12, 2024Updated 2 years ago
- This dataset contains about 110k images annotated with the depth and occlusion relationships between arbitrary objects. It enables resear…☆16Apr 28, 2021Updated 5 years ago
- MLCD-Seg is a zero-shot segmentation model from DeepGlint.☆18Jul 4, 2025Updated last year
- Iterate fast on your RAG pipelines☆24Jun 21, 2025Updated last year
- ☆30Sep 3, 2021Updated 5 years ago
- Replication package for evaluation of code generation metrics☆17Nov 24, 2025Updated 9 months ago
- Code for "What really matters in matrix-whitening optimizers?"☆25Oct 31, 2025Updated 10 months ago
- Human Protein Atlas - Single Cell Classification 2nd place solution Dual Head pipeline☆13May 25, 2021Updated 5 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Training HuggingFace models using fastai☆11Jul 22, 2021Updated 5 years ago
- [ISPRS.J'24] HCTO: Optimality-aware LiDAR inertial odometry with hybrid continuous time optimization for compact wearable mapping system☆42Apr 25, 2024Updated 2 years ago
- ☆15Aug 26, 2023Updated 3 years ago
- Example of using the SAINT architecture in fastai☆12Aug 25, 2021Updated 5 years ago
- ☆38Jul 5, 2024Updated 2 years ago
- IET's Hostel Management App☆25Feb 20, 2024Updated 2 years ago
- PyTorch implementation of the End-to-End Memory Network with attention layer vizualisation support.☆12Jun 30, 2018Updated 8 years ago
- Forecastable Component Analysis (ForeCA) in Python☆38Nov 21, 2025Updated 9 months ago
- Various R files☆10Dec 4, 2021Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Tricks used in deep learning. Including papers read recently.☆16Jul 10, 2018Updated 8 years ago
- ☆10Sep 26, 2019Updated 6 years ago
- Utilizing nbdev in Google Colaboratory☆14Apr 12, 2023Updated 3 years ago
- A curated list of resources dedicated to Code-mixed Natural Language Processing (NLP).☆18Jun 23, 2026Updated 2 months ago
- Mobile Viewer for W&B, built on top of Flutter.☆41Mar 2, 2024Updated 2 years ago
- (NeurIPS-2019 MicroNet Challenge - 3rd Winner) Open source code for "SIPA: A simple framework for efficient networks"☆19Dec 18, 2022Updated 3 years ago
- Use AI to preview how garments look on you directly on product pages from Amazon and Coupang. Upload your photo, click "Try On," and see …☆22Apr 13, 2025Updated last year
- ☆49Sep 22, 2023Updated 2 years ago
- Convolutional autoencoder for encoding/decoding RGB images in TensorFlow with high compression ratio☆12Jul 28, 2018Updated 8 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Banana-bot is tool automatically claim rewards from banana-bot telegram☆13Aug 3, 2024Updated 2 years ago
- DMKD- Series2Vec: Similarity-based Representation Learning for Time Series☆44Mar 21, 2025Updated last year
- Brain-Box is a web application that allows students to organize and manage their study materials, including subjects, chapters, notes, an…☆27Feb 20, 2024Updated 2 years ago
- Text Normalization utilities for normalizing text for TTS☆26Mar 4, 2026Updated 6 months ago
- Implementation of <Orthogonal Model Merging>☆35Aug 5, 2026Updated last month
- ☆37May 11, 2024Updated 2 years ago
- CNN ensemble for prostate cancer Gleason grading☆20Jan 28, 2026Updated 7 months ago