Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.
☆20Jul 1, 2026Updated last month
Alternatives and similar repositories for Efficient-LLM-Inference-Serving-Systems
Users that are interested in Efficient-LLM-Inference-Serving-Systems are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- CastleHill: Separable Causal Diffusion / Varitaion Flow Maps for LTX-2 long-form video generation☆15May 19, 2026Updated 2 months ago
- Full End-to-End examples showing how to use First-gen Gaudi and Gaudi2 in common use cases☆13Dec 2, 2024Updated last year
- CLIP is an open source, multimodal computer vision model and it's awesome!☆17Dec 16, 2024Updated last year
- Repository for NLP project. Name to be changed when we decide on a project☆16Apr 19, 2022Updated 4 years ago
- Multiple ways to model user preference in recommender systems☆18May 2, 2024Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Learn the ins and outs of efficiently serving Large Language Models (LLMs). Dive into optimization techniques, including KV caching and L…☆22Apr 12, 2024Updated 2 years ago
- VidKV: Plug-and-Play 1.x-Bit KV Cache Quantization for Video Large Language Models☆25Mar 26, 2025Updated last year
- Iterate fast on your RAG pipelines☆24Jun 21, 2025Updated last year
- Repository for ACM India Summer School on Generative AI for Text☆13Jul 11, 2024Updated 2 years ago
- Replication package for evaluation of code generation metrics☆17Nov 24, 2025Updated 8 months ago
- Training HuggingFace models using fastai☆11Jul 22, 2021Updated 5 years ago
- ☆37Jul 5, 2024Updated 2 years ago
- ☆15Aug 26, 2023Updated 2 years ago
- Example of using the SAINT architecture in fastai☆12Aug 25, 2021Updated 4 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- World Models That Know When They Don't Know: Controllable Video Generation with Calibrated Uncertainty☆25Dec 8, 2025Updated 7 months ago
- ☆28May 11, 2026Updated 2 months ago
- Triton implementation of GPT/LLAMA☆22Aug 28, 2024Updated last year
- IET's Hostel Management App☆25Feb 20, 2024Updated 2 years ago
- Image-to-text translation of chemical molecule structures with deep learning (top-5% Kaggle solution)☆15Sep 4, 2022Updated 3 years ago
- Cross-encoder reranking techniques for information retrieval — from basic scoring to fine-tuning, caching, distillation, and ColBE…☆19Apr 4, 2026Updated 4 months ago
- Various R files☆10Dec 4, 2021Updated 4 years ago
- This repository provides the official implementation of QSVD, a method for efficient low-rank approximation that unifies Query-Key-Value …☆28May 16, 2026Updated 2 months ago
- Tricks used in deep learning. Including papers read recently.☆16Jul 10, 2018Updated 8 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- A curated list of resources dedicated to Code-mixed Natural Language Processing (NLP).☆15Jun 23, 2026Updated last month
- Mobile Viewer for W&B, built on top of Flutter.☆41Mar 2, 2024Updated 2 years ago
- Use AI to preview how garments look on you directly on product pages from Amazon and Coupang. Upload your photo, click "Try On," and see …☆23Apr 13, 2025Updated last year
- Convolutional autoencoder for encoding/decoding RGB images in TensorFlow with high compression ratio☆12Jul 28, 2018Updated 8 years ago
- Weight Predictor Networks with Sparse Feature Selection for Small Size Tabular Biomedical Data. Published at AAAI 2023☆20Jun 29, 2023Updated 3 years ago
- DMKD- Series2Vec: Similarity-based Representation Learning for Time Series☆44Mar 21, 2025Updated last year
- ☆21Jul 21, 2026Updated 2 weeks ago
- Masakhane Web is a translation web application for solely African Languages.☆38Aug 11, 2023Updated 2 years ago
- A LLM Agent with Langchain/Langgraph helps to analyze CV, look relevant jobs via API, and write a cover letter according to it☆61May 1, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Text Normalization utilities for normalizing text for TTS☆26Mar 4, 2026Updated 5 months ago
- Public-facing codebase accompanying: "Reasoning Cache: Continual Improvement Over Long Horizons via Short-Horizon RL"☆36Feb 6, 2026Updated 6 months ago
- CNN ensemble for prostate cancer Gleason grading☆19Jan 28, 2026Updated 6 months ago
- ☆13Jul 3, 2023Updated 3 years ago
- 🔍 GemInsights: Unleash Gemini AI on your data! 🚀 Analyze dataframes for valuable insights, replacing traditional data analysis. 📊 A c…☆21Mar 1, 2024Updated 2 years ago
- A service to batch the http requests.☆37Feb 27, 2026Updated 5 months ago
- zero-to-lightning☆32May 6, 2024Updated 2 years ago