Curated collection of AI inference engineering resources — LLM serving, GPU kernels, quantization, distributed inference, and production deployment. Compiled from the AER Labs community.
☆352Feb 4, 2026Updated 8 months ago
Alternatives and similar repositories for ai-inference-resources
Users that are interested in ai-inference-resources are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Ground Truth - A working developer's path into AI and machine learning.☆38Oct 10, 2026Updated last week
- 100 days of LLM inference engineering — daily posts, experiments, and visualizations☆1,872Apr 30, 2026Updated 5 months ago
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,272Updated this week
- a personal collection of my notes for ml sys☆126Updated this week
- 프롬프트 중 사용자가 지정한 특정 부분만 인용하도록 유도하여, LLM이 정확하고 일관된 답변을 생성할 수 있도록 지원하는 LogitsProcessor☆11Oct 12, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- From-scratch multi-GPU LLM inference engine: paged KV cache, continuous batching, prefix caching, tensor parallelism, OpenAI-compatible s…☆27Sep 25, 2026Updated 3 weeks ago
- Discovering cosmic phenomena and extraterrestrial life with AI.☆11Sep 3, 2024Updated 2 years ago
- 🎹 Instruct.KR 2025 Summer Meetup: 오픈소스 LLM, vLLM으로 Production까지 🎹☆23Aug 2, 2025Updated last year
- A study assistant powered by Claude Opus. It provides various tools to assist with different tasks, such as researching,coding,note-takin…☆29Jul 18, 2024Updated 2 years ago
- A from-scratch Rust inference stack for AMD GPUs that talks straight to the Linux kernel driver. No ROCm, HIP, or HSA runtime. Serves OLM…☆24Aug 15, 2026Updated 2 months ago
- Distributed-AI-Systems☆68Oct 10, 2026Updated last week
- ☆11Mar 12, 2025Updated last year
- GPU Engineering for AI Systems☆603May 17, 2026Updated 5 months ago
- A Practitioner handbook for production llm serving.☆262Updated this week
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆22Aug 19, 2026Updated 2 months ago
- vLLM Daily Summarization of Merged PRs☆56Oct 12, 2026Updated last week
- Become GPU kernel engineer step by step.☆74May 7, 2026Updated 5 months ago
- prompt tuning, management, playground lab(w/ Python Streamlit, FastAPI)☆13Mar 14, 2025Updated last year
- [ EMNLP 2025 ] M3Retrieve is a multimodal benchmark for healthcare data . We have done rigorous evaluation of different multimodal retrie…☆14Nov 4, 2025Updated 11 months ago
- huggingface에 있는 한국어 데이터 세트☆38Oct 10, 2024Updated 2 years ago
- Evaluate gpt-4o on CLIcK (Korean NLP Dataset)☆20May 18, 2024Updated 2 years ago
- AI 법률 어드바이저 모델 : KoAlpaca 모델에 생활법령 데이터를 학습시켜 LoRA finetuning & 생활 법령 100문 100답 데이터 2,195개를 스크랩 하여 LLM 학습을 위한 대화 형식의 json 파일로 제작☆16Sep 20, 2026Updated last month
- ☆16Aug 18, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆19May 28, 2026Updated 4 months ago
- Tracing JIT Deep Learning Library☆32Nov 6, 2025Updated 11 months ago
- a whirlwind tour to deep learning alchemy and deep learning systems☆95Updated this week
- VectraDB is an in-memory, high performance vector database written in Go.☆34Jun 21, 2026Updated 4 months ago
- reinvented groq LPU, built to run inference on MicroGPT☆50Aug 24, 2026Updated last month
- Pytorch를 활용한 WandB의 Sweeps 🧹☆15Dec 24, 2022Updated 3 years ago
- A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchma…☆1,001Aug 14, 2026Updated 2 months ago
- A first-principles climb from the absolute bottom of hardware to the top of GPU architecture.☆103Oct 6, 2026Updated 2 weeks ago
- CSC802: Distributed Computing [DC] & CSL802: Distributed Computing Lab [DC Lab] | BE Semester VIII | Computer Engineering☆20Sep 5, 2026Updated last month
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Optimized GPU compiler for LLM inference. Choose from a list of optimized recipes or optimize your own model via kernel fusion, autotunin…☆111Updated this week
- Korean-MTEB☆107May 12, 2026Updated 5 months ago
- The Universe of Data. All about data, data science, and data engineering☆562Jul 18, 2024Updated 2 years ago
- StrategyQA 데이터 세트 번역☆22Apr 12, 2024Updated 2 years ago
- ☆57Oct 4, 2026Updated 2 weeks ago
- Agent 개발 연구에 진심인 사람들의 성장과정☆21Jan 31, 2026Updated 8 months ago
- A compact implementation of SGLang, designed to demystify the complexities of modern LLM serving systems.☆5,386May 17, 2026Updated 5 months ago