Curated collection of AI inference engineering resources — LLM serving, GPU kernels, quantization, distributed inference, and production deployment. Compiled from the AER Labs community.
☆373Feb 4, 2026Updated 9 months ago
Alternatives and similar repositories for ai-inference-resources
Users that are interested in ai-inference-resources are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆15Mar 11, 2026Updated 8 months ago
- 100 days of LLM inference engineering — daily posts, experiments, and visualizations☆1,925Apr 30, 2026Updated 6 months ago
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,369Oct 17, 2026Updated 3 weeks ago
- Learning notes for understanding modern Machine Learning System.☆66Updated this week
- A Practitioner handbook for production llm serving.☆279Oct 27, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Learn LLM Inference Engineering step by step - from KV cache, PagedAttention, and continuous batching to vLLM, SGLang, and GPUs.☆423Nov 9, 2026Updated last week
- a personal collection of my notes for ml sys☆128Oct 26, 2026Updated 3 weeks ago
- 프롬프트 중 사용자가 지정한 특정 부분만 인용하도록 유도하여, LLM이 정확하고 일관된 답변을 생성할 수 있도록 지원하는 LogitsProcessor☆11Oct 12, 2025Updated last year
- 🎹 Instruct.KR 2025 Summer Meetup: 오픈소스 LLM, vLLM으로 Production까지 🎹☆23Aug 2, 2025Updated last year
- [ACL'26 Workshop] KoViDoRe: A Benchmark for Korean Visual Document Retrieval☆30Jul 2, 2026Updated 4 months ago
- Ground Truth: A working developer's path into AI and machine learning, with interactive explanations, reproducible Python labs, and pract…☆41Updated this week
- ☆12Dec 16, 2022Updated 3 years ago
- A curated list of awesome papers about utilizing large language models for ranking.☆32Apr 12, 2026Updated 7 months ago
- A curated list of resources dedicated to Code-mixed Natural Language Processing (NLP).☆18Oct 15, 2026Updated last month
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Langchain Gradient Integration☆16Aug 14, 2026Updated 3 months ago
- AI Infrastructure Performance Engineer Learning Track - GPU optimization, inference optimization, and cost reduction☆82Nov 8, 2026Updated last week
- GPU Engineering for AI Systems☆610May 17, 2026Updated 5 months ago
- prompt tuning, management, playground lab(w/ Python Streamlit, FastAPI)☆13Mar 14, 2025Updated last year
- ☆11Mar 12, 2025Updated last year
- Evaluate gpt-4o on CLIcK (Korean NLP Dataset)☆20May 18, 2024Updated 2 years ago
- Kernels by MakoraGenerate☆15May 26, 2026Updated 5 months ago
- ☆19May 16, 2024Updated 2 years ago
- a guide to learn and implement inference techniques from scratch☆14Jun 18, 2026Updated 4 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- huggingface에 있는 한국어 데이터 세트☆38Oct 10, 2024Updated 2 years ago
- ☆16Aug 18, 2025Updated last year
- ☆16Sep 29, 2026Updated last month
- GPU programming related news and material links☆2,410Jun 15, 2026Updated 5 months ago
- Online-adaptive multi-agent framework for enterprise API orchestration. Adaptive circuit breakers, SLA-driven routing, hybrid LLM+rule in…☆126Sep 20, 2026Updated last month
- Agent Harness prototype with main components and deployment on Railway☆27Aug 21, 2026Updated 2 months ago
- LLM 모델의 외국어 토큰 생성을 막는 코드 구현☆87Aug 7, 2025Updated last year
- ☆18Mar 6, 2026Updated 8 months ago
- VectraDB is an in-memory, high performance vector database written in Go.☆34Jun 21, 2026Updated 4 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- the hacker’s whirlwind tour to deep learning alchemy and deep learning systems☆98Updated this week
- ☆105Oct 4, 2026Updated last month
- Tracing JIT Deep Learning Library☆32Nov 6, 2025Updated last year
- Korean MTEB Retrieval Evaluators for SPLADE, Dense, and Reranking models☆14Jul 21, 2026Updated 3 months ago
- Pytorch를 활용한 WandB의 Sweeps 🧹☆15Dec 24, 2022Updated 3 years ago
- Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and…☆2,214Nov 3, 2026Updated last week
- From-scratch multi-GPU LLM inference engine: paged KV cache, continuous batching, prefix caching, tensor parallelism, OpenAI-compatible s…☆28Sep 25, 2026Updated last month
Are these results useful?