everything you need to know about inference engineering
☆44Sep 29, 2026Updated last month
Alternatives and similar repositories for inference-book
Users that are interested in inference-book are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Plug-and-play PyTorch transformer research framework. Swap attention, FFN, norm, positional, connection, projection, optimizer, scheduler…☆18Oct 25, 2026Updated 2 weeks ago
- Terminus is an AI coding agent that works directly inside your terminal and understands your codebase. It investigates bugs, builds fea…☆16Sep 16, 2026Updated last month
- A small, readable PyTorch inference engine for Qwen3-4B with continuous batching, native paged attention, prefix caching, and an OpenAI-c…☆18Oct 24, 2026Updated 2 weeks ago
- A curated resource list for learning post training☆22Oct 11, 2026Updated 3 weeks ago
- ☆16Jun 28, 2026Updated 4 months ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Learning notes for understanding modern Machine Learning System.☆66Oct 30, 2026Updated last week
- A curated list of resources for ML Systems Engineering - hardware, compilers, distributed training, inference, and production operations.☆27Jul 4, 2026Updated 4 months ago
- FlaskGPT is a minimal ChatGPT clone that uses Python, Flask, langchain and Chroma with realtime token output using SSE.☆12Sep 30, 2023Updated 3 years ago
- documenting my work in inference engineering☆25Apr 19, 2026Updated 6 months ago
- a personal collection of my notes for ml sys☆127Oct 26, 2026Updated 2 weeks ago
- Continual Learning Method RWM for AAAI 2024☆21Sep 26, 2024Updated 2 years ago
- observability for speculative decoding☆15Aug 23, 2026Updated 2 months ago
- ☆144Oct 29, 2026Updated last week
- Example apps demonstrating the WRLD Android Java API to display stunning interactive 3D maps. https://www.wrld3d.com/developers/☆13Nov 3, 2022Updated 4 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Node-level enforcement and metrics for fractional GPU allocations made by KAI Scheduler.☆57Updated this week
- A collection of GPU experiments and benchmarks for my personal understanding and research.☆43Aug 19, 2026Updated 2 months ago
- Drop-in GPU kernel optimizer for PyTorch models, no CUDA or GPU expertise required☆28Jun 7, 2026Updated 5 months ago
- ☆24Jul 20, 2026Updated 3 months ago
- ☆18Oct 26, 2025Updated last year
- NCCL communication API layer, and transport layer created from first principles.☆16Aug 20, 2025Updated last year
- Athrix Sleek Portfolio☆32Apr 21, 2026Updated 6 months ago
- FLAT: Flash Linear Attention on TPUs☆24Oct 13, 2026Updated 3 weeks ago
- ☆104Oct 4, 2026Updated last month
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.☆4,354Oct 17, 2026Updated 3 weeks ago
- A Python bootcamp for medical students designed to teach basic DICOM and imaging data handling skills.☆10May 2, 2026Updated 6 months ago
- Very simple example to use Dagger 2☆12Dec 19, 2017Updated 8 years ago
- Adaptive batch inference pipeline with semantic complexity routing, circuit breaker fault tolerance, and SSE-streamed progress, built for…☆26May 19, 2026Updated 5 months ago
- ☆11May 21, 2026Updated 5 months ago
- BMSC-GA 4493 Deep Learning in Medicine - Spring 2020☆11Apr 29, 2020Updated 6 years ago
- Card Game 60K☆15Nov 16, 2023Updated 2 years ago
- Megatron-LM setup in the smol-cluster☆12Aug 26, 2024Updated 2 years ago
- Exercises for the semi-supervised summer school https://semisupervised-learning.compute.dtu.dk.☆11Aug 11, 2016Updated 10 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ☆18Sep 24, 2026Updated last month
- ☆13Mar 8, 2025Updated last year
- ☆11Apr 22, 2026Updated 6 months ago
- ☆11Aug 19, 2026Updated 2 months ago
- Integrative Transcriptome-Histopathology Analysis for Breast Cancer Classification☆10Jan 15, 2021Updated 5 years ago
- ☆28Oct 9, 2026Updated 3 weeks ago
- Our Clone of Orca used for experimentation☆21Oct 15, 2024Updated 2 years ago