Lightweight Python Wrapper for OpenVINO, enabling LLM inference on NPUs
☆30Dec 17, 2024Updated last year
Alternatives and similar repositories for nitro
Users that are interested in nitro are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- SGLang kernel library for Intel XPU☆37Updated this week
- ☆13May 11, 2023Updated 3 years ago
- ☆54May 19, 2025Updated last year
- ☆23May 5, 2025Updated last year
- (WIP) Parallel inference for black-forest-labs' FLUX model.☆19Nov 18, 2024Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- ☆23Aug 14, 2024Updated 2 years ago
- ☆24Oct 6, 2025Updated last year
- ☆13Oct 13, 2026Updated 3 weeks ago
- Quantized Attention on GPU☆45Nov 22, 2024Updated last year
- ☆13Apr 21, 2026Updated 6 months ago
- Cluster doctor skills☆14May 23, 2026Updated 5 months ago
- 使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention☆82Aug 12, 2024Updated 2 years ago
- triton ver of gqa flash attn, based on the tutorial☆12Aug 4, 2024Updated 2 years ago
- ☆16May 23, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A side project that follows all the acceleration tricks in tinyllama, with the minimal modification to the huggingface transformers code.☆13Sep 2, 2024Updated 2 years ago
- Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.☆91Nov 23, 2022Updated 3 years ago
- Multi-Layer Key-Value sharing experiments on Pythia models☆35Jun 14, 2024Updated 2 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆124Updated this week
- a repo to study TensorRT Plugin☆11Apr 30, 2025Updated last year
- Pm2 cli logs interface☆12May 21, 2015Updated 11 years ago
- ☆107Sep 9, 2024Updated 2 years ago
- 很好用的tnn classify demo☆11Mar 24, 2021Updated 5 years ago
- SGLang wheels for multiple platforms☆11Oct 13, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆58May 3, 2026Updated 6 months ago
- Lightweight framework for 3D rendering.☆12Jun 14, 2026Updated 4 months ago
- Foundation AI - Reinforcement Learning Library☆14Updated this week
- A professional post-production, color grading, and sharpness suite for ComfyUI☆16Jun 9, 2026Updated 4 months ago
- Loopbackd - connect to remote machines shell via Telegram/Whatsapp/SMS☆10Nov 21, 2022Updated 3 years ago
- Multi-DNN Inference Engine for Heterogeneous Mobile Processors☆40Jul 24, 2024Updated 2 years ago
- Production RAG hallucination detection + self-healing pipeline — 5 checks, 3 healing strategies, zero external APIs☆13May 7, 2026Updated 5 months ago
- Exploring how optimizations for GEMMs work☆36Feb 28, 2026Updated 8 months ago
- Distillation pipeline from pretrained Transformers to customized FLA models☆13Aug 4, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- A script to reorganize 'Want to go' Saved places in Google Maps into separate lists by category.☆12May 14, 2024Updated 2 years ago
- ☆22Oct 1, 2024Updated 2 years ago
- Community maintained hardware plugin for vLLM on AWS Neuron☆59Oct 27, 2026Updated last week
- ☆10Mar 31, 2025Updated last year
- ☆21Oct 24, 2026Updated last week
- A simple demonstration of how PyTorch autograd works☆16Sep 23, 2021Updated 5 years ago
- Codebase for ICML'24 paper: Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs☆28Jun 25, 2024Updated 2 years ago