Lightweight Python Wrapper for OpenVINO, enabling LLM inference on NPUs
☆30Dec 17, 2024Updated last year
Alternatives and similar repositories for nitro
Users that are interested in nitro are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- SGLang kernel library for Intel XPU☆31Updated this week
- ☆54May 19, 2025Updated last year
- This repository contains the official implementation for the ECCV'22 paper, "SPIN: An Empirical Evaluation on Sharing Parameters of Isotr…☆20Sep 9, 2023Updated 2 years ago
- A forked version of flux-fast that makes flux-fast even faster with cache-dit, 3.3x speedup on NVIDIA L20.☆24Jul 18, 2025Updated last year
- ☆23Aug 14, 2024Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- ☆13Aug 18, 2026Updated 2 weeks ago
- 使用 cutlass 仓库在 ada 架构上实现 fp8 的 flash attention☆83Aug 12, 2024Updated 2 years ago
- Quantized Attention on GPU☆45Nov 22, 2024Updated last year
- Multi-Layer Key-Value sharing experiments on Pythia models☆35Jun 14, 2024Updated 2 years ago
- triton ver of gqa flash attn, based on the tutorial☆12Aug 4, 2024Updated 2 years ago
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆58May 3, 2026Updated 4 months ago
- ☆107Sep 9, 2024Updated last year
- ☆16Aug 19, 2024Updated 2 years ago
- Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.☆92Nov 23, 2022Updated 3 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- A high-throughput and memory-efficient inference and serving engine for LLMs☆123Updated this week
- A script to reorganize 'Want to go' Saved places in Google Maps into separate lists by category.☆12May 14, 2024Updated 2 years ago
- 很好用的tnn classify demo☆11Mar 24, 2021Updated 5 years ago
- Community maintained hardware plugin for vLLM on AWS Neuron☆49Aug 17, 2026Updated 2 weeks ago
- SGLang kernel library for NPU☆174Updated this week
- [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving☆347Jul 2, 2024Updated 2 years ago
- DISB is a new DNN inference serving benchmark with diverse workloads and models, as well as real-world traces.☆58Aug 21, 2024Updated 2 years ago
- ☆15May 13, 2024Updated 2 years ago
- TileFusion is an experimental C++ macro kernel template library that elevates the abstraction level in CUDA C for tile processing.☆117Aug 4, 2026Updated 3 weeks ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Test Orchestrator for Performance and Scalability of AI pLatforms☆18Jun 23, 2026Updated 2 months ago
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆160Aug 21, 2025Updated last year
- IntelliJ platform plugin for Wavefront OBJ format☆15Aug 16, 2026Updated 2 weeks ago
- ☆21May 11, 2026Updated 3 months ago
- PolarEngine: vLLM plugin for PolarQuant quantized LLM inference — 75% FP16 speed at 2.3x less VRAM☆35Apr 13, 2026Updated 4 months ago
- ☆14Mar 5, 2024Updated 2 years ago
- Fast low-bit matmul kernels in Triton☆483Aug 9, 2026Updated 3 weeks ago
- [MobiCom 24] Adaptive DNN inference under memory constraints☆59Jan 22, 2025Updated last year
- [CVPR 2025] The official implementation of "CacheQuant: Comprehensively Accelerated Diffusion Models"☆51Nov 2, 2025Updated 10 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 6 months ago
- Commands that will make you more comfortable with the ROCm toolkit.☆18Aug 1, 2024Updated 2 years ago
- Vocabulary Parallelism☆26Mar 10, 2025Updated last year
- ☆156Mar 4, 2025Updated last year
- use yolov3 onnx model to implement object detection☆11Apr 25, 2019Updated 7 years ago
- KV cache compression for high-throughput LLM inference☆161Feb 5, 2025Updated last year
- ONNX Runtime tiny wrapper for openFrameworks☆15Jan 21, 2022Updated 4 years ago