abdelfattah-lab / nitroLinks
Lightweight Python Wrapper for OpenVINO, enabling LLM inference on NPUs
☆26Updated last year
Alternatives and similar repositories for nitro
Users that are interested in nitro are comparing it to the libraries listed below
Sorting:
- Repository for CPU Kernel Generation for LLM Inference☆27Updated 2 years ago
- ☆59Updated 2 years ago
- Beyond KV Caching: Shared Attention for Efficient LLMs☆20Updated last year
- ☆114Updated last month
- QuIP quantization☆61Updated last year
- PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation☆31Updated last year
- Efficient Long-context Language Model Training by Core Attention Disaggregation☆73Updated this week
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆53Updated last year
- The evaluation framework for training-free sparse attention in LLMs☆108Updated 2 months ago
- ☆68Updated 6 months ago
- Compression for Foundation Models☆34Updated 5 months ago
- Accelerate LLM preference tuning via prefix sharing with a single line of code☆51Updated 5 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆42Updated last year
- ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization☆111Updated last year
- ☆43Updated 7 months ago
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆135Updated last year
- Quantize transformers to any learned arbitrary 4-bit numeric format☆50Updated 5 months ago
- ☆66Updated last month
- ☆63Updated 7 months ago
- KVTuner: Sensitivity-Aware Layer-wise Mixed Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference☆25Updated 7 months ago
- ☆54Updated last year
- ☆133Updated 7 months ago
- 🔥 LLM-powered GPU kernel synthesis: Train models to convert PyTorch ops into optimized Triton kernels via SFT+RL. Multi-turn compilation…☆110Updated last month
- ☆52Updated 7 months ago
- ☆115Updated 7 months ago
- Work in progress.☆76Updated last month
- Compressing Large Language Models using Low Precision and Low Rank Decomposition☆106Updated last month
- ☆31Updated last year
- Easy, Fast, and Scalable Multimodal AI☆82Updated last week
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆175Updated last year