A novel and efficient post-training INT6 quantization framework tailored for LLM inference.
☆11Aug 29, 2026Updated 2 months ago
Alternatives and similar repositories for FlexQ
Users that are interested in FlexQ are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Benchmark tests supporting the TiledCUDA library.☆19Nov 19, 2024Updated last year
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆70Mar 25, 2025Updated last year
- More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression（EMNLP 2025）☆12Jan 15, 2025Updated last year
- TensorRT-in-Action 是一个 GitHub 代码库，提供了使用 TensorRT 的代码示例，并有对应 Jupyter Notebook。☆15Jun 1, 2023Updated 3 years ago
- ☆16Feb 20, 2024Updated 2 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Official implementation of the ICLR'25 paper "QERA: an Analytical Framework for Quantization Error Reconstruction".☆14Feb 4, 2025Updated last year
- Tutorials of Extending and importing TVM with CMAKE Include dependency.☆16Oct 11, 2024Updated 2 years ago
- ☆12Dec 17, 2023Updated 2 years ago
- [NeurIPS 2023] ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer☆31Dec 6, 2023Updated 2 years ago
- Accepted paper of SIGMOD 2023, DUCATI: A Dual-Cache Training System for Graph Neural Networks on Giant Graphs with the GPU☆16Dec 15, 2023Updated 2 years ago
- ☆92Jan 23, 2025Updated last year
- ☆16Nov 11, 2025Updated 11 months ago
- BGEMM-CUDA is a CUDA-based low-bit GEMM kernel library for efficient neural network inference. It implements optimized binary and ternary…☆21Aug 30, 2024Updated 2 years ago
- Official implementation of ICML'24 paper "LQER: Low-Rank Quantization Error Reconstruction for LLMs"☆19Jul 11, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Model Compression Toolbox for Large Language Models and Diffusion Models☆16Aug 19, 2026Updated 2 months ago
- HKUST(GZ) Marpit Template for Presentation☆10Nov 21, 2022Updated 3 years ago
- ☆11Nov 22, 2025Updated 11 months ago
- AFPQ code implementation☆23Nov 6, 2023Updated 3 years ago
- High Performance FP8 GEMM Kernels for SM89 and later GPUs.☆21Jan 24, 2025Updated last year
- [EMNLP24]LPZero: Language Model Zero-Cost Proxy Search from Zero☆11Oct 21, 2024Updated 2 years ago
- Decoding Attention is specially optimized for MHA, MQA, GQA and MLA using CUDA core for the decoding stage of LLM inference.☆48Jun 11, 2025Updated last year
- ☆11Mar 9, 2022Updated 4 years ago
- 用pytorch训练18层残差神经网络，用pyqt设计界面☆12Jun 23, 2020Updated 6 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- [IPDPS 2024] Adaptive neighbor sampling for temporal GNN☆16Feb 17, 2025Updated last year
- The official repository for PTQTP implementation☆11Sep 24, 2025Updated last year
- ☆40Dec 14, 2025Updated 10 months ago
- Mixed precision training from scratch with Tensors and CUDA☆30May 14, 2024Updated 2 years ago
- My learning notes for ML&SYS.☆13Mar 27, 2026Updated 7 months ago
- ☆14Mar 19, 2026Updated 7 months ago
- ☆29Aug 14, 2024Updated 2 years ago
- ☆18May 27, 2026Updated 5 months ago
- An official implement of CVPR 2023 paper - NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers☆28Mar 13, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆15Jul 10, 2026Updated 3 months ago
- Lightweight LLM inference engine inspired by nano-vllm, with radix-tree based prefix cache, tp & pp, cuda graph, openai api, async schedu…☆17Mar 29, 2026Updated 7 months ago
- LLM Inference with Microscaling Format☆35Nov 12, 2024Updated last year
- ☆34May 4, 2020Updated 6 years ago
- ☆35Mar 28, 2025Updated last year
- Super fast serving stack for LLM on Windows/Linux/Macos☆17Oct 22, 2026Updated 2 weeks ago
- Benchmarking code for running quantized kernels from vLLM and other libraries☆14Dec 3, 2024Updated last year
Are these results useful?