TLLM_QMM strips the implementation of quantized kernels of Nvidia's TensorRT-LLM, removing NVInfer dependency and exposes ease of use Pytorch module. We modified the dequantation and weight preprocessing to align with popular quantization alogirthms such as AWQ and GPTQ, and combine them with new FP8 quantization.
☆16Jul 5, 2024Updated 2 years ago
Alternatives and similar repositories for TLLM_QMM
Users that are interested in TLLM_QMM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- OpenAI compatible API for open source LLMs☆16Oct 30, 2023Updated 3 years ago
- Deep Variational Information Bottleneck (DVIB) in PyTorch.☆10Apr 25, 2020Updated 6 years ago
- ☆10Mar 6, 2016Updated 10 years ago
- PyTorch code for our paper "Progressive Binarization with Semi-Structured Pruning for LLMs"☆12Jul 11, 2026Updated 3 months ago
- Let's rob the fat guys, and publish everything into feeds.☆11Jun 1, 2015Updated 11 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- An operator for managing Alluxio system on Kubernetes cluster☆13Jan 9, 2024Updated 2 years ago
- A highly optimized LLM inference acceleration engine for Llama and its variants.☆906Mar 18, 2026Updated 7 months ago
- PyTorch implementation of "Deep Transferring Quantization" (ECCV2020)☆18Jun 22, 2022Updated 4 years ago
- ☆27Feb 17, 2025Updated last year
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Feb 29, 2024Updated 2 years ago
- Experiments codes for SIGKDD '22 paper "User-Event Graph Embedding Learning for Context-Aware Recommendation"☆10Aug 14, 2022Updated 4 years ago
- Wallace and Dadda tree multiplier generator in vhdl and verilog☆14Mar 14, 2026Updated 7 months ago
- Quantize pytorch model, support post-training quantization and quantization aware training methods☆15Jun 15, 2023Updated 3 years ago
- Adaptive floating-point based numerical format for resilient deep learning☆14Apr 11, 2022Updated 4 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- 16 bit serial multiplier in SystemVerilog☆13Oct 13, 2018Updated 8 years ago
- ☆12Mar 13, 2023Updated 3 years ago
- High performance NCCL plugin for Bagua.☆15Sep 15, 2021Updated 5 years ago
- Routing framework based on Promise using CoffeeScript☆13Jun 28, 2015Updated 11 years ago
- Core communication lib for Bagua.☆48Sep 15, 2021Updated 5 years ago
- ☆107Sep 9, 2024Updated 2 years ago
- Writing FLUX in Triton☆42Sep 22, 2024Updated 2 years ago
- Pytorch implementation of RAPQ, IJCAI 2022☆23Jul 19, 2023Updated 3 years ago
- A C++ port of karpathy/micrograd, a tiny scalar-valued autograd engine and a neural net library☆13Nov 24, 2023Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A c++ hash map/table which utilizes simd (specifically Intel x86 SSE/AVX)☆12Apr 30, 2019Updated 7 years ago
- Models and training scripts for "LSTMs for Keyword Spotting with ReRAM-based Compute-In-Memory Architectures" (ISCAS 2021).☆17Mar 25, 2021Updated 5 years ago
- A C++-based RPC framework☆12Oct 28, 2021Updated 5 years ago
- ☆22Jul 11, 2023Updated 3 years ago
- ☆13Oct 5, 2020Updated 6 years ago
- A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer☆96Jun 21, 2026Updated 4 months ago
- ☆10Apr 30, 2021Updated 5 years ago
- FlexFlow Serve: Low-Latency, High-Performance LLM Serving☆90Sep 15, 2025Updated last year
- Web应用生成动漫角色黑板粉笔画 - 基于Cloudflare + Google Gemini AI☆12Dec 12, 2025Updated 10 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- This project is for dealing with dynamic multiobjective optimization problems using a Multiobjective Evolutionary Algorithm.☆21May 6, 2018Updated 8 years ago
- LRU Cache for node.js/browser.☆11Jul 5, 2017Updated 9 years ago
- **DEPRECATED** A simple file sharing application in PHP.☆19Aug 1, 2013Updated 13 years ago
- OneFlow Serving☆20Apr 10, 2025Updated last year
- SRAM☆24Sep 6, 2020Updated 6 years ago
- qwen2 and llama3 cpp implementation☆50Jun 7, 2024Updated 2 years ago
- ECE 5745 Tutorial 8: SRAM Generators☆19Mar 5, 2022Updated 4 years ago