benja263 / Integer-Only-Inference-for-Deep-Learning-in-Native-C
Converting a deep neural network to integer-only inference in native C via uniform quantization and the fixed-point representation.
☆20Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for Integer-Only-Inference-for-Deep-Learning-in-Native-C
- Penn CIS 5650 (GPU Programming and Architecture) Final Project☆24Updated 10 months ago
- ☆40Updated this week
- llama INT4 cuda inference with AWQ☆47Updated 4 months ago
- PyTorch extension for emulating FP8 data formats on standard FP32 Xeon/GPU hardware.☆100Updated 11 months ago
- PyTorch compilation tutorial covering TorchScript, torch.fx, and Slapo☆19Updated last year
- ☆121Updated last year
- Nsight Compute in Docker☆11Updated 10 months ago
- CUDA Matrix Multiplication Optimization☆139Updated 3 months ago
- ResNet Implementation, Training, and Inference Using LibTorch C++ API☆35Updated 5 months ago
- End to End steps for adding custom ops in PyTorch.☆19Updated 4 years ago
- Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.☆48Updated 2 months ago
- ☆18Updated last month
- An Open Convolutional Neural Network Framework in C++ From Scratch☆59Updated 3 years ago
- Standalone Flash Attention v2 kernel without libtorch dependency☆98Updated last month
- This is the open-source version of TinyTS. The code is dirty so far. We may clean the code in the future.☆10Updated 3 months ago
- PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation☆11Updated 2 months ago
- SparseTIR: Sparse Tensor Compiler for Deep Learning☆131Updated last year
- ☆24Updated 7 months ago
- A standalone GEMM kernel for fp16 activation and quantized weight, extracted from FasterTransformer☆85Updated 8 months ago
- Swin Transformer C++ Implementation☆54Updated 3 years ago
- GEMM and Winograd based convolutions using CUTLASS☆25Updated 4 years ago
- Benchmark code for the "Online normalizer calculation for softmax" paper☆59Updated 6 years ago
- High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.☆87Updated 3 months ago
- ☆67Updated last year
- You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms☆10Updated last year
- Awesome Quantization Paper lists with Codes☆12Updated 3 years ago
- A Winograd Minimal Filter Implementation in CUDA☆23Updated 3 years ago
- This is a demo how to write a high performance convolution run on apple silicon☆52Updated 2 years ago
- CUDA 8-bit Tensor Core Matrix Multiplication based on m16n16k16 WMMA API☆26Updated last year