andreyanufr / who_what_benchmarkLinks
☆20Updated last year
Alternatives and similar repositories for who_what_benchmark
Users that are interested in who_what_benchmark are comparing it to the libraries listed below
Sorting:
- 🤗 Optimum Intel: Accelerate inference with Intel optimization tools☆477Updated this week
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆305Updated last month
- An innovative library for efficient LLM inference via low-bit quantization☆349Updated 10 months ago
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆190Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆77Updated this week
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU. Seamlessly integrated with Torchao, Tra…☆528Updated this week
- A high-throughput and memory-efficient inference and serving engine for LLMs☆264Updated 9 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆845Updated last week
- SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX R…☆2,452Updated this week
- Neural Network Compression Framework for enhanced OpenVINO™ inference☆1,056Updated last week
- Tools for easier OpenVINO development/debugging☆9Updated 4 months ago
- A pytorch quantization backend for optimum☆963Updated last week
- Reference models for Intel(R) Gaudi(R) AI Accelerator☆166Updated last week
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆405Updated this week
- Easy and Efficient Quantization for Transformers☆198Updated 3 weeks ago
- A collection of LogitsProcessors to customize and enhance LLM behavior for specific tasks.☆315Updated last week
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆1,640Updated this week
- For releasing code related to compression methods for transformers, accompanying our publications☆435Updated 6 months ago
- Prune a model while finetuning or training.☆403Updated 3 years ago
- Code repo for the paper "SpinQuant LLM quantization with learned rotations"☆302Updated 5 months ago
- [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization☆695Updated 11 months ago
- ☆274Updated last month
- LLM Workshop by Sourab Mangrulkar☆387Updated last year
- The Triton TensorRT-LLM Backend☆863Updated last week
- [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Se…☆717Updated 4 months ago
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,169Updated 9 months ago
- The repository for the code of the UltraFastBERT paper☆516Updated last year
- FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.☆855Updated 10 months ago
- Official PyTorch implementation of QA-LoRA☆138Updated last year
- This repository contains tutorials and examples for Triton Inference Server☆735Updated last month