huggingface / inference-benchmarkerLinks
Inference server benchmarking tool
☆74Updated 2 months ago
Alternatives and similar repositories for inference-benchmarker
Users that are interested in inference-benchmarker are comparing it to the libraries listed below
Sorting:
- ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)☆119Updated this week
- Benchmark suite for LLMs from Fireworks.ai☆76Updated 2 weeks ago
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆137Updated 11 months ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆264Updated 8 months ago
- ☆221Updated this week
- Easy and Efficient Quantization for Transformers☆199Updated 4 months ago
- Load compute kernels from the Hub☆191Updated this week
- ☆159Updated this week
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆87Updated this week
- Manage scalable open LLM inference endpoints in Slurm clusters☆260Updated 11 months ago
- ☆54Updated 7 months ago
- 👷 Build compute kernels☆68Updated this week
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆304Updated 3 weeks ago
- Cray-LM unified training and inference stack.☆22Updated 4 months ago
- ☆68Updated this week
- experiments with inference on llama☆104Updated last year
- A safetensors extension to efficiently store sparse quantized tensors on disk☆129Updated this week
- ☆55Updated 9 months ago
- ☆66Updated last year
- IBM development fork of https://github.com/huggingface/text-generation-inference☆60Updated last month
- KV cache compression for high-throughput LLM inference☆131Updated 4 months ago
- Data preparation code for Amber 7B LLM☆91Updated last year
- Google TPU optimizations for transformers models☆113Updated 5 months ago
- ☆194Updated last month
- Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)☆188Updated this week
- Comparison of Language Model Inference Engines☆217Updated 6 months ago
- ☆213Updated 5 months ago
- The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…☆137Updated last month
- vLLM performance dashboard☆30Updated last year
- Experiments on speculative sampling with Llama models☆128Updated 2 years ago