Inference server benchmarking tool
☆176Sep 23, 2026Updated 3 weeks ago
Alternatives and similar repositories for inference-benchmarker
Users that are interested in inference-benchmarker are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Minimal implementation of a Byte Pair Encoding (BPE) tokenizer in Zig☆15Apr 7, 2025Updated last year
- ANE accelerated embedding models!☆20Dec 11, 2024Updated last year
- Build compute kernels and load them from the Hub.☆773Updated this week
- A way to visualize nginx config files☆10Nov 21, 2022Updated 3 years ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆97Oct 5, 2026Updated last week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Where GPUs get cooked 👩🍳🔥☆408Sep 24, 2026Updated 3 weeks ago
- A course on building Large Language Models☆20Mar 24, 2025Updated last year
- 👷 Build compute kernels☆213Apr 6, 2026Updated 6 months ago
- Benchmark suite for LLMs from Fireworks.ai☆117Updated this week
- Simple examples using Argilla tools to build AI☆60Nov 18, 2024Updated last year
- Testbench for llama.cpp llama-server☆15Aug 20, 2025Updated last year
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆27Apr 24, 2025Updated last year
- ☆24Oct 6, 2026Updated last week
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆773Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- ☆17Aug 14, 2026Updated 2 months ago
- TUI for LM Studio☆17Oct 19, 2025Updated 11 months ago
- Lightweight Python Wrapper for OpenVINO, enabling LLM inference on NPUs☆30Dec 17, 2024Updated last year
- ☆29Sep 23, 2026Updated 3 weeks ago
- Rust client for the huggingface hub aiming for minimal subset of features over `huggingface-hub` python package☆337Oct 8, 2026Updated last week
- ☆25Oct 10, 2025Updated last year
- Cloud Native Benchmarking of Foundation Models☆46Jul 31, 2025Updated last year
- Manage scalable open LLM inference endpoints in Slurm clusters☆297Jul 11, 2024Updated 2 years ago
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆1,694Updated this week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆340Sep 29, 2026Updated 2 weeks ago
- An OpenAI API compatible images server to generate or manipulate images.☆18Feb 2, 2025Updated last year
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆2,561Oct 6, 2026Updated last week
- Synthetic Data Generation with Execution-Based Verification and Grounding for LLM Training.☆24Feb 7, 2025Updated last year
- ☆33Apr 19, 2025Updated last year
- An experimental autonomous research system that conducts comprehensive, multi-hour research sessions and produces book-length reports wit…☆51Dec 7, 2025Updated 10 months ago
- ☆20Dec 22, 2023Updated 2 years ago
- ☆164Oct 8, 2026Updated last week
- Official Code for ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Users (NeurIPS 2024)☆26Oct 23, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Prometheus exporter for Linux based GDDR6/GDDR6X VRAM and GPU Core Hot spot temperature reader for NVIDIA RTX 3000/4000 series GPUs.☆26Oct 2, 2024Updated 2 years ago
- ☆16Dec 16, 2024Updated last year
- Large Language Model Text Generation Inference☆10,880Mar 21, 2026Updated 6 months ago
- LLM inference in C/C++☆24Oct 8, 2026Updated last week
- A Datacenter Scale Distributed Inference Serving Framework☆8,300Updated this week
- Command Line Interface for running 🤗 Transformers Image Classification locally☆19Oct 1, 2026Updated 2 weeks ago
- A blazing fast inference solution for text embeddings models☆5,076Oct 6, 2026Updated last week