Inference server benchmarking tool
☆165Jun 9, 2026Updated last month
Alternatives and similar repositories for inference-benchmarker
Users that are interested in inference-benchmarker are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Minimal implementation of a Byte Pair Encoding (BPE) tokenizer in Zig☆15Apr 7, 2025Updated last year
- ANE accelerated embedding models!☆20Dec 11, 2024Updated last year
- Build compute kernels and load them from the Hub.☆715Updated this week
- Crossword puzzles in your terminal.☆22Feb 4, 2026Updated 5 months ago
- Kubernetes autoscaler for deployments that consume queue in RMQ☆18Aug 6, 2021Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆95Updated this week
- Where GPUs get cooked 👩🍳🔥☆402May 26, 2026Updated 2 months ago
- A course on building Large Language Models☆19Mar 24, 2025Updated last year
- 👷 Build compute kernels☆213Apr 6, 2026Updated 3 months ago
- Benchmark suite for LLMs from Fireworks.ai☆111Updated this week
- Simple examples using Argilla tools to build AI☆59Nov 18, 2024Updated last year
- Testbench for llama.cpp llama-server☆15Aug 20, 2025Updated 11 months ago
- CodeQUEST is a generalizable framework which leverages LLMs to iteratively evaluate and enhance code quality across multiple dimensions f…☆17Feb 11, 2026Updated 5 months ago
- AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solu…☆470Updated this week
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆27Apr 24, 2025Updated last year
- A Go library for semantic caching with LRU eviction, supporting vector-based similarity search with pluggable embedding backends (local o…☆27Jul 20, 2026Updated last week
- Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.☆702Updated this week
- Lightweight Python Wrapper for OpenVINO, enabling LLM inference on NPUs☆29Dec 17, 2024Updated last year
- ☆16Dec 16, 2025Updated 7 months ago
- TUI for LM Studio☆17Oct 19, 2025Updated 9 months ago
- Implementation of the HuggingFace Xet Protocol.☆27Jun 30, 2026Updated 3 weeks ago
- ☆17Oct 21, 2025Updated 9 months ago
- ☆29May 26, 2026Updated 2 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Hugging Face Jobs☆20Jul 11, 2025Updated last year
- ☆25Oct 10, 2025Updated 9 months ago
- ☆25Dec 13, 2024Updated last year
- Manage scalable open LLM inference endpoints in Slurm clusters☆289Jul 11, 2024Updated 2 years ago
- Decodes Compact Disc data from microscope images of a CD's surface☆12Jan 14, 2023Updated 3 years ago
- Patch management tool for git submodules☆16Jul 25, 2025Updated last year
- Cross-platform Python client for the CodeReef.ai portal to manage portable workflows, reusable automation actions, software detection plu…☆11Mar 27, 2020Updated 6 years ago
- 🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…☆338May 26, 2026Updated 2 months ago
- An OpenAI API compatible images server to generate or manipulate images.☆18Feb 2, 2025Updated last year
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends☆2,498Jun 29, 2026Updated 3 weeks ago
- A collection of portable workflows, automation recipes and components for MLOps in a unified CK format. Note that this repository is outd…☆19Dec 10, 2024Updated last year
- ☆33Apr 19, 2025Updated last year
- An experimental autonomous research system that conducts comprehensive, multi-hour research sessions and produces book-length reports wit…☆47Dec 7, 2025Updated 7 months ago
- Clue inspired puzzles for testing LLM deduction abilities☆47Mar 19, 2026Updated 4 months ago
- ☆151Updated this week
- Implementation of Tree esoteric programming language☆10Mar 3, 2023Updated 3 years ago