Alternatives and similar repositories for mlperf_client

Users that are interested in mlperf_client are comparing it to the libraries listed below

• onnx / turnkeyml
  No-code CLI designed for accelerating ONNX workflows
  ☆227Updated 7 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆68Updated last year
• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆97Updated 3 months ago
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆93Updated this week
• tenstorrent / tt-buda-demos
  Repository of model demos using TT-Buda
  ☆63Updated 10 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆137Updated 8 months ago
• groq / mlagility
  Machine Learning Agility (MLAgility) benchmark and benchmarking tools
  ☆40Updated 6 months ago
• anthonix / llm.c
  LLM training in simple, raw C/HIP for AMD GPUs
  ☆58Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆168Updated 2 weeks ago
• intel / intel-ai-super-builder
  Intel® AI Super Builder
  ☆156Updated 2 weeks ago
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆104Updated last week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆94Updated 11 months ago
• google / minja
  A minimalistic C++ Jinja templating engine for LLM chat templates
  ☆203Updated 4 months ago
• openvinotoolkit / openvino.genai
  Run Generative AI models with simple C++/Python API and using OpenVINO Runtime
  ☆428Updated this week
• ROCm / pyrsmi
  python package of rocm-smi-lib
  ☆24Updated last month
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆97Updated 2 months ago
• ROCm / hipBLASLt
  [DEPRECATED] Moved to ROCm/rocm-libraries repo
  ☆113Updated this week
• gpu-mode / kernelbot
  Write a fast kernel and run it on Discord. See how you compare against the best!
  ☆68Updated last week
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆352Updated last year
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆50Updated this week
• amd / ryzen-ai-documentation
  Onboarding documentation source for the AMD Ryzen™ AI Software Platform. The AMD Ryzen™ AI Software Platform enables developers to take…
  ☆92Updated this week
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆142Updated 4 months ago
• bentoml / llm-optimizer
  Benchmark and optimize LLM inference across frameworks with ease
  ☆161Updated 4 months ago
• salykova / sgemm.cu
  High-Performance FP32 GEMM on CUDA devices
  ☆117Updated last year
• amd / ZenDNN
  ☆137Updated last week
• gpu-mode / popcorn-cli
  ☆95Updated this week
• chu-tianxiang / llama-cpp-torch
  llama.cpp to PyTorch Converter
  ☆37Updated last year
• mozilla-ai / agent.cpp
  Building blocks for agents in C++
  ☆139Updated 2 weeks ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆63Updated 4 months ago
• Chen-zexi / vllm-cli
  A command-line interface tool for serving LLM using vLLM.
  ☆468Updated 2 weeks ago