Alternatives and similar repositories for mlperf_client

Users that are interested in mlperf_client are comparing it to the libraries listed below

• onnx / turnkeyml
  No-code CLI designed for accelerating ONNX workflows
  ☆214Updated 4 months ago
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆102Updated last month
• nomic-ai / llama.cpp
  llama.cpp fork used by GPT4All
  ☆57Updated 8 months ago
• AlexBodner / How_Much_VRAM
  ☆102Updated last year
• NVIDIA / RTX-AI-Toolkit
  The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…
  ☆176Updated 11 months ago
• intel / intel-ai-assistant-builder
  Intel® AI Assistant Builder
  ☆111Updated this week
• tensorwavecloud / ScalarLM
  ScalarLM - a unified training and inference stack
  ☆85Updated 2 weeks ago
• Cerebras / DocChat
  GPT-4 Level Conversational QA Trained In a Few Hours
  ☆65Updated last year
• google-ai-edge / LiteRT-LM
  ☆413Updated last week
• openvinotoolkit / openvino.genai
  Run Generative AI models with simple C++/Python API and using OpenVINO Runtime
  ☆359Updated this week
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆151Updated last week
• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆90Updated last month
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 7 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆134Updated 4 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆66Updated last year
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆349Updated last year
• ibm-granite / granite-3.0-language-models
  ☆264Updated 3 months ago
• sgl-project / sgl-project.github.io
  This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.
  ☆80Updated last week
• NVIDIA / trt-llm-as-openai-windows
  This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows inste…
  ☆125Updated last year
• backprop-ai / vllm-benchmark
  Benchmarking the serving capabilities of vLLM
  ☆53Updated last year
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆109Updated 3 weeks ago
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆90Updated last week
• bentoml / llm-optimizer
  Benchmark and optimize LLM inference across frameworks with ease
  ☆121Updated last month
• anthonix / llm.c
  LLM training in simple, raw C/HIP for AMD GPUs
  ☆51Updated last year
• Cornell-RelaxML / yaqa-quantization
  ☆60Updated 4 months ago
• Infini-AI-Lab / UMbreLLa
  LLM Inference on consumer devices
  ☆124Updated 7 months ago
• Chen-zexi / vllm-cli
  A command-line interface tool for serving LLM using vLLM.
  ☆427Updated last month
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆56Updated 11 months ago
• catid / bitnet_cpu
  Experiments with BitNet inference on CPU
  ☆54Updated last year
• weaviate / structured-rag
  Experimental Code for StructuredRAG: JSON Response Formatting with Large Language Models
  ☆111Updated 6 months ago