Alternatives and similar repositories for mlperf_client

Users that are interested in mlperf_client are comparing it to the libraries listed below

• onnx / turnkeyml
  No-code CLI designed for accelerating ONNX workflows
  ☆216Updated 4 months ago
• intel / intel-ai-assistant-builder
  Intel® AI Assistant Builder
  ☆117Updated last week
• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆94Updated 3 weeks ago
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆103Updated last week
• nomic-ai / llama.cpp
  llama.cpp fork used by GPT4All
  ☆57Updated 8 months ago
• google-ai-edge / LiteRT-LM
  ☆456Updated this week
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆349Updated last year
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆92Updated last week
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆67Updated last year
• andrewkchan / deepseek.cpp
  CPU inference for the DeepSeek family of large language models in C++
  ☆315Updated last month
• exo-explore / mlx-bitnet
  1.58 Bit LLM on Apple Silicon using MLX
  ☆225Updated last year
• Chen-zexi / vllm-cli
  A command-line interface tool for serving LLM using vLLM.
  ☆440Updated 3 weeks ago
• janhq / cortex.tensorrt-llm
  Cortex.Tensorrt-LLM is a C++ inference library that can be loaded by any server at runtime. It submodules NVIDIA’s TensorRT-LLM for GPU a…
  ☆42Updated last year
• anthonix / llm.c
  LLM training in simple, raw C/HIP for AMD GPUs
  ☆53Updated last year
• sgl-project / sgl-project.github.io
  This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.
  ☆88Updated this week
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆154Updated last week
• NVIDIA / RTX-AI-Toolkit
  The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…
  ☆176Updated 11 months ago
• IST-DASLab / Quartet
  ☆105Updated last week
• apple / ml-recurrent-drafter
  ☆218Updated 9 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 8 months ago
• openvinotoolkit / openvino.genai
  Run Generative AI models with simple C++/Python API and using OpenVINO Runtime
  ☆368Updated this week
• google / minja
  A minimalistic C++ Jinja templating engine for LLM chat templates
  ☆195Updated last month
• AlexBodner / How_Much_VRAM
  ☆102Updated last year
• Cerebras / DocChat
  GPT-4 Level Conversational QA Trained In a Few Hours
  ☆65Updated last year
• unslothai / unsloth-zoo
  Utils for Unsloth https://github.com/unslothai/unsloth
  ☆165Updated this week
• NVIDIA-AI-Blueprints / llm-router
  Route LLM requests to the best model for the task at hand.
  ☆122Updated last week
• Cornell-RelaxML / yaqa-quantization
  ☆60Updated 4 months ago
• run-ai / runai-model-streamer
  ☆264Updated this week
• Infini-AI-Lab / UMbreLLa
  LLM Inference on consumer devices
  ☆125Updated 7 months ago
• hscspring / llama.np
  Inference Llama/Llama2/Llama3 Modes in NumPy
  ☆21Updated last year