Alternatives and similar repositories for mlperf_client

Users that are interested in mlperf_client are comparing it to the libraries listed below

• unslothai / llama.cpp
  LLM inference in C/C++
  ☆101Updated last month
• Cerebras / DocChat
  GPT-4 Level Conversational QA Trained In a Few Hours
  ☆64Updated last year
• intel / intel-ai-assistant-builder
  Intel® AI Assistant Builder
  ☆106Updated this week
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆66Updated last year
• onnx / turnkeyml
  No-code CLI designed for accelerating ONNX workflows
  ☆214Updated 3 months ago
• AlexBodner / How_Much_VRAM
  ☆102Updated last year
• google-ai-edge / LiteRT-LM
  ☆338Updated this week
• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆88Updated last month
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 6 months ago
• IST-DASLab / gptq-gguf-toolkit
  GPTQ and efficient search for GGUF
  ☆48Updated last week
• Cornell-RelaxML / yaqa-quantization
  ☆57Updated 3 months ago
• Infini-AI-Lab / UMbreLLa
  LLM Inference on consumer devices
  ☆124Updated 6 months ago
• sgl-project / sgl-project.github.io
  This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.
  ☆78Updated this week
• IST-DASLab / Quartet
  ☆97Updated last month
• exo-explore / mlx-bitnet
  1.58 Bit LLM on Apple Silicon using MLX
  ☆223Updated last year
• Chen-zexi / vllm-cli
  A command-line interface tool for serving LLM using vLLM.
  ☆414Updated last month
• QuixiAI / dolphin-logger
  ☆104Updated 3 months ago
• charmandercha / ArchiDoc
  ☆17Updated 9 months ago
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆348Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆150Updated 2 weeks ago
• wdlctc / headinfer
  ☆57Updated 4 months ago
• nomic-ai / llama.cpp
  llama.cpp fork used by GPT4All
  ☆56Updated 7 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆132Updated 3 months ago
• mzbac / mlx_sharding
  Distributed Inference for mlx LLm
  ☆95Updated last year
• rafacelente / bllama
  1.58-bit LLaMa model
  ☆82Updated last year
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆89Updated this week
• reka-ai / rekaquant
  ☆62Updated 2 months ago
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆55Updated 10 months ago
• google / minja
  A minimalistic C++ Jinja templating engine for LLM chat templates
  ☆181Updated 2 weeks ago
• andrewkchan / deepseek.cpp
  CPU inference for the DeepSeek family of large language models in C++
  ☆314Updated 3 months ago