Alternatives and similar repositories for mlperf_client

Users that are interested in mlperf_client are comparing it to the libraries listed below

• AlexBodner / How_Much_VRAM
  ☆102Updated last year
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆91Updated 6 months ago
• Cerebras / DocChat
  GPT-4 Level Conversational QA Trained In a Few Hours
  ☆64Updated last year
• nomic-ai / llama.cpp
  llama.cpp fork used by GPT4All
  ☆56Updated 6 months ago
• unslothai / llama.cpp
  LLM inference in C/C++
  ☆101Updated last week
• onnx / turnkeyml
  No-code CLI designed for accelerating ONNX workflows
  ☆210Updated 2 months ago
• google-ai-edge / LiteRT-LM
  ☆315Updated this week
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆131Updated 2 months ago
• NVIDIA / RTX-AI-Toolkit
  The NVIDIA RTX™ AI Toolkit is a suite of tools and SDKs for Windows developers to customize, optimize, and deploy AI models across RTX PC…
  ☆170Updated 9 months ago
• Cornell-RelaxML / yaqa-quantization
  ☆54Updated 2 months ago
• the-crypt-keeper / tcurtsni
  Tcurtsni: Reverse Instruction Chat, ever wonder what your LLM wants to ask you?
  ☆22Updated last year
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆348Updated last year
• cpldcpu / LRMTokenEconomy
  Measuring Thinking Efficiency in Reasoning Models - Research Repository
  ☆31Updated this week
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆55Updated 9 months ago
• catid / bitnet_cpu
  Experiments with BitNet inference on CPU
  ☆54Updated last year
• menloresearch / cortex.tensorrt-llm
  Cortex.Tensorrt-LLM is a C++ inference library that can be loaded by any server at runtime. It submodules NVIDIA’s TensorRT-LLM for GPU a…
  ☆42Updated 11 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆140Updated this week
• google / minja
  A minimalistic C++ Jinja templating engine for LLM chat templates
  ☆171Updated 3 weeks ago
• QuixiAI / dolphin-logger
  ☆102Updated 2 months ago
• microsoft / GRIN-MoE
  GRadient-INformed MoE
  ☆264Updated 11 months ago
• huggingface / optimum-amd
  AMD related optimizations for transformer models
  ☆83Updated last week
• nyunAI / PruneGPT
  ☆51Updated last year
• run-ai / runai-model-streamer
  ☆238Updated this week
• Codys12 / airllm
  AirLLM 70B inference with single 4GB GPU
  ☆14Updated 2 months ago
• huseinzol05 / transformers-continuous-batching
  Lightweight continuous batching OpenAI compatibility using HuggingFace Transformers include T5 and Whisper.
  ☆26Updated 5 months ago
• ibm-granite / granite-3.0-language-models
  ☆262Updated 2 months ago
• sgl-project / sgl-project.github.io
  This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.
  ☆72Updated this week
• deepreinforce-ai / CUDA-L1
  CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
  ☆182Updated 3 weeks ago
• fairydreaming / llama.cpp
  LLM inference in C/C++
  ☆21Updated 5 months ago
• IST-DASLab / Quartet
  ☆85Updated last week