Alternatives and similar repositories for benchmarks:

Users that are interested in benchmarks are comparing it to the libraries listed below

• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆224Updated this week
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated 9 months ago
• huggingface / llm-swarm
  Manage scalable open LLM inference endpoints in Slurm clusters
  ☆253Updated 8 months ago
• mixedbread-ai / batched
  The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…
  ☆125Updated 3 months ago
• Preemo-Inc / text-generation-inference
  ☆199Updated last year
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆262Updated 5 months ago
• cognitivecomputations / spectrum
  ☆113Updated 5 months ago
• Locutusque / TPU-Alignment
  Fully fine-tune large models like Mistral, Llama-2-13B, or Qwen-14B completely for free
  ☆230Updated 4 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆208Updated 3 months ago
• h2oai / enterprise-h2ogpte
  Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform
  ☆83Updated last week
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆87Updated this week
• astramind-ai / BitMat
  An efficent implementation of the method proposed in "The Era of 1-bit LLMs"
  ☆154Updated 5 months ago
• huggingface / optimum-benchmark
  🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of O…
  ☆289Updated last month
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆94Updated this week
• mixedbread-ai / baguetter
  Baguetter is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, imp…
  ☆173Updated 6 months ago
• thomasgauthier / LoRD
  Low-Rank adapter extraction for fine-tuned transformers models
  ☆171Updated 10 months ago
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆201Updated 7 months ago
• pacman100 / openhathi_instruct
  This repository contains the code for dataset curation and finetuning of instruct variant of the Bilingual OpenHathi model. The resultin…
  ☆23Updated last year
• intel / neural-speed
  An innovative library for efficient LLM inference via low-bit quantization
  ☆351Updated 6 months ago
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆196Updated 8 months ago
• cohere-ai / DiskVectorIndex
  ☆208Updated 8 months ago
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆58Updated 2 months ago
• arcee-ai / EvolKit
  EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language M…
  ☆207Updated 4 months ago
• DeployQL / LintDB
  Vector Database with support for late interaction and token level embeddings.
  ☆53Updated 5 months ago
• triton-inference-server / vllm_backend
  ☆237Updated last week
• daniel-furman / sft-demos
  Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
  ☆73Updated 5 months ago
• cognitivecomputations / laserRMT
  This is our own implementation of 'Layer Selective Rank Reduction'
  ☆233Updated 9 months ago
• taprosoft / llm_finetuning
  Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…
  ☆147Updated last year
• anyscale / e2e-llm-workflows
  End-to-End LLM Guide
  ☆104Updated 8 months ago
• PrithivirajDamodaran / Route0x
  Low latency, High Accuracy, Custom Query routers for Humans and Agents. Built by Prithivi Da
  ☆101Updated last week