Alternatives and similar repositories for benchmarks:

Users that are interested in benchmarks are comparing it to the libraries listed below

• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated 10 months ago
• Preemo-Inc / text-generation-inference
  ☆199Updated last year
• arcee-ai / EvolKit
  EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language M…
  ☆210Updated 5 months ago
• daniel-furman / sft-demos
  Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
  ☆73Updated 5 months ago
• huggingface / llm-swarm
  Manage scalable open LLM inference endpoints in Slurm clusters
  ☆254Updated 9 months ago
• cognitivecomputations / spectrum
  ☆112Updated this week
• Locutusque / TPU-Alignment
  Fully fine-tune large models like Mistral, Llama-2-13B, or Qwen-14B completely for free
  ☆231Updated 5 months ago
• mixedbread-ai / batched
  The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…
  ☆128Updated 3 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆262Updated 6 months ago
• arcee-ai / DALM
  Domain Adapted Language Modeling Toolkit - E2E RAG
  ☆319Updated 5 months ago
• cohere-ai / BinaryVectorDB
  Efficient vector database for hundred millions of embeddings.
  ☆205Updated 10 months ago
• mixedbread-ai / baguetter
  Baguetter is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, imp…
  ☆174Updated 7 months ago
• h2oai / enterprise-h2ogpte
  Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform
  ☆86Updated last month
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆248Updated this week
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆48Updated last week
• IBM / fastfit
  FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes
  ☆193Updated 6 months ago
• cohere-ai / DiskVectorIndex
  ☆209Updated 9 months ago
• cognitivecomputations / laserRMT
  This is our own implementation of 'Layer Selective Rank Reduction'
  ☆233Updated 10 months ago
• huggingface / competitions
  ☆122Updated 5 months ago
• taprosoft / llm_finetuning
  Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…
  ☆147Updated last year
• jlscheerer / xtr-warp
  XTR/WARP is an extremely fast and accurate retrieval engine based on Stanford's ColBERTv2/PLAID and Google DeepMind's XTR.
  ☆122Updated 5 months ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆100Updated this week
• AblateIt / finetune-study
  Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.
  ☆82Updated last year
• michaelfeil / embed
  A stable, fast and easy-to-use inference library with a focus on a sync-to-async API
  ☆45Updated 6 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆212Updated 3 months ago
• PrithivirajDamodaran / SPLADERunner
  Lite weight wrapper for the independent implementation of SPLADE++ models for search & retrieval pipelines. Models and Library created by…
  ☆30Updated 7 months ago
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆195Updated 8 months ago
• substratusai / vllm-docker
  ☆57Updated 2 weeks ago
• codelion / adaptive-classifier
  A flexible, adaptive classification system for dynamic text classification
  ☆150Updated last month
• astramind-ai / BitMat
  An efficent implementation of the method proposed in "The Era of 1-bit LLMs"
  ☆153Updated 5 months ago