Alternatives and similar repositories for llm-perf-backend

Users that are interested in llm-perf-backend are comparing it to the libraries listed below

• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆32Updated 3 months ago
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆103Updated last year
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆59Updated last week
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated 2 years ago
• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆32Updated last year
• stas00 / ml-ways
  ML/DL Math and Method notes
  ☆66Updated 2 years ago
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆44Updated last year
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆139Updated last year
• titanml / takeoff-community
  TitanML Takeoff Server is an optimization, compression and deployment platform that makes state of the art machine learning models access…
  ☆114Updated last year
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆84Updated last month
• substratusai / vllm-docker
  ☆67Updated 9 months ago
• deshwalmahesh / PHUDGE
  Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…
  ☆51Updated last year
• Preemo-Inc / text-generation-inference
  ☆198Updated last year
• louisbrulenaudet / ragoon
  High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡
  ☆69Updated last month
• huggingface / huggingface-inference-toolkit
  Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.
  ☆89Updated last month
• hamelsmu / ft-drift
  Check for data drift between two OpenAI multi-turn chat jsonl files.
  ☆39Updated last year
• Upaya07 / NeurIPS-llm-efficiency-challenge
  Code for NeurIPS LLM Efficiency Challenge
  ☆59Updated last year
• muellerzr / nbdistributed
  Seemless interface of using PyTOrch distributed with Jupyter notebooks
  ☆57Updated 4 months ago
• anyscale / e2e-llm-workflows
  Fine-tune an LLM to perform batch inference and online serving.
  ☆115Updated 7 months ago
• ChrisHayduk / qlora-multi-gpu
  QLoRA with Enhanced Multi GPU Support
  ☆37Updated 2 years ago
• kevinwu23 / StanfordFineTuneBench
  ☆31Updated last year
• rragundez / chunkdot
  Multi-threaded matrix multiplication and cosine similarity calculations for dense and sparse matrices. Appropriate for calculating the K …
  ☆86Updated last year
• pacman100 / peft-codegen-25
  ☆23Updated 2 years ago
• davanstrien / data-for-fine-tuning-llms
  ☆80Updated last year
• AnswerDotAI / ModernBERT-Instruct-mini-cookbook
  ☆53Updated 11 months ago
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆119Updated last year
• s-smits / grpo-optuna
  Optimizing Causal LMs through GRPO with weighted reward functions and automated hyperparameter tuning using Optuna
  ☆59Updated 2 months ago
• awslabs / llm-hosting-container
  Large Language Model Hosting Container
  ☆91Updated 3 months ago
• TuanaCelik / unstructuredio-haystack
  💙 Unstructured Data Connectors for Haystack 2.0
  ☆17Updated 2 years ago
• krypticmouse / matryoshka-representation-learning
  PyTorch implementation for MRL
  ☆20Updated last year