Alternatives and similar repositories for friendli-client:

Users that are interested in friendli-client are comparing it to the libraries listed below

• friendliai / friendli-model-optimizer
  FMO (Friendli Model Optimizer)
  ☆12Updated 2 months ago
• friendliai / LLMServingPerfEvaluator
  ☆45Updated 6 months ago
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆19Updated 3 months ago
• friendliai / FAI-Model
  FriendliAI Model Hub
  ☆92Updated 2 years ago
• skypilot-org / skypilot-catalog
  ☆22Updated this week
• friendliai / periflow-cli
  Welcome to PeriFlow CLI ☁︎
  ☆12Updated last year
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆25Updated this week
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated last year
• ml-energy / leaderboard
  How much energy do GenAI models consume?
  ☆42Updated 5 months ago
• runpod-workers / worker-sglang
  SGLang is fast serving framework for large language models and vision language models.
  ☆20Updated last month
• jaywonchung / dotfiles
  Dotfile management with bare git
  ☆19Updated last week
• friendliai / aipm
  AI Agent who manages your Jira project
  ☆16Updated 9 months ago
• DeepAuto-AI / hip-attention
  Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.
  ☆128Updated this week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆82Updated last month
• SqueezeBits / Torch-TRTLLM
  Ditto is an open-source framework that enables direct conversion of HuggingFace PreTrainedModels into TensorRT-LLM engines.
  ☆31Updated this week
• tyler-griggs / melange-release
  ☆45Updated 9 months ago
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆60Updated 3 months ago
• langchain-ai / langchain-upstage
  ☆11Updated last month
• snuspl / cruise
  Cruise: A Distributed Machine Learning Framework with Automatic System Configuration
  ☆26Updated 6 years ago
• rtzr / tritony
  Tiny configuration for Triton Inference Server
  ☆45Updated 2 months ago
• substratusai / vllm-docker
  ☆56Updated this week
• yujonglee / eval
  Evaluate your LLM apps, RAG pipeline, any generated text, and more!
  Updated 10 months ago
• StableFluffy / EasyLLMFeaturePorter
  1-Click is all you need.
  ☆59Updated 11 months ago
• yale-sys / prompt-cache
  Modular and structured prompt caching for low-latency LLM inference
  ☆89Updated 4 months ago
• swsnu / aisys2023
  ☆102Updated last year
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆87Updated this week
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆111Updated 3 months ago
• moreh-io / motif-llm
  ☆31Updated 4 months ago
• deepgrove-ai / Bonsai
  ☆17Updated last week
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆73Updated last month