Alternatives and similar repositories for friendli-client

Users that are interested in friendli-client are comparing it to the libraries listed below

• friendliai / friendli-model-optimizer
  FMO (Friendli Model Optimizer)
  ☆12Updated 4 months ago
• friendliai / LLMServingPerfEvaluator
  ☆46Updated 8 months ago
• nexusflowai / NexusBench
  Nexusflow function call, tool use, and agent benchmarks.
  ☆19Updated 5 months ago
• friendliai / FAI-Model
  FriendliAI Model Hub
  ☆92Updated 2 years ago
• skypilot-org / skypilot-tutorial
  Tutorial to get started with SkyPilot!
  ☆57Updated last year
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated last year
• huggingface / optimum-furiosa
  Accelerated inference of 🤗 models using FuriosaAI NPU chips.
  ☆26Updated 11 months ago
• SqueezeBits / Torch-TRTLLM
  Ditto is an open-source framework that enables direct conversion of HuggingFace PreTrainedModels into TensorRT-LLM engines.
  ☆41Updated this week
• substratusai / vllm-docker
  ☆60Updated last month
• friendliai / periflow-cli
  Welcome to PeriFlow CLI ☁︎
  ☆12Updated last year
• cloneofsimo / fim-llama-deepspeed
  ☆31Updated last year
• DeepAuto-AI / hip-attention
  Training-free Post-training Efficient Sub-quadratic Complexity Attention. Implemented with OpenAI Triton.
  ☆130Updated this week
• dwarvesf / llm-hosting
  This repository is designed for deploying and managing server processes that handle embeddings using the Infinity Embedding model or Larg…
  ☆22Updated 2 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆87Updated 2 months ago
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆30Updated 3 weeks ago
• cg123 / rathe
  Tools for formatting large language model prompts.
  ☆13Updated last year
• ajtejankar / mixtral-vis-moe
  Visualize expert firing frequencies across sentences in the Mixtral MoE model
  ☆17Updated last year
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆29Updated this week
• simon-mo / vLLM-Benchmark
  ☆25Updated 3 weeks ago
• skypilot-org / skypilot-catalog
  ☆24Updated this week
• Snowflake-Labs / vllm
  ☆15Updated last month
• NEOS-AI / Neosearch
  AI-based search engine done right
  ☆16Updated this week
• deepgrove-ai / Bonsai
  ☆19Updated last month
• Birch-san / llama-play
  Command-line script for inferencing from models such as LLaMA, in a chat scenario, with LoRA adaptations
  ☆33Updated last year
• coreweave / ml-containers
  ☆32Updated this week
• runpod-workers / worker-sglang
  SGLang is fast serving framework for large language models and vision language models.
  ☆23Updated 3 months ago
• tyler-griggs / melange-release
  ☆45Updated 10 months ago
• rtzr / tritony
  Tiny configuration for Triton Inference Server
  ☆45Updated 4 months ago
• axiomic-ai / axiomic
  Creating Generative AI Apps which work
  ☆17Updated last month
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆116Updated 5 months ago