Alternatives and similar repositories for olah:

Users that are interested in olah are comparing it to the libraries listed below

• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆263Updated last year
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆165Updated this week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆65Updated last year
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆99Updated this week
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆145Updated this week
• replicate / pget
  parallel fetch
  ☆123Updated this week
• run-ai / runai-model-streamer
  ☆191Updated 2 weeks ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆102Updated this week
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆223Updated last month
• nekomeowww / ollama-operator
  🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
  ☆184Updated this week
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆29Updated 3 weeks ago
• nvwacloud / tensorlink
  Unlock Unlimited Potential! Share Your GPU Power Across Your Local Network!
  ☆52Updated 9 months ago
• huggingface / xet-core
  xet client tech, used in huggingface_hub
  ☆80Updated this week
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆205Updated 8 months ago
• cloud-gpus / cloud-gpus.github.io
  ☆48Updated 2 weeks ago
• jquesnelle / transformers-openai-api
  An OpenAI Completions API compatible server for NLP transformers models
  ☆65Updated last year
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆70Updated 2 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆212Updated 4 months ago
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆151Updated 9 months ago
• Yard1 / Ray-DeepSpeed-Inference
  ☆17Updated 2 years ago
• bentoml / llm-bench
  ☆49Updated 4 months ago
• huggingface / hf_transfer
  ☆450Updated this week
• sasha0552 / pascal-pkgs-ci
  The main repository for building Pascal-compatible versions of ML applications and libraries.
  ☆68Updated last week
• wangcx18 / llm-vscode-inference-server
  An endpoint server for efficiently serving quantized open-source LLMs for code.
  ☆54Updated last year
• coreweave / kubernetes-cloud
  Getting Started with the CoreWeave Kubernetes GPU Cloud
  ☆70Updated last month
• canvrno / ProxmoxMCP
  MCP for Proxmox integration in Cline
  ☆77Updated last month
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆26Updated this week
• LLM-inference-router / vllm-router
  vLLM Router
  ☆26Updated last year
• coreweave / ml-containers
  ☆30Updated this week
• triton-inference-server / vllm_backend
  ☆241Updated this week