Alternatives and similar repositories for olah

Users that are interested in olah are comparing it to the libraries listed below

• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆273Updated last month
• huggingface / xet-core
  xet client tech, used in huggingface_hub
  ☆219Updated this week
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆205Updated last month
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆29Updated 5 months ago
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆274Updated last year
• run-ai / runai-model-streamer
  ☆249Updated this week
• matpool / mpu
  A shim driver allows in-docker nvidia-smi showing correct process list without modify anything
  ☆93Updated 2 months ago
• leptonai / gpud
  GPUd automates monitoring, diagnostics, and issue identification for GPUs
  ☆427Updated last week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆72Updated last year
• huggingface / hf_transfer
  ☆509Updated 5 months ago
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆161Updated 2 months ago
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆214Updated last year
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆265Updated last month
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆149Updated last week
• rag-wtf / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆160Updated last year
• nvwacloud / tensorlink
  Unlock Unlimited Potential! Share Your GPU Power Across Your Local Network!
  ☆65Updated 4 months ago
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆229Updated 9 months ago
• LLM-inference-router / vllm-router
  vLLM Router
  ☆43Updated last year
• clearml / clearml-fractional-gpu
  ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing
  ☆80Updated last year
• nekomeowww / ollama-operator
  🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
  ☆216Updated last week
• substratusai / stapi
  Sentence Transformers API: An OpenAI compatible embedding API server
  ☆67Updated last year
• sasha0552 / pascal-pkgs-ci
  The main repository for building Pascal-compatible versions of ML applications and libraries.
  ☆128Updated last month
• substratusai / vllm-docker
  ☆64Updated 5 months ago
• woct0rdho / transformers-qwen3-moe-fused
  Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth
  ☆176Updated this week
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆50Updated 7 months ago
• replicate / pget
  parallel fetch
  ☆138Updated last week
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆83Updated 2 weeks ago
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆132Updated 2 weeks ago
• tensorchord / modelz-llm
  OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)
  ☆276Updated last year
• LlmKira / fast-langdetect
  ⚡️ 80x faster Fasttext language detection out of the box | Split text by language
  ☆238Updated this week