Alternatives and similar repositories for olah

Users that are interested in olah are comparing it to the libraries listed below

• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆236Updated this week
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆185Updated last week
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆270Updated last year
• run-ai / runai-model-streamer
  ☆228Updated last week
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆134Updated last week
• nvwacloud / tensorlink
  Unlock Unlimited Potential! Share Your GPU Power Across Your Local Network!
  ☆61Updated last month
• huggingface / xet-core
  xet client tech, used in huggingface_hub
  ☆127Updated this week
• huggingface / hf_transfer
  ☆483Updated 3 months ago
• sasha0552 / pascal-pkgs-ci
  The main repository for building Pascal-compatible versions of ML applications and libraries.
  ☆100Updated last month
• NVIDIA / vgpu-device-manager
  NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes
  ☆136Updated last week
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆28Updated 3 months ago
• leptonai / gpud
  GPUd automates monitoring, diagnostics, and issue identification for GPUs
  ☆387Updated this week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆68Updated last year
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆209Updated 11 months ago
• matpool / mpu
  A shim driver allows in-docker nvidia-smi showing correct process list without modify anything
  ☆88Updated 2 weeks ago
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆134Updated 2 weeks ago
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆155Updated last year
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆219Updated 7 months ago
• nekomeowww / ollama-operator
  🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
  ☆196Updated this week
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆248Updated last week
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆129Updated last week
• coreweave / kubernetes-cloud
  Getting Started with the CoreWeave Kubernetes GPU Cloud
  ☆73Updated last month
• huggingface / frp
  FRP Fork
  ☆171Updated 3 months ago
• replicate / pget
  parallel fetch
  ☆134Updated last week
• vllm-project / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆405Updated this week
• woct0rdho / transformers-qwen3-moe-fused
  Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth
  ☆122Updated this week
• akx / ggify
  Tool to download models from Huggingface Hub and convert them to GGML/GGUF for llama.cpp
  ☆154Updated 2 months ago
• akx / ollama-dl
  Download models from the Ollama library, without Ollama
  ☆89Updated 8 months ago
• clearml / clearml-fractional-gpu
  ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing
  ☆78Updated 11 months ago
• tc-mb / ollama
  Get up and running with Llama 3, Mistral, Gemma, and other large language models.
  ☆27Updated this week