Alternatives and similar repositories for olah

Users that are interested in olah are comparing it to the libraries listed below

• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆173Updated this week
• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆118Updated this week
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆203Updated last week
• chaunceyjiang / fake-gpu
  This project is designed to simulate GPU information, making it easier to test scenarios where a GPU is not available.
  ☆45Updated 3 months ago
• matpool / mpu
  A shim driver allows in-docker nvidia-smi showing correct process list without modify anything
  ☆86Updated 2 months ago
• run-ai / runai-model-streamer
  ☆215Updated this week
• huggingface / xet-core
  xet client tech, used in huggingface_hub
  ☆111Updated this week
• SlinkyProject / slurm-operator
  Run Slurm on Kubernetes. A Slinky project.
  ☆108Updated last week
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆208Updated 10 months ago
• NVIDIA / vgpu-device-manager
  NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes
  ☆131Updated this week
• leptonai / gpud
  GPUd automates monitoring, diagnostics, and issue identification for GPUs
  ☆367Updated this week
• huggingface / hf_transfer
  ☆468Updated last month
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆267Updated last year
• Yard1 / Ray-DeepSpeed-Inference
  ☆17Updated 2 years ago
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆127Updated last month
• tensorchord / deepseek-api-arena
  A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.
  ☆29Updated 2 months ago
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆234Updated last week
• nekomeowww / ollama-operator
  🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
  ☆192Updated this week
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆114Updated last week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆67Updated last year
• tc-mb / ollama
  Get up and running with Llama 3, Mistral, Gemma, and other large language models.
  ☆26Updated 3 weeks ago
• and270 / thinking_effort_processor
  ☆89Updated 2 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆64Updated last year
• makllama / makllama
  MaK(Mac+Kubernetes)llama - Running and orchestrating large language models (LLMs) on Kubernetes with macOS nodes.
  ☆40Updated last year
• ray-project / kuberay-helm
  Helm charts for the KubeRay project
  ☆43Updated 2 months ago
• huggingface / frp
  FRP Fork
  ☆166Updated last month
• coreweave / kubernetes-cloud
  Getting Started with the CoreWeave Kubernetes GPU Cloud
  ☆71Updated 2 months ago
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆153Updated 10 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆75Updated 3 weeks ago
• tensorchord / modelz-llm
  OpenAI compatible API for LLMs and embeddings (LLaMA, Vicuna, ChatGLM and many others)
  ☆275Updated last year