Alternatives and similar repositories for olah:

Users that are interested in olah are comparing it to the libraries listed below

• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆70Updated 2 months ago
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆131Updated this week
• run-ai / runai-model-streamer
  ☆169Updated this week
• substratusai / stapi
  Sentence Transformers API: An OpenAI compatible embedding API server
  ☆46Updated 6 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆63Updated 11 months ago
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆147Updated 8 months ago
• nekomeowww / ollama-operator
  🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫
  ☆166Updated this week
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆200Updated 7 months ago
• replicate / pget
  parallel fetch
  ☆119Updated this week
• OpenBMB / ollama
  Get up and running with Llama 3, Mistral, Gemma, and other large language models.
  ☆26Updated this week
• leptonai / gpud
  GPUd automates monitoring, diagnostics, and issue identification for GPUs
  ☆286Updated this week
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆257Updated last year
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆217Updated 3 weeks ago
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆120Updated 2 weeks ago
• padeoe / hf-mirror-site
  a huggingface mirror site.
  ☆272Updated 11 months ago
• Yard1 / Ray-DeepSpeed-Inference
  ☆17Updated last year
• muyuworks / myla
  a local implementation of OpenAI Assistants API: myla stands for MY Local Assistant
  ☆53Updated 6 months ago
• chu-tianxiang / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆130Updated 8 months ago
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆127Updated last week
• nyunAI / PruneGPT
  ☆53Updated 9 months ago
• clearml / clearml-fractional-gpu
  ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing
  ☆73Updated 7 months ago
• coreweave / ml-containers
  ☆30Updated this week
• karminski / streaming-json-py
  A streamlined, user-friendly JSON streaming preprocessor, crafted in Python.
  ☆96Updated 5 months ago
• substratusai / vllm-docker
  ☆54Updated 2 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆69Updated last month
• triton-inference-server / vllm_backend
  ☆235Updated this week
• huggingface / hf_transfer
  ☆430Updated last week
• InftyAI / Awesome-LLMOps
  🎉 An awesome & curated list of best LLMOps tools.
  ☆42Updated this week