Alternatives and similar repositories for olah:

Users that are interested in olah are comparing it to the libraries listed below

• gpustack / vox-box
  A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.
  ☆25Updated 2 weeks ago
• gpustack / llama-box
  LM inference server implementation based on *.cpp.
  ☆66Updated this week
• limcheekin / open-text-embeddings
  Open Source Text Embedding Models with OpenAI Compatible API
  ☆142Updated 6 months ago
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆248Updated last year
• run-ai / runai-model-streamer
  ☆153Updated last week
• Contextualist / lone-arena
  Self-hosted LLM chatbot arena, with yourself as the only judge
  ☆36Updated 11 months ago
• user1342 / Tomato
  LLM steganography with minimum-entropy coupling - Hiding encrypted messages in natural language.
  ☆78Updated 4 months ago
• nyunAI / PruneGPT
  ☆52Updated 7 months ago
• substratusai / stapi
  Sentence Transformers API: An OpenAI compatible embedding API server
  ☆41Updated 4 months ago
• jquesnelle / transformers-openai-api
  An OpenAI Completions API compatible server for NLP transformers models
  ☆62Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆79Updated last week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆60Updated 9 months ago
• h2oai / enterprise-h2ogpte
  Client Code Examples, Use Cases and Benchmarks for Enterprise h2oGPTe RAG-Based GenAI Platform
  ☆82Updated 3 weeks ago
• adrienbrault / hf-gguf-to-ollama
  Dagger functions to import Hugging Face GGUF models into a local ollama instance and optionally push them to ollama.com.
  ☆113Updated 8 months ago
• fixie-ai / ai-benchmarks
  Benchmarking suite for popular AI APIs
  ☆80Updated 2 months ago
• gpustack / gguf-parser-go
  Review/Check GGUF files and estimate the memory usage and maximum tokens per second.
  ☆74Updated 2 weeks ago
• substratusai / vllm-docker
  ☆52Updated 3 weeks ago
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆184Updated last month
• muyuworks / myla
  a local implementation of OpenAI Assistants API: myla stands for MY Local Assistant
  ☆50Updated 4 months ago
• Dan-wanna-M / formatron
  Formatron empowers everyone to control the format of language models' output with minimal overhead.
  ☆177Updated 3 weeks ago
• SivilTaram / code-html-to-markdown
  A lightweight script for processing HTML page to markdown format with support for code blocks
  ☆78Updated 9 months ago
• uukuguy / speechless
  LLM based agents with proactive interactions, long-term memory, external tool integration, and local deployment capabilities.
  ☆94Updated this week
• modelscope / modelscope-studio
  A third-party component library based on Gradio.
  ☆73Updated this week
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆187Updated 5 months ago
• BerriAI / liteLLM-proxy
  ☆132Updated 11 months ago
• rodrigobaron / anthill
  ☆25Updated last week
• hargup / reader
  Clone of https://r.jina.ai which is deployable locally
  ☆34Updated 4 months ago
• leafspark / AutoGGUF
  automatically quant GGUF models
  ☆151Updated this week
• lapp0 / lm-inference-engines
  Comparison of Language Model Inference Engines
  ☆203Updated last month
• AUGMXNT / deccp
  Evaling and unaligning Chinese LLM censorship
  ☆46Updated 4 months ago