vtuber-plan / olahLinks
Self-hosted huggingface mirror service. 自建huggingface镜像服务。
☆189Updated last month
Alternatives and similar repositories for olah
Users that are interested in olah are comparing it to the libraries listed below
Sorting:
- LM inference server implementation based on *.cpp.☆271Updated 2 weeks ago
- xet client tech, used in huggingface_hub☆183Updated this week
- Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)☆272Updated last year
- Review/Check GGUF files and estimate the memory usage and maximum tokens per second.☆202Updated 2 weeks ago
- A shim driver allows in-docker nvidia-smi showing correct process list without modify anything☆93Updated last month
- ☆502Updated 4 months ago
- The main repository for building Pascal-compatible versions of ML applications and libraries.☆121Updated last week
- ☆238Updated this week
- A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.☆71Updated last year
- A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.☆29Updated 5 months ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆415Updated this week
- Module, Model, and Tensor Serialization/Deserialization☆260Updated last week
- 🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫☆208Updated last week
- a huggingface mirror site.☆296Updated last year
- A text-to-speech and speech-to-text server compatible with the OpenAI API, supporting Whisper, FunASR, Bark, and CosyVoice backends.☆154Updated last month
- This is the documentation repository for SGLang. It is auto-generated from https://github.com/sgl-project/sglang/tree/main/docs.☆72Updated this week
- ☆38Updated this week
- ClearML Fractional GPU - Run multiple containers on the same GPU with driver level memory limitation ✨ and compute time-slicing☆79Updated last year
- OpenAI compatible API for TensorRT LLM triton backend☆214Updated last year
- NVIDIA vGPU Device Manager manages NVIDIA vGPU devices on top of Kubernetes☆141Updated last week
- Open Source Text Embedding Models with OpenAI Compatible API☆159Updated last year
- Getting Started with the CoreWeave Kubernetes GPU Cloud☆74Updated 2 months ago
- Triton Inference Server Web UI☆15Updated last year
- Download models from the Ollama library, without Ollama☆95Updated 9 months ago
- An Envoy inspired, ultimate LLM-first gateway for LLM serving and downstream application developers and enterprises☆23Updated 4 months ago
- Comparison of Language Model Inference Engines☆229Updated 8 months ago
- The LLM API Benchmark Tool is a flexible Go-based utility designed to measure and analyze the performance of OpenAI-compatible API endpoi…☆38Updated last week
- ☆63Updated 5 months ago
- Unlock Unlimited Potential! Share Your GPU Power Across Your Local Network!☆64Updated 3 months ago
- 🎉 An awesome & curated list of best LLMOps tools.☆149Updated last week