coreweave / ml-containersLinks
☆38Updated this week
Alternatives and similar repositories for ml-containers
Users that are interested in ml-containers are comparing it to the libraries listed below
Sorting:
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆106Updated this week
- ☆231Updated this week
- Module, Model, and Tensor Serialization/Deserialization☆250Updated this week
- High-performance safetensors model loader☆52Updated 2 weeks ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆61Updated 2 months ago
- Helm charts for llm-d☆51Updated last week
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆128Updated 3 weeks ago
- ☆30Updated 3 months ago
- CUDA checkpoint and restore utility☆353Updated 6 months ago
- Benchmark suite for LLMs from Fireworks.ai☆76Updated 3 weeks ago
- xet client tech, used in huggingface_hub☆140Updated last week
- A collection of reproducible inference engine benchmarks☆32Updated 3 months ago
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Updated 2 years ago
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆202Updated this week
- Repository for open inference protocol specification☆59Updated 2 months ago
- A benchmarking tool for comparing different LLM API providers' DeepSeek model deployments.☆28Updated 4 months ago
- The driver for LMCache core to run in vLLM☆45Updated 5 months ago
- A top-like tool for monitoring GPUs in a cluster☆85Updated last year
- vLLM adapter for a TGIS-compatible gRPC server.☆33Updated this week
- ☆15Updated 4 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆364Updated last month
- ☆55Updated 8 months ago
- The NVIDIA GPU driver container allows the provisioning of the NVIDIA driver through the use of containers.☆121Updated last week
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆401Updated this week
- ☆313Updated 11 months ago
- An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.☆119Updated this week
- Home for OctoML PyTorch Profiler☆113Updated 2 years ago
- Cloud Native Benchmarking of Foundation Models☆39Updated this week
- Transformer GPU VRAM estimator☆66Updated last year
- MLFlow Deployment Plugin for Ray Serve☆46Updated 3 years ago