coreweave / ml-containersLinks
☆43Updated this week
Alternatives and similar repositories for ml-containers
Users that are interested in ml-containers are comparing it to the libraries listed below
Sorting:
- Module, Model, and Tensor Serialization/Deserialization☆283Updated 4 months ago
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆118Updated this week
- IBM development fork of https://github.com/huggingface/text-generation-inference☆62Updated 3 months ago
- ☆275Updated last week
- Helm charts for llm-d☆52Updated 5 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆131Updated 3 months ago
- Benchmark suite for LLMs from Fireworks.ai☆84Updated last month
- High-performance safetensors model loader☆92Updated 3 weeks ago
- xpk (Accelerated Processing Kit, pronounced x-p-k,) is a software tool to help Cloud developers to orchestrate training jobs on accelerat…☆159Updated this week
- Home for OctoML PyTorch Profiler☆114Updated 2 years ago
- ☆31Updated 8 months ago
- CUDA checkpoint and restore utility☆401Updated 3 months ago
- WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.☆17Updated 3 years ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆398Updated last week
- vLLM adapter for a TGIS-compatible gRPC server.☆47Updated this week
- An experimental implementation of compiler-driven automatic sharding of models across a given device mesh.☆48Updated this week
- A top-like tool for monitoring GPUs in a cluster☆84Updated last year
- ☆59Updated last week
- The driver for LMCache core to run in vLLM☆58Updated 11 months ago
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆251Updated this week
- Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, T…☆355Updated this week
- Simple dependency injection framework for Python☆21Updated last year
- ☆321Updated last year
- NVIDIA NCCL Tests for Distributed Training☆132Updated this week
- A collection of reproducible inference engine benchmarks☆38Updated 8 months ago
- Repository for open inference protocol specification☆61Updated 8 months ago
- Pipeline parallelism for the minimalist☆37Updated 5 months ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆468Updated last week
- MLPerf™ logging library☆38Updated 3 weeks ago
- ☆13Updated 2 years ago