coreweave / ml-containersLinks
☆40Updated this week
Alternatives and similar repositories for ml-containers
Users that are interested in ml-containers are comparing it to the libraries listed below
Sorting:
- Module, Model, and Tensor Serialization/Deserialization☆268Updated last month
- High-performance safetensors model loader☆62Updated 2 months ago
- ☆255Updated this week
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆111Updated last week
- IBM development fork of https://github.com/huggingface/text-generation-inference☆61Updated 2 weeks ago
- Helm charts for llm-d☆50Updated 2 months ago
- ☆31Updated 5 months ago
- A top-like tool for monitoring GPUs in a cluster☆85Updated last year
- CUDA checkpoint and restore utility☆371Updated 2 weeks ago
- xpk (Accelerated Processing Kit, pronounced x-p-k,) is a software tool to help Cloud developers to orchestrate training jobs on accelerat…☆143Updated this week
- Cloud Native Benchmarking of Foundation Models☆42Updated 2 months ago
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆132Updated last week
- Benchmark suite for LLMs from Fireworks.ai☆83Updated this week
- Distributed KV cache coordinator☆72Updated this week
- WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.☆18Updated 3 years ago
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆279Updated last week
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆149Updated 11 months ago
- vLLM adapter for a TGIS-compatible gRPC server.☆41Updated this week
- ☆59Updated last year
- Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…☆214Updated last week
- ☆13Updated 2 years ago
- The driver for LMCache core to run in vLLM☆51Updated 8 months ago
- NVIDIA NCCL Tests for Distributed Training☆111Updated last week
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Updated 2 years ago
- GPUd automates monitoring, diagnostics, and issue identification for GPUs☆436Updated last week
- Simple dependency injection framework for Python☆21Updated last year
- Optimized primitives for collective multi-GPU communication☆10Updated last year
- ☆315Updated last year
- A collection of reproducible inference engine benchmarks☆33Updated 5 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆380Updated 3 months ago