coreweave / ml-containersLinks
☆42Updated 2 weeks ago
Alternatives and similar repositories for ml-containers
Users that are interested in ml-containers are comparing it to the libraries listed below
Sorting:
- Module, Model, and Tensor Serialization/Deserialization☆272Updated 2 months ago
- IBM development fork of https://github.com/huggingface/text-generation-inference☆62Updated last month
- Kubernetes Operator, ansible playbooks, and production scripts for large-scale AIStore deployments on Kubernetes.☆110Updated 3 weeks ago
- High-performance safetensors model loader☆71Updated this week
- Helm charts for llm-d☆50Updated 3 months ago
- ☆264Updated last week
- A top-like tool for monitoring GPUs in a cluster☆85Updated last year
- Pretrain, finetune and serve LLMs on Intel platforms with Ray☆131Updated last month
- vLLM adapter for a TGIS-compatible gRPC server.☆44Updated this week
- A collection of reproducible inference engine benchmarks☆37Updated 6 months ago
- xet client tech, used in huggingface_hub☆316Updated this week
- Repository for open inference protocol specification☆59Updated 6 months ago
- xpk (Accelerated Processing Kit, pronounced x-p-k,) is a software tool to help Cloud developers to orchestrate training jobs on accelerat…☆151Updated this week
- CUDA checkpoint and restore utility☆381Updated last month
- ☆31Updated 6 months ago
- JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs wel…☆388Updated 5 months ago
- MLFlow Deployment Plugin for Ray Serve☆46Updated 3 years ago
- Pipeline parallelism for the minimalist☆37Updated 3 months ago
- Benchmark suite for LLMs from Fireworks.ai☆83Updated 2 weeks ago
- OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)☆307Updated last week
- Transformer GPU VRAM estimator☆66Updated last year
- Some microbenchmarks and design docs before commencement☆12Updated 4 years ago
- Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)☆28Updated 2 years ago
- This is a landscape of the infrastructure that powers the generative AI ecosystem☆149Updated last year
- Simple dependency injection framework for Python☆21Updated last year
- ☆316Updated last year
- Home for OctoML PyTorch Profiler☆114Updated 2 years ago
- GPU Environment Management for Visual Studio Code☆39Updated 2 years ago
- MLPerf™ logging library☆37Updated 3 weeks ago
- Rust crates for XetHub☆71Updated last year