Alternatives and similar repositories for runai-model-streamer

Users that are interested in runai-model-streamer are comparing it to the libraries listed below

• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆240Updated last week
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆73Updated last month
• NVIDIA / cuda-checkpoint
  CUDA checkpoint and restore utility
  ☆345Updated 4 months ago
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆129Updated last month
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆264Updated 8 months ago
• pytorch / torchft
  PyTorch per step fault tolerance (actively under development)
  ☆304Updated last week
• triton-inference-server / vllm_backend
  ☆264Updated last week
• huggingface / gpu-fryer
  Where GPUs get cooked 👩‍🍳🔥
  ☆234Updated 3 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆76Updated 2 weeks ago
• leptonai / gpud
  GPUd automates monitoring, diagnostics, and issue identification for GPUs
  ☆375Updated this week
• run-ai / llmperf
  ☆55Updated 9 months ago
• NVIDIA / nvidia-resiliency-ext
  NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …
  ☆177Updated 2 weeks ago
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆348Updated this week
• NVIDIA / k8s-nim-operator
  An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.
  ☆114Updated this week
• foundation-model-stack / fastsafetensors
  High-performance safetensors model loader
  ☆39Updated last week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆67Updated last year
• coreweave / ml-containers
  ☆36Updated this week
• imbue-ai / cluster-health
  ☆310Updated 10 months ago
• triton-inference-server / triton_distributed
  ☆49Updated 3 months ago
• snowflakedb / ArcticInference
  ☆155Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆60Updated last month
• foundation-model-stack / fms-fsdp
  🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…
  ☆253Updated this week
• ai-dynamo / nixl
  NVIDIA Inference Xfer Library (NIXL)
  ☆413Updated this week
• run-ai / rntop
  A top-like tool for monitoring GPUs in a cluster
  ☆85Updated last year
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆209Updated 10 months ago
• huggingface / kernels
  Load compute kernels from the Hub
  ☆172Updated this week
• coreweave / nccl-tests
  NVIDIA NCCL Tests for Distributed Training
  ☆97Updated this week
• NVIDIA / KAI-Scheduler
  KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale
  ☆656Updated this week
• kserve / modelmesh-serving
  Controller for ModelMesh
  ☆232Updated last week
• neuralmagic / AutoFP8
  ☆194Updated last month