Alternatives and similar repositories for mosec

Users that are interested in mosec are comparing it to the libraries listed below

• triton-inference-server / pytriton
  PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.
  ☆832Updated 5 months ago
• ray-project / ray-llm
  RayLLM - LLMs on Ray (Archived). Read README for more info.
  ☆1,263Updated 10 months ago
• triton-inference-server / fastertransformer_backend
  ☆413Updated 2 years ago
• bentoml / Yatai
  Model Deployment at Scale on Kubernetes 🦄️
  ☆833Updated last year
• triton-inference-server / python_backend
  Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.
  ☆667Updated this week
• deepspeedai / DeepSpeed-MII
  MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
  ☆2,092Updated 7 months ago
• hpcaitech / EnergonAI
  Large-scale model inference.
  ☆627Updated 2 years ago
• vectorch-ai / ScaleLLM
  A high-performance inference system for large language models, designed for production environments.
  ☆491Updated last month
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆279Updated 2 years ago
• punica-ai / punica
  Serving multiple LoRA finetuned LLM as one
  ☆1,140Updated last year
• triton-inference-server / model_analyzer
  Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Serv…
  ☆504Updated last week
• ray-project / llmperf
  LLMPerf is a library for validating and benchmarking LLMs
  ☆1,084Updated last year
• WarrenWen666 / AI-Software-Startups
  A Survey of AI startups
  ☆402Updated 2 years ago
• tensorchord / envd
  🏕️ Reproducible development environment for humans and agents
  ☆2,178Updated this week
• ELS-RD / transformer-deploy
  Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
  ☆1,688Updated last year
• triton-inference-server / vllm_backend
  ☆328Updated last week
• hpcaitech / SwiftInfer
  Efficient AI Inference & Serving
  ☆479Updated 2 years ago
• triton-inference-server / model_navigator
  Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.
  ☆217Updated last week
• jina-ai / vectordb
  A Python vector database you just need - no more, no less.
  ☆642Updated last year
• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆566Updated last year
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆220Updated last year
• mlc-ai / xgrammar
  Fast, Flexible and Portable Structured Generation
  ☆1,527Updated last week
• ninehills / llm-inference-benchmark
  LLM Inference benchmark
  ☆433Updated last year
• run-ai / genv
  GPU environment and cluster management with LLM support
  ☆656Updated last year
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆286Updated 5 months ago
• triton-inference-server / backend
  Common source, scripts and utilities for creating Triton backends.
  ☆366Updated last week
• BaguaSys / bagua
  Bagua Speeds up PyTorch
  ☆884Updated last year
• huggingface / hf_transfer
  ☆541Updated 4 months ago
• SeldonIO / MLServer
  An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
  ☆873Updated this week
• triton-inference-server / client
  Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
  ☆677Updated this week