Alternatives and similar repositories for mosec:

Users that are interested in mosec are comparing it to the libraries listed below

• deepspeedai / DeepSpeed-MII
  MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
  ☆1,990Updated this week
• triton-inference-server / pytriton
  PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.
  ☆781Updated last month
• ray-project / ray-llm
  RayLLM - LLMs on Ray (Archived). Read README for more info.
  ☆1,263Updated last week
• tensorchord / openmodelz
  Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)
  ☆260Updated last year
• punica-ai / punica
  Serving multiple LoRA finetuned LLM as one
  ☆1,040Updated 10 months ago
• triton-inference-server / fastertransformer_backend
  ☆411Updated last year
• huggingface / optimum
  🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization…
  ☆2,812Updated 2 weeks ago
• ELS-RD / transformer-deploy
  Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
  ☆1,680Updated 5 months ago
• srush / llama2.rs
  A fast llama2 decoder in pure Rust.
  ☆1,046Updated last year
• intel / intel-extension-for-transformers
  ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…
  ☆2,161Updated 5 months ago
• jina-ai / vectordb
  A Python vector database you just need - no more, no less.
  ☆597Updated last year
• unum-cloud / uform
  Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and 🔜 video, up to 5x faster than…
  ☆1,099Updated 2 months ago
• S-LoRA / S-LoRA
  S-LoRA: Serving Thousands of Concurrent LoRA Adapters
  ☆1,801Updated last year
• triton-inference-server / python_backend
  Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.
  ☆594Updated this week
• ray-project / llmperf
  LLMPerf is a library for validating and benchmarking LLMs
  ☆826Updated 3 months ago
• triton-inference-server / tensorrtllm_backend
  The Triton TensorRT-LLM Backend
  ☆806Updated last week
• qdrant / quaterion
  Blazing fast framework for fine-tuning similarity learning models
  ☆656Updated 2 months ago
• huggingface / text-embeddings-inference
  A blazing fast inference solution for text embeddings models
  ☆3,321Updated this week
• triton-inference-server / vllm_backend
  ☆237Updated last week
• pytorch / torchx
  TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and sup…
  ☆354Updated this week
• hpcaitech / EnergonAI
  Large-scale model inference.
  ☆628Updated last year
• microsoft / Llama-2-Onnx
  ☆1,025Updated last year
• hpcaitech / SwiftInfer
  Efficient AI Inference & Serving
  ☆468Updated last year
• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆562Updated 5 months ago
• zilliztech / feder
  Visualize hnsw, faiss and other anns index
  ☆425Updated 2 years ago
• criteo / autofaiss
  Automatically create Faiss knn indices with the most optimal similarity search parameters.
  ☆843Updated 10 months ago
• vectorch-ai / ScaleLLM
  A high-performance inference system for large language models, designed for production environments.
  ☆423Updated this week
• qdrant / fastembed
  Fast, Accurate, Lightweight Python library to make State of the Art Embedding
  ☆1,882Updated this week
• flexflow / flexflow-train
  Automatically Discovering Fast Parallelization Strategies for Distributed Deep Neural Network Training
  ☆1,773Updated this week
• skeskinen / bert.cpp
  ggml implementation of BERT
  ☆486Updated last year