Alternatives and similar repositories for ci-infra

Users that are interested in ci-infra are comparing it to the libraries listed below

• NVIDIA-NeMo / Automodel
  Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
  ☆214Updated this week
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆102Updated this week
• snowflakedb / ArcticInference
  ArcticInference: vLLM plugin for high-throughput, low-latency inference
  ☆354Updated this week
• ailzhang / minPP
  Pipeline parallelism for the minimalist
  ☆37Updated 4 months ago
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆46Updated this week
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆135Updated last year
• intel / llm-on-ray
  Pretrain, finetune and serve LLMs on Intel platforms with Ray
  ☆131Updated 3 months ago
• vllm-project / speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  ☆174Updated last week
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆47Updated 8 months ago
• LMCache / lmcache-vllm
  The driver for LMCache core to run in vLLM
  ☆59Updated 10 months ago
• cornserve-ai / cornserve
  Easy, Fast, and Scalable Multimodal AI
  ☆81Updated last week
• run-ai / runai-model-streamer
  ☆273Updated last week
• vllm-project / recipes
  Common recipes to run vLLM
  ☆305Updated this week
• huggingface / tgi-gaudi
  Large Language Model Text Generation Inference on Habana Gaudi
  ☆34Updated 9 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆148Updated 10 months ago
• intel / ai-containers
  This repository contains Dockerfiles, scripts, yaml files, Helm charts, etc. used to scale out AI containers with versions of TensorFlow …
  ☆56Updated last week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆93Updated 9 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆84Updated last month
• deepseek-ai / LPLB
  An early research stage expert-parallel load balancer for MoE models based on linear programming.
  ☆476Updated last month
• sgl-project / genai-bench
  Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serv…
  ☆250Updated 2 weeks ago
• meta-pytorch / torchcomms
  torchcomms: a modern PyTorch communications API
  ☆314Updated this week
• perplexityai / pplx-garden
  Perplexity open source garden for inference technology
  ☆313Updated this week
• huggingface / kernel-builder
  👷 Build compute kernels
  ☆195Updated this week
• IBM / text-generation-inference
  IBM development fork of https://github.com/huggingface/text-generation-inference
  ☆62Updated 3 months ago
• lm-sys / lm-sys.github.io
  The source of LMSYS website and blogs
  ☆72Updated this week
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆162Updated last month
• hao-ai-lab / Dynasor
  [NeurIPS 2025] Simple extension on vLLM to help you speed up reasoning model without training.
  ☆215Updated 6 months ago
• intel / intel-extension-for-deepspeed
  Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…
  ☆63Updated 5 months ago
• AI9Stars / AutoTriton
  ☆56Updated 5 months ago
• ServiceNow / Fast-LLM
  Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research
  ☆273Updated this week