Alternatives and similar repositories for text-generation-inference

Users that are interested in text-generation-inference are comparing it to the libraries listed below

• foundation-model-stack / fms-hf-tuning
  🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.
  ☆44Updated this week
• run-ai / runai-model-streamer
  ☆214Updated this week
• open-lm-engine / lm-engine
  LM engine is a library for pretraining/finetuning LLMs
  ☆55Updated this week
• huggingface / inference-benchmarker
  Inference server benchmarking tool
  ☆67Updated last month
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆75Updated 2 weeks ago
• instructlab / sdg
  Python library for Synthetic Data Generation
  ☆42Updated this week
• instructlab / training
  InstructLab Training Library - Efficient Fine-Tuning with Message-Format Data
  ☆42Updated this week
• substratusai / vllm-docker
  ☆60Updated 2 months ago
• arcee-ai / DAM
  ☆49Updated 6 months ago
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆30Updated this week
• run-ai / llmperf
  ☆53Updated 8 months ago
• snowflakedb / ArcticTraining
  ArcticTraining is a framework designed to simplify and accelerate the post-training process for large language models (LLMs)
  ☆98Updated this week
• cognitivecomputations / kraken
  ☆66Updated last year
• snowflakedb / ArcticInference
  ☆99Updated this week
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆112Updated 4 months ago
• abacusai / gh200-llm
  Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning
  ☆41Updated 3 months ago
• LLM360 / crystalcoder-train
  Pre-training code for CrystalCoder 7B LLM
  ☆54Updated last year
• EmbeddedLLM / vllm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆86Updated this week
• facebookresearch / matrix
  Matrix (Multi-Agent daTa geneRation Infra and eXperimentation framework) is a versatile engine for multi-agent conversational data genera…
  ☆60Updated this week
• guidance-ai / jsonschemabench
  ☆38Updated last month
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆127Updated last month
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆137Updated 10 months ago
• cognitivecomputations / spectrum
  ☆121Updated last month
• bentoml / llm-bench
  ☆52Updated 6 months ago
• runpod-workers / worker-sglang
  SGLang is fast serving framework for large language models and vision language models.
  ☆23Updated 3 months ago
• nyunAI / PruneGPT
  ☆53Updated last year
• cray-lm / cray-lm
  Cray-LM unified training and inference stack.
  ☆22Updated 4 months ago
• coreweave / tensorizer
  Module, Model, and Tensor Serialization/Deserialization
  ☆232Updated last week
• huggingface / tgi-gaudi
  Large Language Model Text Generation Inference on Habana Gaudi
  ☆33Updated 2 months ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆90Updated last year