Alternatives and similar repositories for LLM-Inference-Deployment-Tutorial

Users that are interested in LLM-Inference-Deployment-Tutorial are comparing it to the libraries listed below

• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• myscale / Retrieval-QA-Benchmark
  Benchmark baseline for retrieval qa applications
  ☆118Updated last year
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆76Updated last year
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 10 months ago
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆108Updated last year
• liutiedong / goat
  a Fine-tuned LLaMA that is Good at Arithmetic Tasks
  ☆178Updated 2 years ago
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆46Updated 7 months ago
• BaichuanSEED / BaichuanSEED.github.io
  Official Repository for Paper "BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Compet…
  ☆18Updated last year
• microsoft / prose-benchmarks
  PROSE Public Benchmark Suite
  ☆28Updated 3 months ago
• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆45Updated last year
• zsc / llama_infer
  Inference script for Meta's LLaMA models using Hugging Face wrapper
  ☆110Updated 2 years ago
• WENGSYX / LMTuner
  LMTuner: Make the LLM Better for Everyone
  ☆37Updated 2 years ago
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated 2 years ago
• Aegis1863 / LLMs-Distillation-Quantification
  Repo of ACL 2025 Paper "Quantification of Large Language Model Distillation"
  ☆94Updated 4 months ago
• tanyuqian / cappy
  NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
  ☆45Updated last year
• gomate-community / rageval
  Evaluation tools for Retrieval-augmented Generation (RAG) methods.
  ☆167Updated last year
• codefuse-ai / D2LLM
  ☆35Updated last year
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆40Updated last year
• jackfsuia / nanoRLHF
  RLHF experiments on a single A100 40G GPU. Support PPO, GRPO, REINFORCE, RAFT, RLOO, ReMax, DeepSeek R1-Zero reproducing.
  ☆76Updated 10 months ago
• Bui1dMySea / MemLong
  ☆95Updated last year
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆56Updated 2 years ago
• liuqi6777 / pe_rank
  Leveraging passage embeddings for efficient listwise reranking with large language models.
  ☆49Updated last year
• camille-vanhoffelen / langchain-huggingGPT
  Langchain implementation of HuggingGPT
  ☆134Updated 2 years ago
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆70Updated 2 years ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆126Updated 11 months ago
• vllm-project / vllm-nccl
  Manages vllm-nccl dependency
  ☆17Updated last year
• LowinLi / transformers-stream-generator
  This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/…
  ☆97Updated last year
• SeunghyunSEO / optimized_hf_llama_class_for_training
  ☆48Updated last year
• GeneZC / MiniMA
  Code for paper titled "Towards the Law of Capacity Gap in Distilling Language Models"
  ☆102Updated last year
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆137Updated last year