Alternatives and similar repositories for LLM-Inference-Deployment-Tutorial

Users that are interested in LLM-Inference-Deployment-Tutorial are comparing it to the libraries listed below

• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• LowinLi / transformers-stream-generator
  This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/…
  ☆97Updated last year
• tanyuqian / cappy
  NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
  ☆43Updated last year
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 7 months ago
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆40Updated 10 months ago
• zsc / llama_infer
  Inference script for Meta's LLaMA models using Hugging Face wrapper
  ☆110Updated 2 years ago
• liutiedong / goat
  a Fine-tuned LLaMA that is Good at Arithmetic Tasks
  ☆177Updated 2 years ago
• UbiquitousLearning / SLM_Survey
  ☆97Updated 11 months ago
• Aegis1863 / LLMs-Distillation-Quantification
  Repo of ACL 2025 Paper "Quantification of Large Language Model Distillation"
  ☆92Updated last month
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆33Updated 5 months ago
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆107Updated 11 months ago
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆56Updated 2 years ago
• Anni-Zou / Meta-CoT
  Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models
  ☆97Updated last year
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆77Updated last year
• vllm-project / vllm-nccl
  Manages vllm-nccl dependency
  ☆17Updated last year
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• cxcscmu / RAGViz
  Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]
  ☆85Updated 8 months ago
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆40Updated last year
• codefuse-ai / D2LLM
  ☆32Updated last year
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆64Updated last year
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆42Updated 4 months ago
• zhaochenyang20 / Prompt2Model-Self-Guide
  SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper
  ☆33Updated last year
• ctlllll / understanding_llm_benchmarks
  Understanding the correlation between different LLM benchmarks
  ☆29Updated last year
• anyscale / long-context-fine-tuning-blogpost
  ☆17Updated last year
• cuplv / text-to-sql-wizardcoder
  Leveraging large language models for text-to-SQL synthesis, this project fine-tunes WizardLM/WizardCoder-15B-V1.0 with QLoRA on a custom …
  ☆44Updated last year
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆85Updated 2 years ago
• terryyz / llm-benchmark
  A list of LLM benchmark frameworks.
  ☆70Updated last year
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆93Updated last year
• camille-vanhoffelen / langchain-huggingGPT
  Langchain implementation of HuggingGPT
  ☆133Updated 2 years ago
• daniel-furman / sft-demos
  Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
  ☆78Updated 11 months ago