Alternatives and similar repositories for LLM-Inference-Deployment-Tutorial

Users that are interested in LLM-Inference-Deployment-Tutorial are comparing it to the libraries listed below

• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆40Updated last year
• withmartian / leaderboard-backend
  Open sourced backend for Martian's LLM Inference Provider Leaderboard
  ☆19Updated 11 months ago
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 6 months ago
• tanyuqian / cappy
  NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
  ☆43Updated last year
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆40Updated 9 months ago
• OpenLLMAI / OpenLLMDE
  OpenLLMDE: An open source data engineering framework for LLMs
  ☆17Updated last year
• LowinLi / transformers-stream-generator
  This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/…
  ☆97Updated last year
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆40Updated 3 months ago
• vllm-project / vllm-nccl
  Manages vllm-nccl dependency
  ☆17Updated last year
• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆45Updated last year
• Lightning-Universe / lightning-ColossalAI
  Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI
  ☆56Updated last year
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆76Updated last year
• liutiedong / goat
  a Fine-tuned LLaMA that is Good at Arithmetic Tasks
  ☆178Updated last year
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆105Updated 10 months ago
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• Bui1dMySea / MemLong
  ☆94Updated 8 months ago
• ctlllll / understanding_llm_benchmarks
  Understanding the correlation between different LLM benchmarks
  ☆29Updated last year
• zsc / llama_infer
  Inference script for Meta's LLaMA models using Hugging Face wrapper
  ☆110Updated 2 years ago
• WENGSYX / LMTuner
  LMTuner: Make the LLM Better for Everyone
  ☆36Updated last year
• bigcode-project / astraios
  Astraios: Parameter-Efficient Instruction Tuning Code Language Models
  ☆59Updated last year
• locuslab / scaling_laws_data_filtering
  ☆65Updated last year
• explodinggradients / nemesis
  Reward Model framework for LLM RLHF
  ☆61Updated 2 years ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆91Updated last year
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated last year
• laramohan / wikillm
  LLMs as Collaboratively Edited Knowledge Bases
  ☆45Updated last year
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆56Updated 2 weeks ago
• Anni-Zou / Meta-CoT
  Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models
  ☆97Updated last year
• GasolSun36 / Iter-CoT
  [NAACL 2024] Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models
  ☆85Updated last year
• microsoft / prose-benchmarks
  PROSE Public Benchmark Suite
  ☆27Updated 2 weeks ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆123Updated 6 months ago