Alternatives and similar repositories for LLM-Inference-Deployment-Tutorial

Users that are interested in LLM-Inference-Deployment-Tutorial are comparing it to the libraries listed below

• tanyuqian / cappy
  NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
  ☆44Updated last year
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆46Updated 7 months ago
• LowinLi / transformers-stream-generator
  This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/…
  ☆97Updated last year
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 10 months ago
• withmartian / leaderboard-backend
  Open sourced backend for Martian's LLM Inference Provider Leaderboard
  ☆19Updated last year
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆76Updated last year
• liutiedong / goat
  a Fine-tuned LLaMA that is Good at Arithmetic Tasks
  ☆178Updated 2 years ago
• nyunAI / Faster-LLM-Survey
  ☆42Updated last year
• zsc / llama_infer
  Inference script for Meta's LLaMA models using Hugging Face wrapper
  ☆110Updated 2 years ago
• anyscale / llm-continuous-batching-benchmarks
  ☆122Updated last year
• myscale / Retrieval-QA-Benchmark
  Benchmark baseline for retrieval qa applications
  ☆118Updated last year
• salesforce / summary-of-a-haystack
  Codebase accompanying the Summary of a Haystack paper.
  ☆79Updated last year
• Anni-Zou / Meta-CoT
  Meta-CoT: Generalizable Chain-of-Thought Prompting in Mixed-task Scenarios with Large Language Models
  ☆99Updated 2 years ago
• daniel-furman / sft-demos
  Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
  ☆78Updated last year
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• rayleizhu / vllm-ra
  [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts
  ☆40Updated last year
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆107Updated last year
• anyscale / long-context-fine-tuning-blogpost
  ☆17Updated last year
• EasyShopAI / rag-lab
  Lighter, cheaper and faster RAG toolkit (Graph RAG) supported by TargetPilot
  ☆46Updated 5 months ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆93Updated last year
• HITsz-TMG / KaLM-Embedding
  Code for KaLM-Embedding models
  ☆99Updated 4 months ago
• CLUEbenchmark / SuperCLUE-Fin
  中文金融大模型测评基准，六大类二十五任务、等级化评价，国内模型获得A级
  ☆10Updated last year
• Bui1dMySea / MemLong
  ☆95Updated 11 months ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆57Updated this week
• UbiquitousLearning / SLM_Survey
  ☆100Updated last year
• SkyworkAI / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆16Updated last year
• WENGSYX / LMTuner
  LMTuner: Make the LLM Better for Everyone
  ☆37Updated 2 years ago
• vtuber-plan / langport
  Langport is a language model inference service
  ☆95Updated last year
• bentoml / BentoLMDeploy
  Self-host LLMs with LMDeploy and BentoML
  ☆21Updated 4 months ago