Alternatives and similar repositories for BentoLMDeploy

Users that are interested in BentoLMDeploy are comparing it to the libraries listed below

• wdlctc / headinfer
  ☆55Updated 3 months ago
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆118Updated last year
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆47Updated 4 months ago
• hao-ai-lab / Dynasor
  Simple extension on vLLM to help you speed up reasoning model without training.
  ☆175Updated 2 months ago
• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆45Updated last year
• hetailang / SqueezeAttention
  ☆38Updated 10 months ago
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆36Updated last month
• StigLidu / DualDistill
  The official implementation for paper "Agentic-R1: Distilled Dual-Strategy Reasoning"
  ☆92Updated last month
• nyunAI / PruneGPT
  ☆51Updated last year
• UbiquitousLearning / SLM_Survey
  ☆95Updated 10 months ago
• 18907305772 / FuseAI
  FuseAI Project
  ☆87Updated 6 months ago
• Zoeyyao27 / SirLLM
  This repository contains the code for the paper: SirLLM: Streaming Infinite Retentive LLM
  ☆59Updated last year
• Infini-AI-Lab / MagicDec
  [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆124Updated 8 months ago
• lfsszd / CS-Drafting
  Cascade Speculative Drafting
  ☆29Updated last year
• wdlctc / mini-s
  ☆51Updated 9 months ago
• golololologol / LLM-Distillery
  A pipeline for LLM knowledge distillation
  ☆108Updated 4 months ago
• Digitous / LLM-SLERP-Merge
  Spherical Merge Pytorch/HF format Language Models with minimal feature loss.
  ☆135Updated last year
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆91Updated last year
• chu-tianxiang / QuIP-for-all
  QuIP quantization
  ☆57Updated last year
• SalesforceAIResearch / GemFilter
  ☆85Updated 7 months ago
• imagination-research / sot
  [ICLR 2024] Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
  ☆174Updated last year
• InternLM / SWE-Fixer
  ☆110Updated 3 months ago
• IsaacRe / vllm-kvcompress
  KV cache compression for high-throughput LLM inference
  ☆134Updated 6 months ago
• siyan-zhao / prepacking
  The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …
  ☆61Updated 10 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆136Updated last year
• Bui1dMySea / MemLong
  ☆94Updated 8 months ago
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆41Updated 3 months ago
• HanGuo97 / lq-lora
  ☆127Updated last year
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)
  ☆160Updated 4 months ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆26Updated 2 years ago