Alternatives and similar repositories for dash-infer:

Users that are interested in dash-infer are comparing it to the libraries listed below

• OpenPPL / ppl.llm.serving
  ☆127Updated 2 months ago
• pcg-mlp / KsanaLLM
  ☆323Updated 2 months ago
• alibaba / rtp-llm
  RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
  ☆667Updated 2 months ago
• ninehills / llm-inference-benchmark
  LLM Inference benchmark
  ☆404Updated 8 months ago
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆93Updated last year
• DeepLink-org / dlinfer
  ☆44Updated this week
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆238Updated last year
• FlagOpen / FlagScale
  FlagScale is a large model toolkit based on open-sourced projects.
  ☆250Updated this week
• OpenPPL / ppl.nn.llm
  ☆139Updated 11 months ago
• InternLM / InternEvo
  InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencie…
  ☆368Updated this week
• madsys-dev / deepseekv2-profile
  ☆125Updated 2 weeks ago
• luchangli03 / export_llama_to_onnx
  export llama to onnx
  ☆115Updated 2 months ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆185Updated 6 months ago
• hpcaitech / SwiftInfer
  Efficient AI Inference & Serving
  ☆468Updated last year
• ModelTC / llmc
  [EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a V…
  ☆434Updated last week
• void-main / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆59Updated last year
• wangzhaode / llm-export
  llm-export can export llm model to onnx.
  ☆271Updated 2 months ago
• mindspore-lab / mindformers
  ☆157Updated this week
• alibaba / ChatLearn
  A flexible and efficient training framework for large-scale alignment tasks
  ☆333Updated last month
• QwenLM / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆132Updated 3 months ago
• pandada8 / llm-inference-benchmark
  LLM 推理服务性能测试
  ☆37Updated last year
• Ascend / AscendSpeed
  ☆78Updated last year
• THUDM / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆39Updated 2 years ago
• bytedance / ByteTransformer
  optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052
  ☆471Updated last year
• bytedance / decoupleQ
  A quantization algorithm for LLM
  ☆136Updated 9 months ago
• neuralmagic / AutoFP8
  ☆180Updated 5 months ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆447Updated last month
• HandH1998 / QQQ
  QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.
  ☆108Updated last week
• intel / xFasterTransformer
  ☆411Updated this week