Alternatives and similar repositories for EnergonAI

Users that are interested in EnergonAI are comparing it to the libraries listed below

• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆565Updated last year
• triton-inference-server / fastertransformer_backend
  ☆413Updated 2 years ago
• hpcaitech / ColossalAI-Examples
  Examples of training models with hybrid parallelism using ColossalAI
  ☆339Updated 2 years ago
• Oneflow-Inc / libai
  LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
  ☆406Updated 4 months ago
• OpenBMB / BMTrain
  Efficient Training (including pre-training and fine-tuning) for Big Models
  ☆614Updated last month
• Vahe1994 / SpQR
  ☆551Updated last year
• volcengine / veGiantModel
  ☆219Updated 2 years ago
• Azure / MS-AMP
  Microsoft Automatic Mixed Precision Library
  ☆630Updated 3 weeks ago
• hpcaitech / SwiftInfer
  Efficient AI Inference & Serving
  ☆478Updated last year
• bytedance / ByteTransformer
  optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052
  ☆476Updated last year
• bigscience-workshop / Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆1,426Updated last year
• Tencent / PatrickStar
  PatrickStar enables Larger, Faster, Greener Pretrained Models for NLP and democratizes AI for everyone.
  ☆773Updated last month
• bytedance / effective_transformer
  Running BERT without Padding
  ☆476Updated 3 years ago
• fpgaminer / GPTQ-triton
  GPTQ inference Triton kernel
  ☆317Updated 2 years ago
• alibaba / Megatron-LLaMA
  Best practice for training LLaMA models in Megatron-LM
  ☆664Updated last year
• DachengLi1 / LongChat
  Official repository for LongChat and LongEval
  ☆533Updated last year
• OpenBMB / BMInf
  Efficient Inference for Big Models
  ☆585Updated 2 years ago
• deepspeedai / DeepSpeed-MII
  MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
  ☆2,083Updated 5 months ago
• vectorch-ai / ScaleLLM
  A high-performance inference system for large language models, designed for production environments.
  ☆489Updated this week
• hpcaitech / PaLM-colossalai
  Scalable PaLM implementation of PyTorch
  ☆189Updated 3 years ago
• tpoisonooo / llama.onnx
  LLaMa/RWKV onnx models, quantization and testcase
  ☆367Updated 2 years ago
• OpenPPL / ppl.llm.serving
  ☆130Updated 11 months ago
• hao-ai-lab / LookaheadDecoding
  [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
  ☆1,307Updated 9 months ago
• THUDM / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆39Updated 2 years ago
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆250Updated last year
• alipay / PainlessInferenceAcceleration
  Accelerate inference without tears
  ☆370Updated last month
• bigscience-workshop / bigscience
  Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
  ☆1,006Updated last year
• BlackSamorez / tensor_parallel
  Automatically split your PyTorch models on multiple GPUs for training & inference
  ☆657Updated last year
• open-compass / MixtralKit
  A toolkit for inference and evaluation of 'mixtral-8x7b-32kseqlen' from Mistral AI
  ☆773Updated 2 years ago
• IST-DASLab / marlin
  FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
  ☆961Updated last year