Alternatives and similar repositories for llm-inference-simulator

Users that are interested in llm-inference-simulator are comparing it to the libraries listed below

• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆33Updated this week
• UmerHA / triton_util
  Make triton easier
  ☆47Updated last year
• huggingface / kernel-builder
  👷 Build compute kernels
  ☆77Updated this week
• zenrran4nlp / Awesome-LLM-Inference-Serving
  ☆36Updated 2 months ago
• katsumiok / pyaskit
  AskIt: Unified programming interface for programming with LLMs (GPT-3.5, GPT-4, Gemini, Claude, Cohere, Llama 2)
  ☆79Updated 6 months ago
• isEmmanuelOlowe / llm-cost-estimator
  Estimating hardware and cloud costs of LLMs and transformer projects
  ☆18Updated 3 weeks ago
• eth-easl / mixtera
  A lightweight, user-friendly data-plane for LLM training.
  ☆20Updated 2 weeks ago
• IST-DASLab / QIGen
  Repository for CPU Kernel Generation for LLM Inference
  ☆26Updated 2 years ago
• yuzhenmao / IceFormer
  Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).
  ☆25Updated this week
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆42Updated last year
• InfiniTensor / ninetoothed
  A domain-specific language (DSL) based on Triton but providing higher-level abstractions.
  ☆24Updated this week
• Michaelvll / llm-ie-benchmarks
  A collection of reproducible inference engine benchmarks
  ☆32Updated 2 months ago
• GreenBitAI / low_bit_llama
  Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs
  ☆110Updated last year
• tile-ai / tvm
  Open deep learning compiler stack for cpu, gpu and specialized accelerators
  ☆19Updated this week
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆80Updated 10 months ago
• michaelfeil / candle-flash-attn-v3
  ☆11Updated 5 months ago
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated 2 weeks ago
• gangiswag / cornstack
  ☆35Updated 3 weeks ago
• argilla-io / distilabel-spin-dibt
  Repository containing the SPIN experiments on the DIBT 10k ranked prompts
  ☆24Updated last year
• deepspeedai / DeepSpeed-Kernels
  ☆74Updated 3 months ago
• sgl-project / tensorrt-demo
  TensorRT LLM Benchmark Configuration
  ☆13Updated 11 months ago
• facebookresearch / Ternary_Binary_Transformer
  ACL 2023
  ☆39Updated 2 years ago
• facebookresearch / lss_eval
  This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…
  ☆31Updated last year
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆66Updated 7 months ago
• FrancescoSaverioZuppichini / pytorch-2.0-benchmark
  Benchmarking PyTorch 2.0 different models
  ☆21Updated 2 years ago
• GATECH-EIC / ShiftAddLLM
  ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
  ☆109Updated 9 months ago
• ROCm / pyrsmi
  python package of rocm-smi-lib
  ☆21Updated 9 months ago
• chu-tianxiang / QuIP-for-all
  QuIP quantization
  ☆54Updated last year
• eth-easl / deltazip
  Compression for Foundation Models
  ☆33Updated 3 months ago
• IlyasMoutawwakil / llm-perf-backend
  The backend behind the LLM-Perf Leaderboard
  ☆10Updated last year