Alternatives and similar repositories for TensorRT-Edge-LLM

Users that are interested in TensorRT-Edge-LLM are comparing it to the libraries listed below

• leimao / TensorRT-Custom-Plugin-Example
  Quick and Self-Contained TensorRT Custom Plugin Implementation and Integration
  ☆79Updated 8 months ago
• NVIDIA / Deep-Learning-Accelerator-SW
  NVIDIA DLA-SW, the recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.
  ☆225Updated last year
• NVIDIA-AI-IOT / tensorrt_plugin_generator
  A simple tool that can generate TensorRT plugin code quickly.
  ☆239Updated 2 years ago
• NVIDIA-AI-IOT / cuDLA-samples
  YOLOv5 on Orin DLA
  ☆221Updated last year
• NVIDIA-AI-IOT / jetson_dla_tutorial
  A tutorial for getting started with the Deep Learning Accelerator (DLA) on NVIDIA Jetson
  ☆364Updated 3 years ago
• HeKun-NVIDIA / AI-Blog
  Collection of blogs on AI development
  ☆23Updated last year
• CalvinXKY / BasicCUDA
  A tutorial for CUDA&PyTorch
  ☆227Updated last week
• HeKun-NVIDIA / TensorRT-Developer_Guide_in_Chinese
  ☆314Updated 3 years ago
• Tencent / TPAT
  TensorRT Plugin Autogen Tool
  ☆366Updated 2 years ago
• lix19937 / tensorrt-insight
  Deep insight tensorrt, including but not limited to qat, ptq, plugin, triton_inference, cuda
  ☆23Updated 3 weeks ago
• agrechnev / trt-cpp-min
  TensorRT 7 C++ (almost) minimal examples
  ☆84Updated 2 years ago
• kalfazed / multi-thread-programming
  This is a repository to practice multi-thread programming in C++
  ☆27Updated last year
• OpenPPL / ppl.kernel.cuda
  ☆38Updated last year
• ModelTC / Dipoorlet
  Offline Quantization Tools for Deploy.
  ☆142Updated 2 years ago
• shouxieai / tensorRT_quantization
  该代码与B站上的视频 https://www.bilibili.com/video/BV18L41197Uz/?spm_id_from=333.788&vd_source=eefa4b6e337f16d87d87c2c357db8ca7 相关联。
  ☆71Updated 2 years ago
• OpenPPL / ppl.pmx
  ☆60Updated last year
• Tencent / hpc-ops
  High Performance LLM Inference Operator Library
  ☆695Updated this week
• hopef / llama3_chat
  Llama3 Streaming Chat Sample
  ☆22Updated last year
• jinmin527 / learning-cuda-trt
  A large number of cuda/tensorrt cases . 大量案例来学习cuda/tensorrt
  ☆170Updated 3 years ago
• DerryHub / BEVFormer_tensorrt
  BEVFormer inference on TensorRT, including INT8 Quantization and Custom TensorRT Plugins (float/half/half2/int8).
  ☆554Updated 2 years ago
• BestAnHongjun / LMDeploy-Jetson
  Deploying LLMs offline on the NVIDIA Jetson platform marks the dawn of a new era in embodied intelligence, where devices can function ind…
  ☆108Updated last year
• ThanatosShinji / onnx-tool
  A parser, editor and profiler tool for ONNX models.
  ☆478Updated 3 months ago
• torchpipe / torchpipe
  Serving Inside Pytorch
  ☆170Updated 2 weeks ago
• RussWong / LLM-engineering
  ☆26Updated 5 months ago
• OpenPPL / ppl.llm.kernel.cuda
  ☆152Updated last year
• AyakaGEMM / Hands-on-GEMM
  ☆145Updated last year
• harleyszhang / lite_llama
  A light llama-like llm inference framework based on the triton kernel.
  ☆171Updated last month
• NVIDIA / DL4AGX
  Deep Learning tools and applications for NVIDIA AGX platforms.
  ☆266Updated last week
• zhangcheng828 / TensorRT-Plugin
  ☆45Updated 3 years ago
• doorteeth / learn_cuda
  ☆43Updated 4 years ago