High-performance, light-weight C++ LLM and VLM Inference Software for Physical AI
☆607Oct 31, 2026Updated this week
Alternatives and similar repositories for TensorRT-Edge-LLM
Users that are interested in TensorRT-Edge-LLM are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative…☆5,488Updated this week
- FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is prod…☆781Updated this week
- TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizat…☆14,985Updated this week
- A project demonstrating Lidar related AI solutions, including three GPU accelerated Lidar/camera DL networks (PointPillars, CenterPoint, …☆1,904Jul 28, 2026Updated 3 months ago
- 边缘端大模型高效推理引擎☆31Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- GitHub repo for Jetson AI Lab☆214Updated this week
- High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)☆31Jan 22, 2026Updated 9 months ago
- NVIDIA Alpamayo 1.5 Nano is an open 10B reasoning VLA model for autonomous vehicles with reinforcement-learning enhanced reasoning, navig…☆372Sep 9, 2026Updated last month
- ☆15Apr 2, 2026Updated 6 months ago
- From PyTorch model to end-to-end TensorRT inference experience in two commands—AI-native, cross-platform, and built for the best possible…☆285Updated this week
- Efficient and unified implementations for TopK-based sparse attention☆41Jul 10, 2026Updated 3 months ago
- Kernel Design Agents (KDA) is a agent-centric workflow to write high-performance CUDA Kernels.☆1,509Oct 19, 2026Updated 2 weeks ago
- NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source compone…☆13,425Oct 9, 2026Updated 3 weeks ago
- Skills for writing tilelang and debugging with CUDA toolkits.☆149Oct 23, 2026Updated last week
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- An Optimizer for Nvidia Compilers.☆146Oct 21, 2026Updated last week
- NVIDIA Alpamayo 1 Nano is an open 10B reasoning VLA model for autonomous vehicles that pairs driving trajectories with Chain-of-Causation…☆2,058Sep 9, 2026Updated last month
- A CUDA kernel optimization toolkit for validation, benchmarking, Nsight Compute profiling, bottleneck analysis, and iterative tuning. It …☆218Sep 5, 2026Updated last month
- FlashInfer: Kernel Library for LLM Serving☆6,828Updated this week
- Deep Learning tools and applications for NVIDIA AGX platforms.☆301Mar 13, 2026Updated 7 months ago
- BEVFormer inference on TensorRT, including INT8 Quantization and Custom TensorRT Plugins (float/half/half2/int8).☆582Nov 20, 2023Updated 2 years ago
- High Performance LLM Inference Operator Library☆1,188Sep 7, 2026Updated last month
- NVIDIA DLA-SW, the recipes and tools for running deep learning workloads on NVIDIA DLA cores for inference applications.☆239Updated this week
- how to optimize some algorithm in cuda.☆3,339Oct 11, 2026Updated 3 weeks ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Qwen3.5-Thor — High-performance BF16/NVFP4 inference engine for Qwen3.5 model family on NVIDIA Jetson AGX Thor (SM110a Blackwell). C++17/…☆13Apr 2, 2026Updated 6 months ago
- State-of-the-art LLM compression, built for production inference with vLLM☆3,931Updated this week
- ☆44Updated this week
- [CoRL'26 Spotlight] Inference engine of "EagleVLA: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference".…☆58Oct 23, 2026Updated last week
- Tile-Based Runtime for Ultra-Low-Latency LLM Inference☆1,881Sep 24, 2026Updated last month
- Machine Learning Containers for NVIDIA Jetson and JetPack-L4T☆4,895Oct 26, 2026Updated last week
- let coding agents use ncu skills analysis cuda program automatically!☆130May 25, 2026Updated 5 months ago
- CUDA Templates and Python DSLs for High-Performance Linear Algebra☆10,684Updated this week
- A simple tool that can generate TensorRT plugin code quickly.☆243Jul 11, 2023Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Some funny cute/cuteDSL code snippets☆35Mar 2, 2026Updated 8 months ago
- Scripts and fixes for running NVIDIA NemoClaw on Jetson AGX Thor☆16Sep 16, 2026Updated last month
- cuTile is a programming model for writing parallel kernels for NVIDIA GPUs☆2,165Updated this week
- ☆16Feb 3, 2026Updated 8 months ago
- A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on H…☆3,597Updated this week
- PhyAI is a high-performance framework for running Physical AI models (VLA, WAM, and beyond), supporting both cloud-based serving and on-d…☆141Updated this week
- Deep insight tensorrt, including but not limited to qat, ptq, plugin, triton_inference, cuda☆24Sep 10, 2026Updated last month