Alternatives and similar repositories for trtllm-llama

Users that are interested in trtllm-llama are comparing it to the libraries listed below

• FeiGeChuanShu / trt2023

  View on GitHub
  NVIDIA TensorRT Hackathon 2023复赛选题：通义千问Qwen-7B用TensorRT-LLM模型搭建及优化
  ☆43Oct 20, 2023Updated 2 years ago
• YdrMaster / cuda-driver

  View on GitHub
  基于 CUDA Driver API 的 cuda 运行时环境
  ☆15Jul 30, 2025Updated 6 months ago
• EdVince / whisper-trtllm

  View on GitHub
  Whisper in TensorRT-LLM
  ☆17Sep 21, 2023Updated 2 years ago
• AwakerLee / CAGAN

  View on GitHub
  CLIP-based Adaptive Graph Attention Network for Large-Scale Unsupervised Multi-modal Hashing Retrieval
  ☆10Mar 18, 2024Updated last year
• cqu20160901 / DETR_onnx_tensorRT_V2

  View on GitHub
  DETR tensor去除推理过程无用辅助头+fp16部署再次加速+解决转tensorrt 输出全为0问题的新方法。
  ☆12Jan 9, 2024Updated 2 years ago
• ACA-Lab-SJTU / token-ring

  View on GitHub
  ☆13Jan 7, 2025Updated last year
• HydraQYH / hp_rms_norm

  View on GitHub
  High performance RMSNorm Implement by using SM Core Storage(Registers and Shared Memory)
  ☆26Jan 22, 2026Updated 3 weeks ago
• yvonwin / qwen2.cpp

  View on GitHub
  qwen2 and llama3 cpp implementation
  ☆49Jun 7, 2024Updated last year
• TRT2022 / ControlNet_TensorRT

  View on GitHub
  天池 NVIDIA TensorRT Hackathon 2023 —— 生成式AI模型优化赛 初赛第三名方案
  ☆50Aug 16, 2023Updated 2 years ago
• fuxuelinwudi / ccks2021_track3_baseline

  View on GitHub
  a baseline to practice
  ☆45Jul 6, 2021Updated 4 years ago
• kyutai-labs / jax-flash-attn3

  View on GitHub
  JAX bindings for the flash-attention3 kernels
  ☆20Jan 2, 2026Updated last month
• mtenenholtz / lmsys-chatbot-arena-solution

  View on GitHub
  ☆14Aug 26, 2024Updated last year
• AXERA-TECH / Qwen2.5-VL-3B-Instruct.axera

  View on GitHub
  Demo for Qwen2.5-VL-3B-Instruct on Axera device.
  ☆17Sep 3, 2025Updated 5 months ago
• coderonion / cuda-beginner-course-cpp-version

  View on GitHub
  bilibili视频【CUDA 12.x 并行编程入门(C++版)】配套代码
  ☆33Aug 12, 2024Updated last year
• ppogg / Deepstream-Box

  View on GitHub
  deepstream + cuda，yolo26，yolo-master，yolo11，yolov8，sam，transformer, etc.
  ☆35Feb 7, 2026Updated last week
• pertusa / mxnet_predict_cc

  View on GitHub
  MXNet prediction example
  ☆13Dec 14, 2015Updated 10 years ago
• plctlab / riscv-cluster

  View on GitHub
  Towards a million-node RISC-V cluster.
  ☆14Mar 6, 2025Updated 11 months ago
• Bruce-Lee-LY / cuda_hgemv

  View on GitHub
  Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.
  ☆72Sep 8, 2024Updated last year
• madsys-dev / deepseekv2-profile

  View on GitHub
  ☆155Mar 4, 2025Updated 11 months ago
• seasonjs / rwkv

  View on GitHub
  pure go for rwkv
  ☆19Dec 31, 2023Updated 2 years ago
• AI-confused / CGEC-with-Pointer-Generator-Network-Bart

  View on GitHub
  基于Bart语言模型的指针生成网络，用于中文语法纠错任务
  ☆16Sep 8, 2022Updated 3 years ago
• mostafa-shalaby84 / hair_segmentation_matting

  View on GitHub
  ☆12Jan 23, 2020Updated 6 years ago
• lin-honghui / data-competition-calendar

  View on GitHub
  国内外数据竞赛资讯整理
  ☆18Nov 6, 2021Updated 4 years ago
• Bruce-Lee-LY / cutlass_gemm

  View on GitHub
  Multiple GEMM operators are constructed with cutlass to support LLM inference.
  ☆20Aug 3, 2025Updated 6 months ago
• SongQiPing / KuiperInfer_rs

  View on GitHub
  使用 Rust 语言重新实现 https://github.com/zjhellofss/KuiperInfer 和 https://github.com/zjhellofss/kuiperdatawhale 中的深度学习推理框架。
  ☆17Apr 9, 2024Updated last year
• Bruce-Lee-LY / flash_attention_inference

  View on GitHub
  Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.
  ☆44Feb 27, 2025Updated 11 months ago
• hova88 / CUDA-MatMul-Practice

  View on GitHub
  ☆18Jan 4, 2024Updated 2 years ago
• yanqiangmiffy / Product-Entity-Recognition

  View on GitHub
  商品标题实体识别
  ☆24Apr 1, 2022Updated 3 years ago
• gzuidhof / jpeg-artifact-reduction

  View on GitHub
  ☆16Apr 27, 2017Updated 8 years ago
• LCH1238 / BEVDet

  View on GitHub
  A fork of the BEVDet series .
  ☆21Oct 8, 2023Updated 2 years ago
• NonvolatileMemory / flash_tree_attn

  View on GitHub
  ☆20Dec 24, 2024Updated last year
• fateshelled / bytetrack_ros

  View on GitHub
  ByteTrack + ROS 2
  ☆28Apr 17, 2024Updated last year
• mscheong01 / speculative_decoding.c

  View on GitHub
  minimal C implementation of speculative decoding based on llama2.c
  ☆25Jul 15, 2024Updated last year
• li199603 / sgemm_with_cuda

  View on GitHub
  SGEMM optimization with cuda step by step
  ☆21Mar 23, 2024Updated last year
• iwatake2222 / InferenceHelper_Sample

  View on GitHub
  Sample projects for InferenceHelper, a Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite, TensorRT, OpenCV, ncnn, MNN,…
  ☆22Mar 27, 2022Updated 3 years ago
• yuxiaoranyu / stable_diffusion_trt_triton

  View on GitHub
  ☆20Dec 29, 2023Updated 2 years ago
• wangzhaode / mnn-asr

  View on GitHub
  mnn asr demo.
  ☆25Mar 24, 2025Updated 10 months ago
• JieRen98 / SGEMM-SASS-Annotation

  View on GitHub
  ☆21Mar 22, 2021Updated 4 years ago
• cvdong / YOLO_TRT_SIM

  View on GitHub
  高效部署：YOLO X, V3, V4, V5, V6, V7, V8, EdgeYOLO TRT推理 ™️ ,前后处理均由CUDA核函数实现 CPP/CUDA🚀
  ☆53Feb 23, 2023Updated 2 years ago