meta-pytorch/tokenizers external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

meta-pytorch/tokenizers

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/meta-pytorch/tokenizers)

Close

meta-pytorch / tokenizersView on GitHubMore

• External links
• Readme badge

C++ implementations for various tokenizers (sentencepiece, tiktoken etc).

☆48Mar 12, 2026Updated last week

Alternatives and similar repositories for tokenizers

Users that are interested in tokenizers are comparing it to the libraries listed below

• bluryar / VoxCPM-ONNX

  View on GitHub
  ☆25Mar 10, 2026Updated last week
• sewenew / tokenizer

  View on GitHub
  C++ implementation of tokenizers, including tiktoken.
  ☆25Dec 7, 2023Updated 2 years ago
• triple-mu / HunyuanDiT-TensorRT-libtorch

  View on GitHub
  HunyuanDiT with TensorRT and libtorch
  ☆17May 22, 2024Updated last year
• Huiyicc / gpt_sovits_cpp

  View on GitHub
  GPT-Sovits的c++实现版本
  ☆22Jan 9, 2026Updated 2 months ago
• ailia-ai / ailia-models-cpp

  View on GitHub
  C++ version of ailia models repository
  ☆24Dec 31, 2025Updated 2 months ago
• muzishen / Deepsort_V2

  View on GitHub
  2020中兴捧月阿尔法赛道多目标检测和跟踪初赛第一名方案
  ☆33Oct 3, 2023Updated 2 years ago
• gh-markt / cpp-tiktoken

  View on GitHub
  ☆24Aug 14, 2025Updated 7 months ago
• Phoenix8215 / learn-TensorRT-from-scratch

  View on GitHub
  learn TensorRT from scratch🥰
  ☆17Sep 29, 2024Updated last year
• KuangjuX / cu-x

  View on GitHub
  🎉My Collections of CUDA Kernels~
  ☆10Jun 25, 2024Updated last year
• morsoli / llmbenchmark

  View on GitHub
  大模型API性能指标比较 - 深入分析TTFT、TPS等关键指标
  ☆19Sep 12, 2024Updated last year
• lovemefan / ggml-learning-notes

  View on GitHub
  ggml学习笔记，ggml是一个机器学习的推理框架
  ☆17Mar 24, 2024Updated last year
• snap-research / efficient-nn-tutorial

  View on GitHub
  Page for the CVPR 2023 Tutorial - Efficient Neural Networks: From Algorithm Design to Practical Mobile Deployments
  ☆12Jun 30, 2023Updated 2 years ago
• pointpillars-on-openvino / pointpillars-on-openvino

  View on GitHub
  ☆11Dec 16, 2021Updated 4 years ago
• InfiniTensor / RefactorGraph

  View on GitHub
  分层解耦的深度学习推理引擎
  ☆78Feb 17, 2025Updated last year
• deepglint / eq-ncnn

  View on GitHub
  ☆42Jun 25, 2020Updated 5 years ago
• elm-community / elm-json-extra

  View on GitHub
  DEPRECATED. Moved to elm-community/json-extra =>
  ☆10Jun 16, 2016Updated 9 years ago
• ZHEQIUSHUI / CLIP-ONNX-AX650-CPP

  View on GitHub
  c++实现的clip推理，模型有一点点改动，但是不大，改动和导出模型的代码可以在readme里找到，模型文件都在Releases里，包括AX650的模型。新增支持ChineseCLIP
  ☆31Jun 19, 2025Updated 9 months ago
• iimmortall / QuantLib

  View on GitHub
  ☆14Feb 3, 2022Updated 4 years ago
• triple-mu / TensorRT2ONNX

  View on GitHub
  A tool convert TensorRT engine/plan to a fake onnx
  ☆40Nov 22, 2022Updated 3 years ago
• liwuhen / CVDeploy-2D

  View on GitHub
  The repository supports TensorRT, QNN platform inference, 2D obstacle detection yolo series (yolov5, yolov8, yolo11, yolox), semantic seg…
  ☆19May 6, 2025Updated 10 months ago
• pytorch-fdn / ecosystem

  View on GitHub
  ☆31Jan 9, 2026Updated 2 months ago
• flagos-ai / libtriton_jit

  View on GitHub
  A Triton JIT runtime and ffi provider in C++
  ☆32Updated this week
• InfiniTensor / gguf

  View on GitHub
  handle gguf files
  ☆13Aug 14, 2025Updated 7 months ago
• zjd1988 / seetaface2_onnx_model

  View on GitHub
  only contain face detect 、5/81 points 、face recognization models
  ☆11Jul 9, 2020Updated 5 years ago
• inisis / OnnxLLM

  View on GitHub
  Large Language Model Onnx Inference Framework
  ☆34Nov 25, 2025Updated 3 months ago
• bobbych94 / yolact-trt

  View on GitHub
  segmentation algorithm yolact use tensorrt deploy
  ☆14May 7, 2022Updated 3 years ago
• leimao / TensorRT-Custom-Plugin-Example

  View on GitHub
  Quick and Self-Contained TensorRT Custom Plugin Implementation and Integration
  ☆81May 26, 2025Updated 9 months ago
• microsoft / Analysis-Framework-for-TVM

  View on GitHub
  Static analysis framework for analyzing programs written in TVM's Relay IR.
  ☆29Oct 31, 2019Updated 6 years ago
• deeprefactoring / deeprefactoring-bot

  View on GitHub
  Community bot
  ☆12Feb 25, 2023Updated 3 years ago
• Oneflow-Inc / oneflow-lite

  View on GitHub
  ☆17Jan 1, 2024Updated 2 years ago
• DataXujing / lanenet-tensorrt

  View on GitHub
  车道线检测Lanenet TensorRT加速C++实现
  ☆22Feb 24, 2022Updated 4 years ago
• NVlabs / mixedproxy

  View on GitHub
  ☆12Nov 14, 2023Updated 2 years ago
• lxl24 / SwinTransformerV2_TensorRT

  View on GitHub
  For 2022 Nvidia Hackathon
  ☆22Jun 28, 2022Updated 3 years ago
• megvii-research / IntLLaMA

  View on GitHub
  IntLLaMA: A fast and light quantization solution for LLaMA
  ☆18Jul 21, 2023Updated 2 years ago
• Tartisan / MMDet3d-PointPillars

  View on GitHub
  PointPillars TensorRT version pretrained on MMDetection3d with WaymoOpenDataset
  ☆22Aug 11, 2022Updated 3 years ago
• promaton / cloudfront-sign-rs

  View on GitHub
  Utility module for AWS CloudFront in Rust allowing you to create signed urls and cookies
  ☆12Mar 14, 2025Updated last year
• heroinlin / SlowFastTRT

  View on GitHub
  使用TensorRT部署SlowFast模型
  ☆24Mar 2, 2022Updated 4 years ago
• neuml / py27hash

  View on GitHub
  Python 2.7 hashing and iteration in Python 3+
  ☆18Nov 20, 2022Updated 3 years ago
• ZHEQIUSHUI / SAM-ONNX-AX650-CPP

  View on GitHub
  SAM and lama inpaint，包含QT的GUI交互界面，实现了交互式可实时显示结果的画点、画框进行SAM，然后通过进行Inpaint，具体操作看readme里的视频。
  ☆52Jan 30, 2024Updated 2 years ago