LeeJuly30 / BERTCpp
implement bert in pure c++
☆36Updated 4 years ago
Alternatives and similar repositories for BERTCpp:
Users that are interested in BERTCpp are comparing it to the libraries listed below
- BERT Tokenizer in C++☆75Updated 4 years ago
- Minimal example of using a traced huggingface transformers model with libtorch☆35Updated 4 years ago
- Möbius Transformation for Fast Inner Product Search on Graph☆23Updated 3 years ago
- Transformer related optimization, including BERT, GPT☆17Updated last year
- transformer tokenizers (e.g. BERT tokenizer) in C++ (WIP)☆17Updated 2 years ago
- lightweighted deep learning inference service framework☆39Updated 3 years ago
- ☆19Updated 10 months ago
- InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.☆66Updated 3 years ago
- Transformer related optimization, including BERT, GPT☆59Updated last year
- ☆23Updated last year
- Simple Dynamic Batching Inference☆145Updated 3 years ago
- ☆52Updated 3 years ago
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆46Updated last year
- A Fast Muti-processing BERT-Inference System☆101Updated 2 years ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆69Updated last year
- Distributed DataLoader For Pytorch Based On Ray☆24Updated 3 years ago
- PyTorch -> ONNX -> TVM for autotuning☆23Updated 5 years ago
- 高性能文本 Tokenizer 库☆28Updated last year
- symmetric int8 gemm☆66Updated 4 years ago
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 3 years ago
- pytorch during training, libtorch during serving via gRPC☆21Updated 5 years ago
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago
- simplify >2GB large onnx model☆54Updated 3 months ago
- ChineseWord correct!!when you input some error words,return some maybe right word☆8Updated 10 years ago
- My solutions to the assignments of dlsys course (CSE599G1: Deep Learning System Spring 2017)☆10Updated 7 years ago
- Using TensorRT and Triton Server to build BERT model as a service☆12Updated 3 years ago
- Whisper in TensorRT-LLM☆15Updated last year
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆35Updated 2 weeks ago
- ☆74Updated 2 years ago
- Bert TensorRT模型加速部署☆9Updated 2 years ago