LeeJuly30 / BERTCpp
implement bert in pure c++
☆36Updated 4 years ago
Alternatives and similar repositories for BERTCpp:
Users that are interested in BERTCpp are comparing it to the libraries listed below
- BERT Tokenizer in C++☆75Updated 4 years ago
- Minimal example of using a traced huggingface transformers model with libtorch☆35Updated 4 years ago
- Möbius Transformation for Fast Inner Product Search on Graph☆23Updated 3 years ago
- Transformer related optimization, including BERT, GPT☆17Updated last year
- ☆19Updated 10 months ago
- InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.☆66Updated 3 years ago
- lightweighted deep learning inference service framework☆39Updated 3 years ago
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 3 years ago
- Transformer related optimization, including BERT, GPT☆59Updated last year
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆46Updated last year
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆69Updated last year
- Simple Dynamic Batching Inference☆145Updated 3 years ago
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆35Updated 3 weeks ago
- 高性能文本 Tokenizer 库☆28Updated last year
- ☆23Updated last year
- OneFlow->ONNX☆42Updated last year
- A Fast Muti-processing BERT-Inference System☆101Updated 2 years ago
- PyTorch -> ONNX -> TVM for autotuning☆23Updated 5 years ago
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago
- ☆52Updated 3 years ago
- ☆124Updated last year
- [ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models☆22Updated last year
- ☆127Updated 2 months ago
- Benchmark code for the "Online normalizer calculation for softmax" paper☆85Updated 6 years ago
- ☆11Updated last year
- Trans different platform's network to International Representation(IR)☆44Updated 6 years ago
- llm deploy project based onnx.☆31Updated 5 months ago
- ☆139Updated 10 months ago
- ☆71Updated 2 years ago
- ☆23Updated last year