mlc-ai / tokenizers-cppLinks
Universal cross-platform tokenizers binding to HF and sentencepiece
☆377Updated 2 weeks ago
Alternatives and similar repositories for tokenizers-cpp
Users that are interested in tokenizers-cpp are comparing it to the libraries listed below
Sorting:
- LLaMa/RWKV onnx models, quantization and testcase☆363Updated 2 years ago
- ☆68Updated 2 years ago
- ☆195Updated 3 months ago
- optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052☆475Updated last year
- Common source, scripts and utilities for creating Triton backends.☆342Updated 3 weeks ago
- ☆140Updated last year
- ☆128Updated 8 months ago
- Transformer related optimization, including BERT, GPT☆59Updated last year
- FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.☆886Updated 11 months ago
- Common utilities for ONNX converters☆276Updated last month
- GPTQ inference Triton kernel☆306Updated 2 years ago
- A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ/VPTQ, and export to onnx/onnx-runtime easily.☆177Updated 4 months ago
- ☆412Updated last year
- export llama to onnx☆132Updated 7 months ago
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆40Updated 6 months ago
- [MLSys'25] QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving; [MLSys'25] LServe: Efficient Long-sequence LLM Se…☆735Updated 5 months ago
- ☆125Updated last year
- ☆163Updated this week
- llm-export can export llm model to onnx.☆303Updated 7 months ago
- A quantization algorithm for LLM☆140Updated last year
- BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.☆660Updated 2 weeks ago
- onnxruntime-extensions: A specialized pre- and post- processing library for ONNX Runtime☆409Updated last week
- Benchmark code for the "Online normalizer calculation for softmax" paper☆98Updated 7 years ago
- Experimental projects related to TensorRT☆111Updated this week
- OpenAI compatible API for TensorRT LLM triton backend☆214Updated last year
- Running BERT without Padding☆475Updated 3 years ago
- The Triton backend for the ONNX Runtime.☆159Updated 3 weeks ago
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆138Updated this week
- ☆291Updated 3 weeks ago
- ☆150Updated 7 months ago