wangkuiyi / huggingface-tokenizer-in-cxx

Related projects: ⓘ

• mlc-ai / tokenizers-cpp
  Universal cross-platform tokenizers binding to HF and sentencepiece
  ☆246Updated last month
• daquexian / faster-rwkv
  ☆123Updated 9 months ago
• dhpollack / huggingface_libtorch
  Minimal example of using a traced huggingface transformers model with libtorch
  ☆35Updated 4 years ago
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ, and export to onnx/onnx-runtime easily.
  ☆141Updated 3 weeks ago
• yvonwin / qwen2.cpp
  qwen2 and llama3 cpp implementation
  ☆34Updated 3 months ago
• triton-inference-server / onnxruntime_backend
  The Triton backend for the ONNX Runtime.
  ☆122Updated this week
• inisis / OnnxSlim
  A Toolkit to Help Optimize Large Onnx Model
  ☆53Updated this week
• wangguojim / LargeScale
  ☆18Updated 4 months ago
• LeeJuly30 / BERTCpp
  implement bert in pure c++
  ☆30Updated 4 years ago
• npuichigo / openai_trtllm
  OpenAI compatible API for TensorRT LLM triton backend
  ☆148Updated last month
• neuralmagic / AutoFP8
  ☆145Updated last month
• microsoft / batch-inference
  Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.
  ☆81Updated last month
• triton-inference-server / common
  Common source, scripts and utilities shared across all Triton repositories.
  ☆62Updated last week
• SkyworkAI / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆15Updated 3 months ago
• triton-inference-server / tensorrt_backend
  The Triton backend for TensorRT.
  ☆59Updated last week
• Rayrtfr / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆17Updated last year
• luchangli03 / onnxsim_large_model
  simplify >2GB large onnx model
  ☆41Updated 6 months ago
• mlc-ai / llm-perf-bench
  ☆110Updated 4 months ago
• void-main / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆58Updated 11 months ago
• xyzhang626 / embeddings.cpp
  ggml implementation of embedding models including SentenceTransformer and BGE
  ☆50Updated 8 months ago
• bytedance / decoupleQ
  A quantization algorithm for LLM
  ☆98Updated 2 months ago
• iamlemec / bert.cpp
  GGML implementation of BERT model with Python bindings and quantization.
  ☆51Updated 7 months ago
• tpoisonooo / llama.onnx
  LLaMa/RWKV onnx models, quantization and testcase
  ☆345Updated last year
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆78Updated 4 months ago
• NVIDIA-AI-IOT / whisper_trt
  A project that optimizes Whisper for low latency inference using NVIDIA TensorRT
  ☆47Updated 2 months ago
• EdVince / whisper-trtllm
  Whisper in TensorRT-LLM
  ☆14Updated 11 months ago
• OpenBMB / cpm_kernels
  ☆23Updated 11 months ago
• NetEase-FuXi / EETQ
  Easy and Efficient Quantization for Transformers
  ☆172Updated 2 months ago
• modelscope / dash-infer
  DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …
  ☆130Updated 3 weeks ago
• triton-inference-server / vllm_backend
  ☆170Updated this week