mithril-security / tokenizers-wasmLinks
wasm bindings for huggingface tokenizers library
☆34Updated 3 years ago
Alternatives and similar repositories for tokenizers-wasm
Users that are interested in tokenizers-wasm are comparing it to the libraries listed below
Sorting:
- ☆39Updated 2 years ago
- GPU accelerated client-side embeddings for vector search, RAG etc.☆66Updated last year
- Fast and versatile tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and…☆29Updated 5 months ago
- GGML implementation of BERT model with Python bindings and quantization.☆57Updated last year
- Modular Rust transformer/LLM library using Candle☆36Updated last year
- Run ONNX and TensorFlow inference in the browser.☆75Updated 2 years ago
- Web browser version of StarCoder.cpp☆45Updated 2 years ago
- A high-performance constrained decoding engine based on context free grammar in Rust☆55Updated 3 months ago
- High-performance MinHash implementation in Rust with Python bindings for efficient similarity estimation and deduplication of large datas…☆199Updated last month
- Python bindings for ggml☆146Updated 11 months ago
- Simple high-throughput inference library☆127Updated 3 months ago
- implement llava using candle☆15Updated last year
- Locality Sensitive Hashing☆72Updated 2 years ago
- Vector Database with support for late interaction and token level embeddings.☆55Updated 2 months ago
- A complete(grpc service and lib) Rust inference with multilingual embedding support. This version leverages the power of Rust for both GR…☆39Updated last year
- This project aims to make RWKV Accessible to everyone using a Hugging Face like interface, while keeping it close to the R and D RWKV bra…☆65Updated 2 years ago
- Proof of concept for running moshi/hibiki using webrtc☆20Updated 6 months ago
- A Demo server serving Bert through ONNX with GPU written in Rust with <3☆41Updated 4 years ago
- Add local LLMs to your Web or Electron apps! Powered by Rust + WebGPU☆103Updated 2 years ago
- Unofficial python bindings for the rust llm library. 🐍❤️🦀☆75Updated 2 years ago
- ☆156Updated 2 years ago
- Optimizing bit-level Jaccard Index and Population Counts for large-scale quantized Vector Search via Harley-Seal CSA and Lookup Tables☆20Updated 3 months ago
- ☆130Updated last year
- ☆49Updated 5 years ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated last year
- Experiments with BitNet inference on CPU☆54Updated last year
- IBM development fork of https://github.com/huggingface/text-generation-inference☆61Updated 3 months ago
- Inference engine for GLiNER models, in Rust☆65Updated last month
- Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs☆110Updated last year
- Tutorial on how to convert machine learned models into ONNX☆16Updated 2 years ago