mithril-security / tokenizers-wasmLinks
wasm bindings for huggingface tokenizers library
☆34Updated 3 years ago
Alternatives and similar repositories for tokenizers-wasm
Users that are interested in tokenizers-wasm are comparing it to the libraries listed below
Sorting:
- GPU accelerated client-side embeddings for vector search, RAG etc.☆65Updated last year
- Fast and versatile tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and…☆36Updated 2 weeks ago
- Simple high-throughput inference library☆149Updated 5 months ago
- A complete(grpc service and lib) Rust inference with multilingual embedding support. This version leverages the power of Rust for both GR…☆39Updated last year
- ☆39Updated 3 years ago
- GGML implementation of BERT model with Python bindings and quantization.☆55Updated last year
- ☆139Updated last year
- Optimizing bit-level Jaccard Index and Population Counts for large-scale quantized Vector Search via Harley-Seal CSA and Lookup Tables☆21Updated 5 months ago
- webassembly binding for Hora Approximate Nearest Neighbor Search Library☆58Updated 4 years ago
- implement llava using candle☆15Updated last year
- Locality Sensitive Hashing☆73Updated 2 years ago
- A high-performance constrained decoding engine based on context free grammar in Rust☆55Updated 5 months ago
- Web browser version of StarCoder.cpp☆44Updated 2 years ago
- Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"☆65Updated 2 years ago
- ReLM is a Regular Expression engine for Language Models☆106Updated 2 years ago
- XTR: Rethinking the Role of Token Retrieval in Multi-Vector Retrieval☆58Updated last year
- Inference engine for GLiNER models, in Rust☆74Updated this week
- Unofficial python bindings for the rust llm library. 🐍❤️🦀☆76Updated 2 years ago
- ☆134Updated last year
- Python bindings for ggml☆146Updated last year
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated last year
- Rust crate for some audio utilities☆25Updated 7 months ago
- Latent Large Language Models☆19Updated last year
- open source tooling for AI search and understanding☆51Updated 2 years ago
- NLP with Rust for Python 🦀🐍☆65Updated 5 months ago
- Plug-and-play Search Interfaces with Pyserini and Hugging Face☆32Updated 2 years ago
- Vector Database with support for late interaction and token level embeddings.☆55Updated 4 months ago
- Pre-train Static Word Embeddings☆87Updated last month
- Tree-based indexes for neural-search☆32Updated last year
- ☆35Updated 2 years ago