lenML / tokenizersLinks
a lightweight no-dependency fork from transformers.js (only tokenizers)
☆20Updated 3 months ago
Alternatives and similar repositories for tokenizers
Users that are interested in tokenizers are comparing it to the libraries listed below
Sorting:
- Simple structured data from any LLM☆47Updated 3 months ago
- GLiNER inference in JavaScript☆19Updated 6 months ago
- Using embeddings compressed by Product Quantization, in Javascript☆31Updated 2 years ago
- LLM tools used in production at Dexa☆103Updated 7 months ago
- ☆31Updated 2 years ago
- ☆26Updated 9 months ago
- The official Node.js / Typescript library for the Steel API☆14Updated last week
- Proof of concept for a generative AI application framework powered by WebAssembly and Extism☆14Updated 2 years ago
- DuckDB NodeJS bindings☆48Updated 2 years ago
- CodeMirror extension to hook up a Model Context Provider (MCP)☆77Updated 3 weeks ago
- Parallel wasm Barnes-Hut t-SNE implementation written in Rust.☆21Updated last year
- GPU accelerated client-side embeddings for vector search, RAG etc.☆66Updated last year
- iterate quickly with llama.cpp hot reloading. use the llama.cpp bindings with bun.sh☆51Updated last year
- TypeScript generator for llama.cpp Grammar directly from TypeScript interfaces☆140Updated last year
- Codeium code completion integration for CodeMirror 6☆81Updated this week
- Annotated AST for the purposes of LLM code repository context. Use this novel approach to pre-process your Github repository. This projec…☆33Updated 8 months ago
- ☆15Updated 2 years ago
- ☆26Updated 2 weeks ago
- A SQLite extension for generating text embeddings from remote APIs (OpenAI, Nomic, Ollama, llamafile...)☆133Updated 10 months ago
- ☆20Updated last year
- Val Town TypeScript SDK☆35Updated last week
- Structured inference with Llama 2 in your browser☆53Updated 10 months ago
- dbSurface is a SQL editor made for pgvector.☆21Updated 2 months ago
- OpenTelemetry Tracing in 200 lines of JavaScript☆36Updated 11 months ago
- AskIt (for JavaScript/TypeScript): Unified programming interface for large language models (GPT-4, GPT-3.5)☆34Updated last year
- Library to generate vector embeddings in NodeJS☆139Updated 4 months ago
- ☆68Updated 2 years ago
- ☆145Updated 2 years ago
- Browse, search, and visualize ONNX models.☆34Updated 3 months ago
- ☆154Updated 2 months ago