transitive-bullshit / compare-tokenizers
A test suite comparing Node.js BPE tokenizers for use with AI models.
☆30Updated last year
Alternatives and similar repositories for compare-tokenizers:
Users that are interested in compare-tokenizers are comparing it to the libraries listed below
- hnswlib-node provides Node.js bindings for Hnswlib☆108Updated 2 weeks ago
- Minimal and opinionated OpenAI client powered by fetch.☆186Updated 3 months ago
- Super Simple API framework, type safe, automatic OpenAPI, MCP support, client RPC, streaming with SSE☆46Updated this week
- WebContainers, except it's a million times easier to use☆73Updated 2 years ago
- tRPC for extension and web 🧩🕸️☆36Updated last month
- Distance functions to compare vectors☆70Updated 5 months ago
- Merge multiple streams into a unified stream☆56Updated 5 months ago
- Enable your AWS Lambda hosted applications to stream responses via Cloudflare Workers.☆23Updated 2 years ago
- support for type acquisition with url imports, npm, jsr prefixes☆16Updated 10 months ago
- Convert Zod schemas to JSON schemas which are optionally compatible with OpenAI's structured outputs.☆17Updated 6 months ago
- Split large texts into chunks with a maximum number of tokens. Split by fixed size or by sentence.☆30Updated last year
- Programmatically start (AI) sandboxes on top of CodeSandbox☆62Updated last week
- A Cloudflare Workers-based API for extracting and converting web page content to Markdown using DOM-Distiller and Turndown.☆20Updated this week
- A Cloudflare Page showing how to do semantic search☆49Updated 4 months ago
- Simple orchestration for AI Agents built around Vercel's streamText. Lightweight alternative to LangGraph for agent handoffs and state tr…☆65Updated 2 months ago
- Partial JSON parser extracted from OpenAI's vendored version.☆13Updated 6 months ago
- 🏜️ A sandbox that runs user-supplied JavaScript/TypeScript scripts.☆51Updated last year
- A base64 encoder/decoder with gzip or deflate abilities.☆36Updated 5 months ago
- This Tailwind CSS plugin provides custom variants to apply styles to apps that are running on the ToDesktop platform☆18Updated last year
- markdown documentation language☆31Updated 5 months ago
- OpenAI Function Calling in Typescript using Zod☆89Updated last year
- A TypeScript library for building real-time, event-driven applications with XState and Cloudflare Workers. Seamlessly manage distributed …☆53Updated 4 months ago
- A VM module for Node.js that utilizes the secure environment provided by Deno☆76Updated 7 months ago
- Codeium code completion integration for CodeMirror 6☆78Updated 8 months ago
- ☆13Updated last year
- Experimental browser ide for building typescript libraries.☆49Updated last year
- Find unused class variables and methods in a typescript project☆18Updated last year
- Download all Cloudflare durable object state to a local SQLite database.☆21Updated last year
- A Firecrawl compatible API implemented in Cloudflare Workers☆31Updated last month
- write your request handlers once, run anywhere☆36Updated 2 weeks ago