transitive-bullshit / compare-tokenizersLinks
A test suite comparing Node.js BPE tokenizers for use with AI models.
☆31Updated last year
Alternatives and similar repositories for compare-tokenizers
Users that are interested in compare-tokenizers are comparing it to the libraries listed below
Sorting:
- Convert Zod schemas to JSON schemas which are optionally compatible with OpenAI's structured outputs.☆17Updated 7 months ago
- OpenAI Function Calling in Typescript using Zod☆89Updated last year
- Minimal and opinionated OpenAI client powered by fetch.☆190Updated 4 months ago
- Super Simple API framework, type safe, automatic OpenAPI, MCP support, client RPC, streaming with SSE☆51Updated this week
- markdown documentation language☆31Updated 7 months ago
- Securely spawn Deno workers from Node.js☆19Updated last week
- Distance functions to compare vectors☆71Updated 7 months ago
- Git on a Durable Object☆34Updated last week
- OpenAI's tiktoken but with node bindings☆104Updated last year
- Yet another library to extract text from MS Office and PDF files☆78Updated 10 months ago
- support for type acquisition with url imports, npm, jsr prefixes☆16Updated 11 months ago
- Split large texts into chunks with a maximum number of tokens. Split by fixed size or by sentence.☆30Updated last year
- ☆13Updated 2 years ago
- Generate RSC Payloads using an LLM☆38Updated 7 months ago
- A Visual Studio Code extension loads types(.d.ts) for esm.sh imports.☆34Updated 7 months ago
- Basic primitives for working with binary data☆24Updated 2 months ago
- write your request handlers once, run anywhere☆36Updated last month
- 🏜️ A sandbox that runs user-supplied JavaScript/TypeScript scripts.☆53Updated last year
- This Tailwind CSS plugin provides custom variants to apply styles to apps that are running on the ToDesktop platform☆18Updated last year
- pnpm update for pnpm workspace catalogs.☆13Updated 2 months ago
- Isomorphic utilities for GPT-3 tokenization and prompt building.☆11Updated 2 years ago
- Merge multiple streams into a unified stream☆57Updated 7 months ago
- A Firecrawl compatible API implemented in Cloudflare Workers☆36Updated 2 months ago
- A VM module for Node.js that utilizes the secure environment provided by Deno☆78Updated last month
- A Cloudflare Workers-based API for extracting and converting web page content to Markdown using DOM-Distiller and Turndown.☆22Updated last week
- JSON parser for streaming objects live from an LLM's output☆33Updated last year
- Continue block comments in CodeMirror☆20Updated 2 weeks ago
- Typescript wrapper for the Hugging Face Inference API.☆27Updated 2 years ago
- Partial JSON parser extracted from OpenAI's vendored version.☆13Updated 7 months ago
- Build your Cloudflare Workers, on Cloudflare Workers☆18Updated 2 years ago