niieani / gpt-tokenizerLinks
The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT models (o1, o3, o4, gpt-4o, gpt-4, etc.). Port of OpenAI's tiktoken with additional features.
☆578Updated last week
Alternatives and similar repositories for gpt-tokenizer
Users that are interested in gpt-tokenizer are comparing it to the libraries listed below
Sorting:
- JS port and JS/WASM bindings for openai/tiktoken☆882Updated 2 months ago
- Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.☆171Updated 2 years ago
- Calculate the token consumption and amount of openai gpt message☆115Updated 4 months ago
- Javascript BPE Encoder Decoder for GPT-2 / GPT-3☆721Updated 2 years ago
- OpenAI's tiktoken but with node bindings☆104Updated last year
- Tools for working with OpenAI streams in Node.js and TypeScript.☆291Updated last year
- Streaming, source-agnostic EventSource/Server-Sent Events parser☆385Updated last month
- Parse incomplete json text in best-effort manner☆227Updated 4 months ago
- 🛠 openai function calling tools for JS/TS☆298Updated last year
- JavaScript/Typescript SDK for Qdrant Vector Database☆342Updated this week
- ☆252Updated 2 months ago
- Everything you need to start building ChatGPT Plugins in JS/TS 🔥☆543Updated 2 years ago
- 🕸️🦀 A WASM vector similarity search written in Rust☆976Updated last year
- Vectra is a local vector database for Node.js with features similar to pinecone but built using local files.☆492Updated last month
- Access to Anthropic's safety-first language model APIs☆1,055Updated this week
- Get structured, fully typed, and validated JSON outputs from OpenAI and Anthropic models.☆625Updated last year
- Parse partial JSON generated by LLM☆141Updated 9 months ago
- pgvector support for Node.js, Deno, and Bun (and TypeScript)☆386Updated last month
- Vector Storage is a vector database that enables semantic similarity searches on text documents in the browser's local storage. It uses O…☆226Updated 6 months ago
- JavaScript implementation of LiteLLM.☆127Updated 3 months ago
- Integrate 200+ LLMs with one TypeScript SDK using OpenAI's format.☆265Updated 2 months ago
- Node.js client for Replicate☆557Updated last month
- 🔎 Search from DuckDuckGo and utilize its spice APIs in Node☆199Updated 3 months ago
- The TypeScript library for building AI applications.☆1,280Updated 11 months ago
- Turn a stream of token into a parsable JSON object as soon as possible. Enable Streaming UI for AI app based on LLM.☆118Updated last year
- A client side vector search library that can embed, store, search, and cache vectors. Works on the browser and node. It outperforms OpenA…☆210Updated last year
- Fast HTML to markdown converter for NodeJS or the browser☆226Updated 10 months ago
- JS tokenizer for LLaMA 1 and 2☆353Updated 11 months ago
- Minimal and opinionated OpenAI client powered by fetch.☆190Updated 5 months ago
- Powerful machine learning library for Node.js – uses Python's scikit-learn under the hood.☆196Updated 7 months ago