Alternatives and similar repositories for gpt-tokenizer

Users that are interested in gpt-tokenizer are comparing it to the libraries listed below

• dqbd / tiktoken

  View on GitHub
  JS port and JS/WASM bindings for openai/tiktoken
  ☆1,008Aug 9, 2025Updated 6 months ago
• latitudegames / GPT-3-Encoder

  View on GitHub
  Javascript BPE Encoder Decoder for GPT-2 / GPT-3
  ☆723Apr 16, 2023Updated 2 years ago
• botisan-ai / gpt3-tokenizer

  View on GitHub
  Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.
  ☆172Jan 27, 2023Updated 3 years ago
• langchain-ai / langchainjs

  View on GitHub
  🦜🔗 Build context-aware reasoning applications
  ☆16,960Updated this week
• dzhng / zod-gpt

  View on GitHub
  Get structured, fully typed, and validated JSON outputs from OpenAI and Anthropic models.
  ☆629Apr 26, 2024Updated last year
• openai / tiktoken

  View on GitHub
  tiktoken is a fast BPE tokeniser for use with OpenAI's models.
  ☆17,258Feb 8, 2026Updated last week
• oramasearch / chunker

  View on GitHub
  Split large texts into chunks with a maximum number of tokens. Split by fixed size or by sentence.
  ☆34Mar 1, 2024Updated last year
• Stevenic / vectra

  View on GitHub
  Vectra is a local vector database for Node.js with features similar to pinecone but built using local files.
  ☆582Jan 20, 2026Updated 3 weeks ago
• openai / openai-node

  View on GitHub
  Official JavaScript / TypeScript library for the OpenAI API
  ☆10,624Updated this week
• vercel / ai

  View on GitHub
  The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered application…
  ☆21,642Updated this week
• johannschopplich / tokenx

  View on GitHub
  📐 Fast token estimation at 96% accuracy of a full tokenizer in a 2kB bundle
  ☆114Jan 29, 2026Updated 2 weeks ago
• unjs / unpdf

  View on GitHub
  📄 PDF extraction and rendering across all JavaScript runtimes
  ☆1,018Nov 29, 2025Updated 2 months ago
• huggingface / transformers.js

  View on GitHub
  State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
  ☆15,352Updated this week
• anysphere / priompt

  View on GitHub
  Prompt design using JSX.
  ☆2,762Oct 15, 2025Updated 4 months ago
• StefanTerdell / zod-to-json-schema

  View on GitHub
  Converts Zod schemas to Json schemas
  ☆1,239Dec 27, 2025Updated last month
• stackblitz-labs / use-stick-to-bottom

  View on GitHub
  A lightweight React Hook intended mainly for AI chat applications, for smoothly sticking to bottom of messages
  ☆676Feb 6, 2026Updated last week
• mlc-ai / web-llm

  View on GitHub
  High-performance In-browser LLM Inference Engine
  ☆17,258Feb 9, 2026Updated last week
• anubra266 / ecopanda

  View on GitHub
  Panda CSS Ecosystem
  ☆12May 10, 2024Updated last year
• promptfoo / promptfoo

  View on GitHub
  Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude,…
  ☆10,462Updated this week
• vercel-labs / blue-green

  View on GitHub
  ☆17Dec 12, 2025Updated 2 months ago
• moeru-ai / xsai

  View on GitHub
  🤖💬 extra-small AI SDK.
  ☆566Feb 3, 2026Updated last week
• microsoft / TypeChat

  View on GitHub
  TypeChat is a library that makes it easy to build natural language interfaces using types.
  ☆8,608Dec 4, 2025Updated 2 months ago
• vercel / modelfusion

  View on GitHub
  The TypeScript library for building AI applications.
  ☆1,316Jul 19, 2024Updated last year
• tantaraio / voy

  View on GitHub
  🕸️🦀 A WASM vector similarity search written in Rust
  ☆1,047Sep 20, 2023Updated 2 years ago
• ceifa / tiktoken-node

  View on GitHub
  OpenAI's tiktoken but with node bindings
  ☆106May 18, 2024Updated last year
• SpellcraftAI / openai-streams

  View on GitHub
  Tools for working with OpenAI streams in Node.js and TypeScript.
  ☆293Jul 20, 2023Updated 2 years ago
• yoshoku / hnswlib-node

  View on GitHub
  hnswlib-node provides Node.js bindings for Hnswlib
  ☆133Feb 6, 2026Updated last week
• xyflow / xyflow

  View on GitHub
  React Flow | Svelte Flow - Powerful open source libraries for building node-based UIs with React (https://reactflow.dev) or Svelte (https…
  ☆35,197Updated this week
• modelcontextprotocol / typescript-sdk

  View on GitHub
  The official TypeScript SDK for Model Context Protocol servers and clients
  ☆11,558Updated this week
• remorses / spiceflow

  View on GitHub
  Super Simple API framework, type safe, automatic OpenAPI, MCP support, client RPC, streaming with SSE
  ☆84Jan 23, 2026Updated 3 weeks ago
• pmndrs / valtio

  View on GitHub
  🧙 Valtio makes proxy-state simple for React and Vanilla
  ☆10,119Feb 10, 2026Updated last week
• transitive-bullshit / agentic

  View on GitHub
  Your API ⇒ Paid MCP. Instantly.
  ☆18,126Oct 9, 2025Updated 4 months ago
• electrovir / pdf-text-reader

  View on GitHub
  Dead simple pdf text reader
  ☆44Jan 19, 2026Updated 3 weeks ago
• Cainier / gpt-tokens

  View on GitHub
  Calculate the token consumption and amount of openai gpt message
  ☆115Feb 9, 2025Updated last year
• PlasmoHQ / plasmo

  View on GitHub
  🧩 The Browser Extension Framework
  ☆12,829Feb 9, 2026Updated last week
• mixmark-io / turndown

  View on GitHub
  🛏 An HTML to Markdown converter written in JavaScript
  ☆10,763Oct 24, 2025Updated 3 months ago
• isaacs / perfalize

  View on GitHub
  ☆17Oct 16, 2023Updated 2 years ago
• inocan-group / inferred-types

  View on GitHub
  primitives for advanced inference in TS
  ☆18Nov 21, 2025Updated 2 months ago
• googleapis / js-genai

  View on GitHub
  TypeScript/JavaScript SDK for Gemini and Vertex AI.
  ☆1,471Updated this week