Alternatives and similar repositories for tiktokenizer:

Users that are interested in tiktokenizer are comparing it to the libraries listed below

• philschmid / clipper.js
  HTML to Markdown converter and crawler.
  ☆518Updated last year
• mckaywrigley / clarity-ai
  A Perplexity clone.
  ☆1,280Updated 10 months ago
• MagnivOrg / prompt-layer-library
  🍰 PromptLayer - Maintain a log of your prompts and OpenAI API requests. Track, debug, and replay old completions.
  ☆551Updated 2 weeks ago
• poe-platform / poe-protocol
  The Poe bot protocol
  ☆441Updated last year
• dqbd / tiktoken
  JS port and JS/WASM bindings for openai/tiktoken
  ☆791Updated this week
• hundredblocks / transcription_demo
  ☆263Updated 11 months ago
• langchain-ai / auto-evaluator
  ☆756Updated last year
• mshumer / anthropic_with_functions
  ☆196Updated last year
• domeccleston / sharegpt
  Easily share permanent links to ChatGPT conversations with your friends
  ☆1,752Updated 4 months ago
• video-db / StreamRAG
  Video Search and Streaming Agent 🕵️‍♂️
  ☆459Updated last year
• e2b-dev / code-interpreter
  Python & JS/TS SDK for running AI-generated code/code interpreting in your AI app
  ☆1,475Updated this week
• fatwang2 / search4all
  Personal AI search copilot, open-source Perplexity
  ☆753Updated 7 months ago
• dillionverma / llm.report
  📊 llm.report is an open-source logging and analytics platform for OpenAI: Log your ChatGPT API requests, analyze costs, and improve your…
  ☆984Updated 9 months ago
• recalign / RecAlign
  Chrome extension to filter your feed with LLM according to an explicitly stated and user-editable preference.
  ☆254Updated last year
• srush / MiniChain
  A tiny library for coding with large language models.
  ☆1,223Updated 7 months ago
• 0xdevalias / chatgpt-source-watch
  Analyzing the evolution of ChatGPT's codebase through time with curated archives and scripts
  ☆276Updated 6 months ago
• botisan-ai / gpt3-tokenizer
  Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.
  ☆171Updated 2 years ago
• zahidkhawaja / langchain-chat-nextjs
  Next.js frontend for LangChain Chat.
  ☆991Updated last year
• msoedov / langcorn
  ⛓️ Serving LangChain LLM apps and agents automagically with FastApi. LLMops
  ☆915Updated 7 months ago
• shroominic / codebox-api
  👾📦 CodeBoxAPI is the simplest sandboxing infrastructure for your LLM Apps and Services.
  ☆334Updated 2 weeks ago
• niieani / gpt-tokenizer
  The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4 / GPT-4o / GPT-o1. Port of OpenAI's tiktoken with…
  ☆520Updated 2 months ago
• freddyaboulton / gradio-tools
  ☆580Updated last year
• langchain-ai / weblangchain
  LangChain-powered web researcher chatbot. Searches for sources on the web and cites them in generated answers.
  ☆515Updated 10 months ago
• langchain-ai / text-split-explorer
  ☆251Updated last year
• michaelthwan / searchGPT
  Grounded search engine (i.e. with source reference) based on LLM / ChatGPT / OpenAI API. It supports web search, file content search etc.
  ☆679Updated 5 months ago
• booydar / recurrent-memory-transformer
  [NeurIPS 22] [AAAI 24] Recurrent Transformer-based long-context architecture.
  ☆760Updated 3 months ago
• mckaywrigley / kindle-gpt
  ☆258Updated last year
• ianarawjo / ChainForge
  An open-source visual programming environment for battle-testing prompts to LLMs.
  ☆2,487Updated this week
• run-llama / finetune-embedding
  Fine-Tuning Embedding for RAG with Synthetic Data
  ☆483Updated last year
• plastic-labs / tutor-gpt
  Theory-of-mind powered AI tutor using o1 style reasoning
  ☆780Updated this week