1rgs/tokenwiz external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

1rgs/tokenwiz

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/1rgs/tokenwiz)

Close

1rgs / tokenwizView on GitHubMore

• External links
• Readme badge

A clone of OpenAI's Tokenizer page for HuggingFace Models

☆46Nov 13, 2023Updated 2 years ago

Alternatives and similar repositories for tokenwiz

Users that are interested in tokenwiz are comparing it to the libraries listed below

• FanaHOVA / smol-scheduler

  View on GitHub
  🐣🕐📅 A simple utility to draft scheduling emails.
  ☆12Sep 13, 2023Updated 2 years ago
• transitive-bullshit / openai-partial-json-parser

  View on GitHub
  Partial JSON parser extracted from OpenAI's vendored version.
  ☆14Jul 18, 2025Updated 7 months ago
• richard-better-archive / note-tools

  View on GitHub
  A collection of my tools related to notetaking
  ☆10Apr 18, 2021Updated 4 years ago
• judell / TextQuoteAndPosition

  View on GitHub
  Encapsulate dom-anchor-text-quote and dom-anchor-text-position for use in browser scripts
  ☆13Sep 2, 2021Updated 4 years ago
• Und3rf10w / OpenJanus

  View on GitHub
  Janus is an opensource IA for Star Citizen
  ☆11Dec 23, 2023Updated 2 years ago
• CodeAlchemyAI / yt-to-blog

  View on GitHub
  ☆15Sep 15, 2023Updated 2 years ago
• ragnarkar / reddit_imitator

  View on GitHub
  ☆16Jun 20, 2023Updated 2 years ago
• cbh123 / youtune

  View on GitHub
  Fine tune SDXL on YouTube videos
  ☆182Aug 20, 2024Updated last year
• distribute-dev / modelplexer

  View on GitHub
  Multiplexer for AI protocols, agents, models, and tools
  ☆24Aug 8, 2025Updated 6 months ago
• pdparchitect / codepal

  View on GitHub
  ☆44Dec 14, 2022Updated 3 years ago
• yoheinakajima / captainfunction

  View on GitHub
  A Python package to dynamically load functions for OpenAI Assistant
  ☆54Dec 6, 2023Updated 2 years ago
• fofr / prompter.fofr.ai

  View on GitHub
  An app for generating prompts
  ☆28Aug 16, 2025Updated 6 months ago
• peterw / NotionWizard

  View on GitHub
  This is a simple Notion Page Assistant that uses OpenAI's functions to create a Notion page.
  ☆19Jul 28, 2023Updated 2 years ago
• camenduru / background-replacement-colab

  View on GitHub
  ☆21Sep 27, 2023Updated 2 years ago
• highlight-ing / mcp-bundler

  View on GitHub
  A microservice for bundling MCP servers from GitHub repositories and preparing it for deployment.
  ☆32Mar 25, 2025Updated 11 months ago
• Alignment-Lab-AI / Our-Projects

  View on GitHub
  A repository of projects and datasets under active development by Alignment Lab AI
  ☆22Dec 22, 2023Updated 2 years ago
• camenduru / SSD-1B-colab

  View on GitHub
  ☆25Dec 18, 2023Updated 2 years ago
• ajndkr / pytest-langchain

  View on GitHub
  ✅ Pytest-style test runner for langchain projects
  ☆25Feb 26, 2023Updated 3 years ago
• SumanthRH / tokenization

  View on GitHub
  A comprehensive deep dive into the world of tokens
  ☆227Jun 24, 2024Updated last year
• i207M / taxy-ai-backup

  View on GitHub
  TaxyAI: Open-source browser automation with GPT-4 (backup)
  ☆55May 8, 2023Updated 2 years ago
• balisujohn / localpilot

  View on GitHub
  codellama on CPU without Docker
  ☆25Feb 8, 2024Updated 2 years ago
• stunningpixels / lou-eval

  View on GitHub
  Track the progress of LLM context utilisation
  ☆55Apr 14, 2025Updated 10 months ago
• deployradiant / pychatml

  View on GitHub
  Chat Markup Language conversation library
  ☆55Jan 3, 2024Updated 2 years ago
• uclaml / PDE

  View on GitHub
  Official repo of Progressive Data Expansion: data, code and evaluation
  ☆29Nov 16, 2023Updated 2 years ago
• tomhazledine / related-posts

  View on GitHub
  ☆30Apr 26, 2024Updated last year
• simonw / llm-anyscale-endpoints

  View on GitHub
  LLM plugin for models hosted by Anyscale Endpoints
  ☆35Apr 22, 2024Updated last year
• NolanoOrg / llama-int4-quant

  View on GitHub
  ☆26Mar 11, 2023Updated 2 years ago
• Superflows-AI / chat-ui

  View on GitHub
  ☆32Jul 12, 2024Updated last year
• kookaburracodes / investor-education-chatchain

  View on GitHub
  Not financial advice.
  ☆28Mar 18, 2023Updated 2 years ago
• visionmedia / stats

  View on GitHub
  JavaScript source statistics with simple API & CLI written with node.js
  ☆79Mar 24, 2015Updated 10 years ago
• jackbandy / bookcorpus-datasheet

  View on GitHub
  Documentation effort for the BookCorpus dataset
  ☆34Jun 2, 2021Updated 4 years ago
• stanford-futuredata / Megatron-LM

  View on GitHub
  Ongoing research training transformer models at scale
  ☆38Jan 19, 2024Updated 2 years ago
• jflam / text-is-all-you-need

  View on GitHub
  Team lazy's approach to UI for the AIs
  ☆46Mar 3, 2023Updated 3 years ago
• youscan / language-models

  View on GitHub
  ☆27Jun 12, 2023Updated 2 years ago
• Steer-AI / SteerCode

  View on GitHub
  ☆38Sep 10, 2025Updated 5 months ago
• yoheinakajima / GPTvsGPT

  View on GitHub
  A playful script to get two AI assistants to converse using OpenAI Assistants API
  ☆205Nov 7, 2023Updated 2 years ago
• zswitten / chat_out_loud_gpt

  View on GitHub
  Hands-free companionship on demand.
  ☆77Mar 23, 2023Updated 2 years ago
• martinseanhunt / slack-gpt

  View on GitHub
  A simple slack app / bot starter that fetches answers to questions using Langchain, OpenAI and a Pincone vectorstore
  ☆71May 3, 2023Updated 2 years ago
• emrgnt-cmplxty / zero-shot-replication

  View on GitHub
  ☆74Sep 5, 2023Updated 2 years ago