Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization. Pure JavaScript port of karpathy/minbpe
☆18Feb 19, 2024Updated 2 years ago
Alternatives and similar repositories for minbpe
Users that are interested in minbpe are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Split large texts into chunks with a maximum number of tokens. Split by fixed size or by sentence.☆34Mar 1, 2024Updated 2 years ago
- data repository☆12Nov 21, 2024Updated last year
- Minimalist (yet helpful) monorepo manager for Deno☆17Dec 13, 2023Updated 2 years ago
- ☆17Jan 4, 2023Updated 3 years ago
- Bites of knowledge to help you build your fundamentals as a frontend developer☆43Aug 28, 2025Updated 10 months ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Homotopy type theory cheatsheets☆12Apr 15, 2026Updated 2 months ago
- A set of macros and functions to make defining a C module easier☆11Sep 9, 2019Updated 6 years ago
- A take on TodoMVC developed using functional effects with @matechs/effect and fp-ts☆15Jan 6, 2023Updated 3 years ago
- A collection of additional language phonology settings for use with VulgarLang.☆13Aug 19, 2022Updated 3 years ago
- Fast Map built for keys that are always fixed size uniformly distributed buffers.☆37Mar 21, 2022Updated 4 years ago
- Non-allocating command line flag parser☆18Mar 26, 2026Updated 3 months ago
- List key diff algorithm.☆15Nov 10, 2015Updated 10 years ago
- Monitor millions of S3 objects without external dependencies☆24Feb 6, 2025Updated last year
- Bento - Rust Container Runtime☆54Feb 23, 2026Updated 4 months ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A small utility for creating warnings and emitting them☆38Jun 17, 2026Updated 2 weeks ago
- Scroll to the current anchor in the url if possible☆11Jun 18, 2017Updated 9 years ago
- NLTK ported to Javascript☆13Sep 3, 2017Updated 8 years ago
- 📤 Magically generate `fetch` types from OpenAPI schemas for zero-cost browser-native api clients☆21Apr 17, 2025Updated last year
- Demo of Apollo Federation☆12Apr 9, 2024Updated 2 years ago
- Generate typescript model definitions with just the JSON schema (including reference resolution).☆14Feb 1, 2021Updated 5 years ago
- Promisify a callback-style function, resolving with [err,val]☆31May 19, 2021Updated 5 years ago
- ☆43Mar 6, 2023Updated 3 years ago
- an animated Mayan Calendar☆12Jan 12, 2014Updated 12 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- [obsolete] [incomplete] typed JavaScript interfaces for Facebook Flow☆15Jan 18, 2015Updated 11 years ago
- Implementation of "practical type inference for arbitrary-rank types" in Javascript☆12Mar 27, 2019Updated 7 years ago
- The same application, implemented and measured in both ReactN and redux.☆11Dec 10, 2020Updated 5 years ago
- HTML5 File and FormData APIs for Node.JS☆31Jan 7, 2017Updated 9 years ago
- (Alternative) Visualizer for XState☆12Mar 2, 2023Updated 3 years ago
- Node 18's node:test, as an npm package☆99Dec 21, 2024Updated last year
- ☆84Apr 23, 2025Updated last year
- Photos of U.S. congressional representatives and scrapers used to collect them.☆14Jan 30, 2026Updated 5 months ago
- Ideas for the decipherment of Easter Island's rongorongo writing☆13Sep 4, 2023Updated 2 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Several fast 32-bit xor-shift random number generators implemented in Javascript.☆14Jul 13, 2022Updated 3 years ago
- Simple and small library that allows disassembly of raw bytes for multiple architectures☆32May 5, 2023Updated 3 years ago
- A Database for the browser using compiled SQL queries in WASM☆15Mar 4, 2023Updated 3 years ago
- Overview and goals.☆15Dec 16, 2015Updated 10 years ago
- JavaScript library for drawing Chemical Structures☆14Oct 16, 2018Updated 7 years ago
- Native app for botlist.co, 100% in JSON☆12Nov 2, 2016Updated 9 years ago
- Generate Sequelize model definitions from a Swagger 2.0 schema☆11Nov 13, 2020Updated 5 years ago