oramasearch / minbpeLinks
Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization. Pure JavaScript port of karpathy/minbpe
☆18Updated last year
Alternatives and similar repositories for minbpe
Users that are interested in minbpe are comparing it to the libraries listed below
Sorting:
- ☆18Updated 4 years ago
- JSON canonicalize function☆51Updated 6 months ago
- Yet another auto expiring, max sizable cache☆50Updated 2 weeks ago
- A bunch of functions for building HTTP servers☆22Updated 7 months ago
- A set helpers for writing dynamic SQL queries with `pg-sql` in Javascript.☆38Updated 4 years ago
- ☆53Updated 4 years ago
- Fast Map built for keys that are always fixed size uniformly distributed buffers.☆37Updated 3 years ago
- A distributed database service for the Concordant platform.☆13Updated 3 years ago
- Promise Semaphore☆26Updated 3 years ago
- Zero-overhead asynchronous series/each/map function calls☆70Updated 5 years ago
- Private symbols for node.js☆14Updated 4 years ago
- a language acceptor for the JavaScript Pattern (regular expression internals) grammar☆16Updated 3 years ago
- Alternate Promise.race() implementation which doesn't leak memory, courtesy Brian Kim (https://github.com/brainkim)☆14Updated 2 years ago
- Node.js Workers, except on the same thread☆77Updated 3 years ago
- A noop logger interface for Node.js modules☆32Updated 5 years ago
- ☆77Updated last year
- Native file system extensions for advanced file operations☆37Updated last month
- common callbags☆28Updated 2 years ago
- Determines whether a Node file is a Module (`import`) or a Script (`require`)☆11Updated 5 years ago
- emoji-index☆15Updated last year
- An Opinionated Message Queue with an emitter-style API☆163Updated 2 months ago
- Same as sqlite-tag but without the native sqlite3 module dependency☆19Updated 7 months ago
- Simple, fun, memory safe language.☆16Updated 2 years ago
- small ~1k unzip library☆16Updated 4 months ago
- Gracefully shutdown fastify☆80Updated last month
- Generate TypeScript runtime types from your database schema.☆37Updated last year
- load a plugin via a synchronous worker☆58Updated last year
- Simple benchmarking suite powered by HDR histograms.☆87Updated 6 months ago
- 🚇 Utility to "pipe" async iterables together☆43Updated 2 years ago
- 👣 Effective tracing for the edge and origins☆48Updated 2 months ago