oramasearch / minbpeLinks
Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization. Pure JavaScript port of karpathy/minbpe
☆18Updated last year
Alternatives and similar repositories for minbpe
Users that are interested in minbpe are comparing it to the libraries listed below
Sorting:
- a language acceptor for the JavaScript Pattern (regular expression internals) grammar☆16Updated 3 years ago
- JSON canonicalize function☆52Updated 9 months ago
- Alternate Promise.race() implementation which doesn't leak memory, courtesy Brian Kim (https://github.com/brainkim)☆13Updated 2 years ago
- A set helpers for writing dynamic SQL queries with `pg-sql` in Javascript.☆38Updated 4 years ago
- FaaS like GraphQL runtime☆27Updated 11 months ago
- Private symbols for node.js☆14Updated 5 years ago
- 👣 Effective tracing for the edge and origins☆48Updated 5 months ago
- Simple benchmarking suite powered by HDR histograms.☆87Updated last month
- small ~1k unzip library☆16Updated 6 months ago
- A distributed database service for the Concordant platform.☆13Updated 3 years ago
- Promise Semaphore☆26Updated 3 years ago
- Fast Map built for keys that are always fixed size uniformly distributed buffers.☆37Updated 3 years ago
- An Opinionated Message Queue with an emitter-style API☆165Updated 3 weeks ago
- A fast fifo implementation similar to the one powering nextTick in Node.js core☆61Updated last year
- Syntactic sugar for creating JSON Type Definitions, and generating compatible JSON Schema from JTD schemas☆18Updated last year
- Yet another auto expiring, max sizable cache☆51Updated 3 months ago
- load a plugin via a synchronous worker☆58Updated 2 years ago
- Perform async work synchronously in Node.js using worker threads☆40Updated 2 years ago
- ☆53Updated 4 years ago
- Tiny, type-safe JSON-RPC 2.0 implementation☆20Updated 2 years ago
- 🐉 An experimental node.js FFI library☆40Updated 6 months ago
- Utility belt for using async iterators with streams☆29Updated 6 years ago
- Run a command if a file changes via Git hooks☆37Updated last week
- ☆40Updated 3 months ago
- Generate TypeScript runtime types from your database schema.☆37Updated last year
- Zero-overhead asynchronous series/each/map function calls☆71Updated 5 years ago
- 🕵️♀️ Light-weight observable implementation and functional utilities in TypeScript☆48Updated 2 years ago
- ☆23Updated 6 years ago
- A noop logger interface for Node.js modules☆33Updated 5 years ago
- Decentralized Operational Transformation (DOT)☆16Updated 8 years ago