oramasearch / minbpeLinks
Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization. Pure JavaScript port of karpathy/minbpe
☆18Updated last year
Alternatives and similar repositories for minbpe
Users that are interested in minbpe are comparing it to the libraries listed below
Sorting:
- Peek a stream!☆13Updated 3 years ago
- JSON canonicalize function☆49Updated 3 months ago
- Private symbols for node.js☆14Updated 4 years ago
- A set helpers for writing dynamic SQL queries with `pg-sql` in Javascript.☆38Updated 3 years ago
- Type-safe SQL builder using ES2015 template tags☆11Updated 3 years ago
- ☆77Updated 9 months ago
- Running Node within Node (a fork of synchronous-worker)☆27Updated 2 years ago
- Speed up zlib operations by running them using the sync APIs but in a Worker☆29Updated 4 years ago
- Promise Semaphore☆26Updated 3 years ago
- Small template string based date formatter for the browser and Node.js.☆15Updated 2 months ago
- A fast fifo implementation similar to the one powering nextTick in Node.js core☆61Updated last year
- Generate TypeScript runtime types from your database schema.☆37Updated last year
- Same as sqlite-tag but without the native sqlite3 module dependency☆19Updated 4 months ago
- JavaScript userland partial implementation of IEEE 754 Decimal128 decimal floating-point arithmetic☆22Updated 6 months ago
- Isomorphic fetch with window.fetch and undici.fetch☆20Updated 2 months ago
- Yet another auto expiring, max sizable cache☆50Updated last year
- Automatically federate a Mercurius GraphQL service☆33Updated 6 months ago
- A template for WebSockets powered Cloudflare Worker project using graphql-ws☆40Updated 3 years ago
- ☆18Updated 3 years ago
- A WIP module aimed at providing a safer, easier to use and beginner friendly crypto API for Node.js☆22Updated 2 years ago
- Dockerfile-js, render well-formed Dockerfiles.☆19Updated 2 years ago
- 💯 A better sync and async iterator API.☆58Updated 9 months ago
- Components for State Machines, using Generator Functions☆34Updated 2 years ago
- ☆53Updated 4 years ago
- JavaScript GRIP library☆47Updated 5 months ago
- ☆10Updated last year
- ☆53Updated 4 years ago
- NPR allows you to run your CLI dependencies without having a copy in node_modules.☆16Updated 2 years ago
- Create an AsyncIterableIterator from anything while handling back-pressure!☆24Updated this week
- load a plugin via a synchronous worker☆58Updated last year