oramasearch / minbpeLinks
Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization. Pure JavaScript port of karpathy/minbpe
☆18Updated last year
Alternatives and similar repositories for minbpe
Users that are interested in minbpe are comparing it to the libraries listed below
Sorting:
- Find and erase strings and comments in JavaScript code☆14Updated 4 years ago
- Private symbols for node.js☆14Updated 4 years ago
- Type-safe SQL builder using ES2015 template tags☆11Updated 3 years ago
- Enables `attributechanged` element's listener.☆15Updated 5 years ago
- Peek a stream!☆13Updated 2 years ago
- Determines whether a Node file is a Module (`import`) or a Script (`require`)☆11Updated 4 years ago
- Running Node within Node (a fork of synchronous-worker)☆27Updated 2 years ago
- Split large texts into chunks with a maximum number of tokens. Split by fixed size or by sentence.☆31Updated last year
- An SSR, islands friendly, way to render ube components.☆15Updated 3 years ago
- Fast Map built for keys that are always fixed size uniformly distributed buffers.☆37Updated 3 years ago
- Tiny internal DSL which allows invocation of different functionality depending on SemVer match.☆11Updated 7 years ago
- ☆23Updated 5 years ago
- Generate TypeScript runtime types from your database schema.☆37Updated 11 months ago
- NPR allows you to run your CLI dependencies without having a copy in node_modules.☆16Updated 2 years ago
- Opentelemetry instrumentation for Prisma ORM☆11Updated 3 years ago
- Yet another auto expiring, max sizable cache☆50Updated last year
- Simple JavaScript HTML parser.☆31Updated 10 years ago
- ☆10Updated last year
- Small template string based date formatter for the browser and Node.js.☆14Updated last month
- Statechart library for diagrams.net (draw.io)☆24Updated 3 years ago
- A set helpers for writing dynamic SQL queries with `pg-sql` in Javascript.☆38Updated 3 years ago
- JSX-IR to Incremental DOM renderer☆14Updated 9 years ago
- Promise Semaphore☆26Updated 3 years ago
- 📋 A polyfill for URLPattern for web-browsers, older versions of deno and non-unstable deno☆12Updated 3 years ago
- A slim, easy-to-use wrapper around EventSource☆30Updated 4 years ago
- Working group notes for graphql-js☆27Updated this week
- Speed up zlib operations by running them using the sync APIs but in a Worker☆29Updated 4 years ago
- ☆26Updated 3 years ago
- Same as sqlite-tag but without the native sqlite3 module dependency☆19Updated 4 months ago
- An essential connected/disconnected DOM helper☆19Updated 4 years ago