Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization. Pure JavaScript port of karpathy/minbpe
☆18Feb 19, 2024Updated 2 years ago
Alternatives and similar repositories for minbpe
Users that are interested in minbpe are comparing it to the libraries listed below
Sorting:
- Split large texts into chunks with a maximum number of tokens. Split by fixed size or by sentence.☆34Mar 1, 2024Updated 2 years ago
- A set of UI components to help you integrate Orama on your website or app.☆17Jun 19, 2025Updated 8 months ago
- 🚒✨ Rescue: better errors through types (a more type directed MonadThrow/MonadCatch)☆20Feb 1, 2022Updated 4 years ago
- NAMM Standards☆10Dec 7, 2021Updated 4 years ago
- Promisify a callback-style function, resolving with [err,val]☆31May 19, 2021Updated 4 years ago
- Fast, StAX-like XML Parser for BEAM Languages☆36Jul 31, 2022Updated 3 years ago
- A collection of additional language phonology settings for use with VulgarLang.☆13Aug 19, 2022Updated 3 years ago
- Repository for go shared libraries (for now).☆11Dec 1, 2025Updated 3 months ago
- A small utility for creating warnings and emitting them☆38Dec 2, 2025Updated 3 months ago
- ☆84Apr 23, 2025Updated 10 months ago
- Fast Map built for keys that are always fixed size uniformly distributed buffers.☆37Mar 21, 2022Updated 3 years ago
- NearForm multi-platform application accelerator☆35Jun 30, 2021Updated 4 years ago
- Lelylan API proxy☆13Dec 12, 2016Updated 9 years ago
- ☆11Oct 2, 2025Updated 5 months ago
- an animated Mayan Calendar☆12Jan 12, 2014Updated 12 years ago
- ☆12Feb 9, 2020Updated 6 years ago
- A microservice to encapsulate our inventory management functionality☆13Oct 25, 2023Updated 2 years ago
- ☆16Dec 2, 2011Updated 14 years ago
- QUIC pluggable crypto to use the protocol as plaintext (for use when cryptography is already handled at another layer, e.g. Wireguard)☆10Aug 27, 2025Updated 6 months ago
- ☆43Mar 6, 2023Updated 2 years ago
- Build Elixir applications with RabbitMQ☆42Dec 13, 2024Updated last year
- Node 18's node:test, as an npm package☆98Dec 21, 2024Updated last year
- DNS-SD (aka Zeroconf, Bonjour, Avahi) in TypeScript for Deno and Node.☆56Jul 14, 2024Updated last year
- Image server for Plug applications for streaming image transformations☆12Dec 7, 2024Updated last year
- A Golang application that demonstrates how to monitor a Golang service using Prometheus and Grafana. This is for Docker's official Deno L…☆15Mar 22, 2025Updated 11 months ago
- The last-write-wins register CRDT☆17Nov 10, 2024Updated last year
- A set of machine learning experiments in Clojure☆30Nov 30, 2012Updated 13 years ago
- A Serverless Scheduler and Queue system built on top of cloudflare workers, D1 and Durable Objects to handle scale and schedule/queue mil…☆11Dec 15, 2024Updated last year
- Graceful HTTP server shutdown in Go☆17Jun 16, 2025Updated 8 months ago
- ☆12Nov 17, 2024Updated last year
- Specification for efficient, compact binary format for serializing data structures with extensible typing☆13Jun 4, 2019Updated 6 years ago
- Building a Go REST API using Gorm and Fiber!☆12Apr 30, 2022Updated 3 years ago
- Bidirectional JavaScript <-> ESI converter. Write javascript code that will be converted to valid ESI (Edge Side Includes), capable of r…☆16May 21, 2018Updated 7 years ago
- Some examples regarding JavaScript powerful iteration protocols enabling `for..of` and `for await...of` syntax☆10May 4, 2021Updated 4 years ago
- WebSocket server implementation of OCaml☆14May 14, 2019Updated 6 years ago
- realistic mouse movement created by a generative AI trained on human mouse data to bypass web-based bot detection and highlight security …☆10Jun 30, 2023Updated 2 years ago
- Toy distributed PostgreSQL by implementing SQL over KV☆11Jan 14, 2026Updated last month
- Go tool to connect to websocket for sending and receiving messages and load testing.☆10Dec 20, 2025Updated 2 months ago
- AWS Route53 provider implementation for libdns☆16Nov 21, 2025Updated 3 months ago