Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization, with PyTorch/CUDA
☆42Feb 27, 2024Updated 2 years ago
Alternatives and similar repositories for minbpe-pytorch
Users that are interested in minbpe-pytorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Project exploring 3D volumetric rendering of NEXRAD radar data.☆13Oct 23, 2023Updated 2 years ago
- The AI that helps you achieve your goals☆11Feb 4, 2024Updated 2 years ago
- Simple (fast) transformer inference in PyTorch with torch.compile + lit-llama code☆11Aug 29, 2023Updated 3 years ago
- An analog touch screen joystick that pretends to be a bevy gamepad☆13Jul 13, 2024Updated 2 years ago
- Customizable charts made with TikZ and LaTeX3☆14Feb 11, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆12Oct 23, 2022Updated 3 years ago
- Flight Recorder allows to record client program execution and examine it later☆11Sep 18, 2020Updated 6 years ago
- ☆11Jun 8, 2023Updated 3 years ago
- A command line utility for doing polarization simulations☆17Aug 21, 2019Updated 7 years ago
- Benchmarking LLM Inference Speeds☆14Sep 2, 2026Updated 2 weeks ago
- Implementation of a multi-agent system for the modeling of carpooling in a city with one-way streets. Used Python and the Mesa package fo…☆14Jan 19, 2022Updated 4 years ago
- Bindings to Nvidia Labs's ꟻLIP image comparison and error visualization library☆23Updated this week
- Export userdata from your reddit accounts. Submissions, comments, saved, upvoted contents are supported.☆23Jul 8, 2026Updated 2 months ago
- Hypercorn is an ASGI and WSGI Server based on Hyper libraries and inspired by Gunicorn.☆19Jan 12, 2026Updated 8 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- GPT4 Tokenizer Visualizer☆23May 21, 2023Updated 3 years ago
- ☆29Jun 30, 2026Updated 2 months ago
- IonSolver is a magnetohydrodynamic simulation software featuring an extended Lattice Boltzmann method and GPU acceleration☆24May 12, 2026Updated 4 months ago
- This is a public repository for collecting excellent visualizations of knowledge and/or data.☆18Jan 6, 2021Updated 5 years ago
- ☆31Jul 17, 2023Updated 3 years ago
- ☆22Sep 9, 2021Updated 5 years ago
- gpt completions in vscode☆35Mar 24, 2023Updated 3 years ago
- Inference Llama 2 in one file of zero-dependency, zero-unsafe Rust☆41Aug 2, 2023Updated 3 years ago
- Python library for analyzing, exploring, and visualizing epitrochoids and hypotrochoids in just a few lines of code☆31Jul 7, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆35Sep 13, 2023Updated 3 years ago
- This contains Helpful PYKD (Python Extension for Windbg) scripts☆50Mar 9, 2015Updated 11 years ago
- Demo of fine-tuning QA models for answering FAQ of cloud providers documentation☆11Jun 20, 2026Updated 3 months ago
- An Easy Annotation Tool for Natural Language Processing☆12May 17, 2024Updated 2 years ago
- Export NetworkX graphs to TikZ directly☆31May 7, 2025Updated last year
- Formulaire en ligne qui génère une attestation de déplacement dérogatoire☆10Mar 18, 2020Updated 6 years ago
- ☆22Sep 11, 2023Updated 3 years ago
- Code and experiments for the COLING2020 paper "Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations".☆11Dec 9, 2020Updated 5 years ago
- ☆36Jan 13, 2022Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Real probability scales for matplotlib☆40Nov 3, 2023Updated 2 years ago
- ☆16Nov 20, 2023Updated 2 years ago
- BabelNet (and WordNet) sense embedding trained with Word2Vec and FastText☆10Sep 3, 2019Updated 7 years ago
- 3d Cellular Automata using WGPU in Rust (for the web and using compute shaders)☆37Mar 4, 2025Updated last year
- ☆41May 7, 2026Updated 4 months ago
- WProofreader software development kit (SDK) offers multilingual spelling & grammar check API and JavaScript libraries for rich text edito…☆13Sep 9, 2026Updated last week
- Rule-based Kurdish Transliterator☆11May 3, 2024Updated 2 years ago