Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization, with PyTorch/CUDA
β43Feb 27, 2024Updated 2 years ago
Alternatives and similar repositories for minbpe-pytorch
Users that are interested in minbpe-pytorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Project exploring 3D volumetric rendering of NEXRAD radar data.β13Oct 23, 2023Updated 3 years ago
- Creative diagrams in TiπZβ13Mar 25, 2021Updated 5 years ago
- Quantifying degeneracy in toy modelsβ11Sep 6, 2024Updated 2 years ago
- Automated terminal emulator benchmarksβ23Updated this week
- An analog touch screen joystick that pretends to be a bevy gamepadβ13Jul 13, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off β’ AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- A command line utility for doing polarization simulationsβ17Aug 21, 2019Updated 7 years ago
- β11Jun 8, 2023Updated 3 years ago
- Implementation of a multi-agent system for the modeling of carpooling in a city with one-way streets. Used Python and the Mesa package foβ¦β14Jan 19, 2022Updated 4 years ago
- Ability is a browser extension that helps people with varying degrees of ability have more control over their browsing experience.β23Sep 24, 2025Updated last year
- Bindings to Nvidia Labs's κ»LIP image comparison and error visualization libraryβ23Updated this week
- A visionOS project that demonstrates how to scale a volume to account for Window Zoom changesβ18Apr 3, 2024Updated 2 years ago
- Hypercorn is an ASGI and WSGI Server based on Hyper libraries and inspired by Gunicorn.β21Jan 12, 2026Updated 9 months ago
- GPT4 Tokenizer Visualizerβ23May 21, 2023Updated 3 years ago
- The missing Xcode Debugger Tools for RealityKitβ11Feb 5, 2022Updated 4 years ago
- GPU virtual machines on DigitalOcean Gradient AI β’ AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A CommonJS port of rtomayko's ronnβ13Oct 12, 2011Updated 15 years ago
- Tephigram plotting in Pythonβ22Oct 5, 2026Updated 3 weeks ago
- gpt completions in vscodeβ35Mar 24, 2023Updated 3 years ago
- β22Sep 9, 2021Updated 5 years ago
- β32Jul 17, 2023Updated 3 years ago
- Windbg Utility Tools based upon PyKDβ41Sep 9, 2020Updated 6 years ago
- Python library for analyzing, exploring, and visualizing epitrochoids and hypotrochoids in just a few lines of codeβ31Jul 7, 2023Updated 3 years ago
- Inference Llama 2 in one file of zero-dependency, zero-unsafe Rustβ42Aug 2, 2023Updated 3 years ago
- Code and experiments for the COLING2020 paper "Conception: Multilingually-Enhanced, Human-Readable Concept Vector Representations".β11Dec 9, 2020Updated 5 years ago
- Deploy on Railway without the complexity - Free Credits Offer β’ AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- π Interactive decision boundary visualizerβ10Sep 4, 2018Updated 8 years ago
- β35Sep 13, 2023Updated 3 years ago
- BabelNet (and WordNet) sense embedding trained with Word2Vec and FastTextβ10Sep 3, 2019Updated 7 years ago
- This contains Helpful PYKD (Python Extension for Windbg) scriptsβ50Mar 9, 2015Updated 11 years ago
- 3d Cellular Automata using WGPU in Rust (for the web and using compute shaders)β37Mar 4, 2025Updated last year
- Demo of fine-tuning QA models for answering FAQ of cloud providers documentationβ11Jun 20, 2026Updated 4 months ago
- Micro-framework for publishing linked dataβ11Aug 1, 2017Updated 9 years ago
- Real probability scales for matplotlibβ41Oct 24, 2026Updated last week
- β11Jul 14, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer β’ AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Analyse (group)chat messages. Currently supports: Facebook Messenger. Planned: Signal, Discord, WhatsAppβ42Jul 18, 2022Updated 4 years ago
- Export NetworkX graphs to TikZ directlyβ31May 7, 2025Updated last year
- β39Jan 13, 2022Updated 4 years ago
- An Easy Annotation Tool for Natural Language Processingβ12May 17, 2024Updated 2 years ago
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.β28Apr 21, 2023Updated 3 years ago
- β19Jun 19, 2025Updated last year
- Agent Based Fire Evacuation Model built using Project Mesaβ41Mar 6, 2023Updated 3 years ago