This repository provides a clear, educational implementation of Byte Pair Encoding (BPE) tokenization in plain Python. The focus is on algorithmic understanding, not raw performance.
☆21Aug 28, 2024Updated 2 years ago
Alternatives and similar repositories for efficient_bpe_explanation
Users that are interested in efficient_bpe_explanation are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- (1) Courses on key machine learning and software engineering topics; (2) and a workflow for creating similar courses with LLMs in Cursor.…☆13Jun 9, 2025Updated last year
- Accelerating core decomposition in billion-scale hypergraphs☆11Mar 3, 2025Updated last year
- VS Code extension that adds syntax highlighting for ssh config files☆13Nov 17, 2025Updated 11 months ago
- Python bindings for the PCRE2 library created by Philip Hazel☆19Jul 26, 2026Updated 3 months ago
- Lucene open-domain QA retrieval in python☆11Feb 18, 2021Updated 5 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- An easy-to-use Python library for lightning-fast persistent function caching.☆18May 3, 2025Updated last year
- ☆10Sep 29, 2024Updated 2 years ago
- A Javascript implementation of Pytorch with WebGPU.☆14Apr 20, 2023Updated 3 years ago
- A Suite of Benchmark Programs for C-based High-Level Synthesis☆11Jun 9, 2025Updated last year
- Acoustic-prosodic entrainment measurement in spoken dialogue and approximation of the evolution of a speaker’s a/p features.☆14Feb 26, 2024Updated 2 years ago
- I modified some code of K-BERT so that it can be fit to English datasets Topics Resources☆11Dec 15, 2022Updated 3 years ago
- text2markdown is a Python library for intelligently converting plain text into Markdown.☆19Jun 1, 2026Updated 5 months ago
- Code for Detecting language from text in python using fasttext☆13May 25, 2020Updated 6 years ago
- KL3M training data collection and preprocessing☆22Sep 29, 2026Updated last month
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [NeurIPS '25] Code for Paper "IF-Guide: Influence Function-Guided Suppression of Harmful Training Data for Reducing LLM Toxicity"☆15Oct 22, 2025Updated last year
- Source Code for "CECI: Compact Embedding Cluster Index for Scalable Subgraph Matching"☆10Apr 15, 2020Updated 6 years ago
- Zhuofan Lin and Shiju Lin, "GPU-Accelerated Efficient Transduction for Logic Optimization", Design, Automation & Test in Europe Conferenc…☆14Sep 19, 2026Updated last month
- Specialization of BERT architecture both for the Spanish language and the Twitter domain☆13Nov 6, 2020Updated 6 years ago
- collection of basic stencil benchmarks to profile multiple computing architectures☆12Nov 13, 2024Updated last year
- A web interface to understand language-specific BERT-models☆18Apr 16, 2024Updated 2 years ago
- Simple library for manipulating strings using OpenFST☆12Sep 26, 2021Updated 5 years ago
- ☆14Jun 25, 2025Updated last year
- AUSTENDER OCDS Search API. This portal will provide users of AusTender data with documentation, code examples, bug notifications and feat…☆24Feb 12, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- We have developed our own PyTorch execution engine with GPU acceleration using WebGPU.☆15Jan 15, 2026Updated 9 months ago
- HLS-based framework to accelerate the implementation of 2-D DP kernels on FPGA☆15Mar 26, 2026Updated 7 months ago
- Official repository of the Hate Speech Detection Tasks at Evalita☆12Dec 16, 2020Updated 5 years ago
- ☆14Oct 25, 2024Updated 2 years ago
- Provincias y departamentos de Argentina renderizados con Leafletjs.com☆17Feb 23, 2019Updated 7 years ago
- The Spezi FHIR Standard & Related Modules☆15Apr 20, 2026Updated 6 months ago
- Source code of "FlowWalker: A Memory-efficient and High-performance GPU-based Dynamic Graph Random Walk Framework"☆11Oct 23, 2024Updated 2 years ago
- ☆10Mar 5, 2025Updated last year
- ☆13May 9, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A Neural Model for User Geolocation and Lexical Dialectology☆16Nov 11, 2018Updated 7 years ago
- QuranX website☆12Sep 1, 2026Updated 2 months ago
- ☆12Sep 24, 2021Updated 5 years ago
- GPU based Compressed Graph Traversal☆16Jan 9, 2026Updated 9 months ago
- Reimplentation of paper using gzip + knn for text classification☆19Aug 1, 2023Updated 3 years ago
- Replication of "Regularizing and Optimizing LSTM Language Models" by Merity et al. (2017).☆12Sep 17, 2019Updated 7 years ago
- A tool to score websites for AI design patterns.☆19Jul 7, 2026Updated 3 months ago