This repository provides a clear, educational implementation of Byte Pair Encoding (BPE) tokenization in plain Python. The focus is on algorithmic understanding, not raw performance.
☆20Aug 28, 2024Updated last year
Alternatives and similar repositories for efficient_bpe_explanation
Users that are interested in efficient_bpe_explanation are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LLM pretraining in pure JAX☆16Apr 1, 2026Updated 4 months ago
- Align your LM to express calibrated verbal statements of confidence in its long-form generations.☆30Jun 4, 2024Updated 2 years ago
- ☆12Nov 25, 2018Updated 7 years ago
- Material de apoyo de la Materia Análisis Predictivo de la Licenciatura en Analítica (Data Science) ITBA☆20Aug 5, 2026Updated last week
- Lucene open-domain QA retrieval in python☆11Feb 18, 2021Updated 5 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Acoustic-prosodic entrainment measurement in spoken dialogue and approximation of the evolution of a speaker’s a/p features.☆14Feb 26, 2024Updated 2 years ago
- ☆14Apr 24, 2024Updated 2 years ago
- I modified some code of K-BERT so that it can be fit to English datasets Topics Resources☆11Dec 15, 2022Updated 3 years ago
- LaTeX snippets for Sublime Text.☆10Sep 20, 2017Updated 8 years ago
- Specialization of BERT architecture both for the Spanish language and the Twitter domain☆13Nov 6, 2020Updated 5 years ago
- Simple library for manipulating strings using OpenFST☆12Sep 26, 2021Updated 4 years ago
- ☆10Mar 2, 2024Updated 2 years ago
- ☆11Feb 27, 2023Updated 3 years ago
- 🏆 The winner code for Neurips'23 BigANN Competition OOD and Sparse track.☆15Jun 17, 2025Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Traffic sign annotations for OS:Car dataset☆11Jul 22, 2019Updated 7 years ago
- Use Generative Adversarial Networks to generate new Minecraft terrains.☆12Oct 19, 2021Updated 4 years ago
- A repo to do interpretability of pre-trained acoustic models☆15Oct 15, 2023Updated 2 years ago
- ☆13Oct 25, 2024Updated last year
- GenDP: A Dynamic Programming Framework for Genome Sequencing Analysis☆17Jan 12, 2024Updated 2 years ago
- Source code of "FlowWalker: A Memory-efficient and High-performance GPU-based Dynamic Graph Random Walk Framework"☆11Oct 23, 2024Updated last year
- Python module providing a Cython implementation of the classic Louvain algorithm for graph clustering☆15Aug 8, 2019Updated 7 years ago
- ☆12Sep 11, 2020Updated 5 years ago
- ☆14Jan 12, 2022Updated 4 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Artifacts for SOSP'19 paper Optimizing Deep Learning Computation with Automatic Generation of Graph Substitutions☆21Apr 15, 2022Updated 4 years ago
- ☆18Mar 23, 2023Updated 3 years ago
- Spack package repository maintained by Student Cluster Competition Team @ Sun Yat-sen University.☆16Aug 20, 2025Updated 11 months ago
- A Parallel Russian-Simple Russian Dataset☆19Mar 30, 2023Updated 3 years ago
- CUDA-DClust+: Fast DBSCAN algorithm implemented on CUDA. Based on the research paper.☆20May 9, 2025Updated last year
- ☆20Jan 17, 2024Updated 2 years ago
- GPU TopK Benchmark☆18Dec 19, 2024Updated last year
- ☆19Nov 21, 2022Updated 3 years ago
- Python-based tools for exploring the block-lattice used by the Nano cryptocurrency☆21Mar 24, 2018Updated 8 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- [ICML 2025] CoreMatching: Co-adaptive Sparse Inference Framework for Comprehensive Acceleration of Vision Language Model☆16May 27, 2025Updated last year
- Large matrix multiplication in CUDA☆17Oct 20, 2023Updated 2 years ago
- Doom RPG Reverse Engineering Project☆17Sep 28, 2023Updated 2 years ago
- ☆21Oct 19, 2022Updated 3 years ago
- A Python Bloomfilter☆22Dec 24, 2023Updated 2 years ago
- ☆17Jan 14, 2025Updated last year
- This repository contains research into the use of graph partitioning algorithms for assigning people shelters based on their social netwo…☆18Apr 30, 2019Updated 7 years ago