Fast bare-bones BPE for modern tokenizer training
☆180Jun 23, 2025Updated last year
Alternatives and similar repositories for bpeasy
Users that are interested in bpeasy are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆11Sep 30, 2015Updated 11 years ago
- Simple Byte pair Encoding mechanism used for tokenization process . written purely in C☆156Nov 11, 2024Updated last year
- ####### ALERT! #########: my fork of the project has moved:☆19Dec 23, 2016Updated 9 years ago
- Google+ Blog☆17Oct 9, 2011Updated 15 years ago
- Pure Python version of the mlabwrap Python to Matlab bridge☆33Nov 21, 2019Updated 6 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- The official PyTorch implementation of Google's Gemma models☆5,710May 30, 2025Updated last year
- UNet diffusion model in pure CUDA☆669Jun 28, 2024Updated 2 years ago
- Teardown of Google Glass☆42Jan 11, 2014Updated 12 years ago
- A benchmark to evaluate language models on questions I've previously asked them to solve.☆1,069Apr 27, 2025Updated last year
- A basic implementation of convolutional neural nets☆60Apr 20, 2014Updated 12 years ago
- Implementation of Diffusion Transformer (DiT) in JAX☆322Jun 11, 2024Updated 2 years ago
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆605Aug 14, 2026Updated 2 months ago
- Artificial Life simulator using canvas. Based on https://github.com/karpathy/scriptsbots☆80Jun 7, 2016Updated 10 years ago
- A wrapper for LibSVM that lets you train SVM's directly on Eigen library matrices in C++☆112Mar 29, 2016Updated 10 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- ScriptBots is an Open Source Evolutionary Artificial Life Simulation of Predator-Prey dynamics, written by Andrej Karpathy.☆67Feb 18, 2011Updated 15 years ago
- gpt-2 from scratch in mlx☆438Jun 12, 2024Updated 2 years ago
- Benchmark testbed for assessing the performance of optimisation algorithms☆87Jan 7, 2015Updated 11 years ago
- Simple MPI implementation for prototyping or learning☆332Aug 6, 2025Updated last year
- Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"☆22Feb 14, 2024Updated 2 years ago
- useful scripts to work with Twitter + Python. Requires the tweepy library.☆93Nov 29, 2012Updated 13 years ago
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆979Nov 16, 2025Updated 11 months ago
- A python script to help manage a Gmail inbox by filtering out promotional emails using GPT-3 or GPT-4.☆467Dec 2, 2023Updated 2 years ago
- Game making library for using Canvas element☆116Oct 17, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Fine-tune mistral-7B on 3090s, a100s, h100s☆737Oct 11, 2023Updated 3 years ago
- Implements SFO minibatch optimizer in Python and MATLAB, and reproduces figures from paper.☆138May 17, 2021Updated 5 years ago
- All stories and comments posted on Hacker News upto May 29, 2014☆136Oct 30, 2018Updated 8 years ago
- FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.☆362Nov 2, 2025Updated 11 months ago
- My hybrid TTS network that combines, VALL-E, VoiceBox, SpeechFlow, Seamless and TortoiseTTS into one☆26Aug 5, 2024Updated 2 years ago
- Schedule-Free Optimization in PyTorch☆2,330Jul 28, 2026Updated 3 months ago
- Cramming the training of a (BERT-type) language model into limited compute.☆1,369Jun 13, 2024Updated 2 years ago
- GPT for FACodec☆13Mar 25, 2024Updated 2 years ago
- Artificial Life simulator using canvas. Based on https://github.com/karpathy/scriptsbots☆123May 27, 2012Updated 14 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Code for SaGe subword tokenizer (EACL 2023)☆28Nov 30, 2024Updated last year
- A collection of Pythonic subword tokenisers and text preprocessing tools.☆15Oct 17, 2026Updated last week
- Train to 94% on CIFAR-10 in <6.3 seconds on a single A100. Or ~95.79% in ~110 seconds (or less!)☆1,317Dec 18, 2024Updated last year
- Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization [ACL 2026]☆20Apr 18, 2026Updated 6 months ago
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…☆15Oct 16, 2023Updated 3 years ago
- Javascript neural networks on top of general scalar/tensor reverse-mode automatic differentiation.☆240Oct 28, 2017Updated 9 years ago
- ☆137Apr 23, 2016Updated 10 years ago