A pure-rust port of tiktoken.
☆17Nov 9, 2024Updated last year
Alternatives and similar repositories for tiktoken-rs
Users that are interested in tiktoken-rs are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Provides extensions for the zip crate☆26Oct 12, 2026Updated last week
- Ready-made tokenizer library for working with GPT and tiktoken☆408Sep 24, 2026Updated 3 weeks ago
- ☆10Oct 2, 2021Updated 5 years ago
- tutorial for training a BPE tokenizer on multiple GPUs☆13May 5, 2025Updated last year
- Wanted to integrate Julius AI in my workflows but couldn't hence this.☆14Jan 1, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- The code for "VISTA: Enhancing Long-Duration and High-Resolution Video Understanding by VIdeo SpatioTemporal Augmentation" [CVPR2025]☆20Feb 27, 2025Updated last year
- [ACL'25] Official code of curiosity-driven RLHF☆16Jun 22, 2025Updated last year
- Simple space game shooter with Phaser 3.x and Matter.js physics with Lance-gg multiplayer client and server build.☆10Apr 25, 2018Updated 8 years ago
- Experimental python optimistic rollup fraud-proof generation☆29Aug 30, 2022Updated 4 years ago
- Rust bindings for CTranslate2☆14Jun 21, 2023Updated 3 years ago
- getting sponsored made easy with widgets !☆17Jul 30, 2022Updated 4 years ago
- Next-JS interface for 🤖 Open-AI based 🕷 spider-man conversation simulator ⚡️☆19Jan 28, 2023Updated 3 years ago
- Play 2.5 silhouette 4.0 seed with Slick 3.1 and PostgreSQL☆12Jun 20, 2017Updated 9 years ago
- ☆28Nov 5, 2022Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Converts an image into funny, smaller amongus characters☆16Apr 30, 2022Updated 4 years ago
- Transformers components but in Triton☆34May 9, 2025Updated last year
- Small extensions of the Bellman-Ford routines in NetworkX, primarily for convenience☆13May 7, 2018Updated 8 years ago
- Example project using GitHub, OpenAI and Defer☆19Mar 8, 2023Updated 3 years ago
- ☆12May 20, 2025Updated last year
- ☆18Dec 8, 2022Updated 3 years ago
- A service for launching containerized processes on cloud infrastructure.☆12Nov 16, 2023Updated 2 years ago
- ☆12Mar 26, 2024Updated 2 years ago
- Counterfactual Evaluation and Learning for Interactive Systems: Foundations, Implementations, and Recent Advances☆12Aug 14, 2022Updated 4 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Rust binding for the sentencepiece library☆24Jul 26, 2026Updated 2 months ago
- A pathway and collection of resources to learning Jax from beginning to advance.☆11Jan 2, 2021Updated 5 years ago
- code for the guestbook blog☆19Jan 15, 2023Updated 3 years ago
- Learning from Graphs: From Mathematical Principles to Practical Tools☆11Apr 16, 2021Updated 5 years ago
- Load environment variables from files and directories using fish shell, like envdir☆26May 18, 2022Updated 4 years ago
- Source code for paper on commonsense reasoning for 2020 Annual Conference of the Association for Computational Linguistics (ACL) 2020.☆29Aug 2, 2024Updated 2 years ago
- Official PyTorch Implementation for Vision-Language Models Create Cross-Modal Task Representations, ICML 2025☆35May 1, 2025Updated last year
- Proof of concept lambda for massive parallelism☆10Nov 2, 2018Updated 7 years ago
- Phaser asset pack generator for the phaser-tiled plugin.☆13Nov 13, 2014Updated 11 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆21Aug 3, 2024Updated 2 years ago
- TSID generator for rust☆13Jun 23, 2026Updated 4 months ago
- Learning the basic fundamentals of high level programming with Python and JavaScript☆11Mar 10, 2023Updated 3 years ago
- Code examples for "Under the hood of calling C/C++ from Python"☆13Sep 16, 2020Updated 6 years ago
- Scala Driver for ArangoDB☆19Apr 27, 2024Updated 2 years ago
- ☆17Jan 13, 2024Updated 2 years ago
- Tutorials for FLAVA model https://arxiv.org/abs/2112.04482☆12Jun 22, 2022Updated 4 years ago