Chinese tokens in tiktoken tokenizers.
☆33May 15, 2024Updated 2 years ago
Alternatives and similar repositories for chinese-tokens-in-tiktoken
Users that are interested in chinese-tokens-in-tiktoken are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆11Apr 21, 2026Updated 2 months ago
- Source code of the double-fetch bug detection approach based on Coccinelle engine☆10Jan 2, 2017Updated 9 years ago
- Repository for the syscall document.☆12Dec 28, 2017Updated 8 years ago
- AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation (EMNLP 2024 Findings)☆17Dec 30, 2024Updated last year
- ☆21May 14, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment☆18Dec 19, 2024Updated last year
- [NeurIPS25] RULE: Reinforcement UnLEarning Achieves Forge-retain Pareto Optimality☆20Oct 22, 2025Updated 8 months ago
- codes for "Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models"☆12Feb 10, 2025Updated last year
- [ICML'25] MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents☆34Jul 31, 2025Updated 11 months ago
- Repository of paper "Establishing Trustworthy LLM Evaluation via Shortcut Neuron Analysis" (ACL 2025 Main)☆19Jul 19, 2025Updated 11 months ago
- ☆12Dec 14, 2024Updated last year
- Library emulating Android APIs over GWT☆21Oct 5, 2022Updated 3 years ago
- [CVPR2025] Official implementation of RAM☆29Nov 4, 2025Updated 8 months ago
- A static analysis tool for Java programs, based on the theory of code property graphs.☆20Mar 27, 2023Updated 3 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- ☆31Dec 8, 2025Updated 7 months ago
- Make a demo using chrome built-in Gemini Nano.☆20Jun 23, 2024Updated 2 years ago
- ☆16Dec 21, 2023Updated 2 years ago
- ☆10Jun 28, 2022Updated 4 years ago
- 清华大学学生健康和出行情况报告每日自动提交☆14Jan 30, 2021Updated 5 years ago
- ☆16Mar 20, 2025Updated last year
- A blazing fast CLIP gRPC service in rust.☆16Aug 9, 2023Updated 2 years ago
- Repository containing the website for the EMNLP 2023 conference☆17Feb 12, 2025Updated last year
- ICML 2025 Spotlight, PCEvolve: Private Contrastive Evolution for Synthetic Dataset Generation via Few-Shot Private Data and Generative AP…☆15Jun 27, 2025Updated last year
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- 基于PaddleNLP的对话意图识别☆10Apr 11, 2023Updated 3 years ago
- MAGIC: Microlensing Analysis Guided by Intelligent Computation. A PyTorch framework for automatic analysis of realistic microlensing ligh…☆13May 30, 2024Updated 2 years ago
- MP3音频编码器 / MP3 Audio Encoder☆16Jun 12, 2025Updated last year
- Unofficial implementation for the paper 'Improving Diffusion Models for Inverse Problems using Manifold Constraints'[https://arxiv.org/ab…☆12Aug 21, 2022Updated 3 years ago
- ☆11Jun 18, 2020Updated 6 years ago
- 清华大学宿舍洗衣机空闲提醒小程序☆14Feb 4, 2021Updated 5 years ago
- ☆47Dec 16, 2025Updated 6 months ago
- [ICLR 2025 SSI-FM] Self-Taught Self-Correction for Small Language Models☆11Sep 19, 2025Updated 9 months ago
- Improving Transformation Invariance in Contrastive Representation Learning☆12Mar 13, 2021Updated 5 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- This project demonstrates the computation process of the RWKV (Receptance Weighted Key Value) model through Excel spreadsheets.☆21Jun 7, 2025Updated last year
- survery of small language models☆18Jul 23, 2024Updated last year
- Some test samples for CPG execution logic.☆21Apr 13, 2024Updated 2 years ago
- OpenLLMDE: An open source data engineering framework for LLMs☆18Sep 9, 2023Updated 2 years ago
- Generates and optimizes Haiku system and user prompts for classification☆15Oct 27, 2025Updated 8 months ago
- WebGL Video Chat - a video chat in WebGL with WebRTC☆38Mar 21, 2013Updated 13 years ago
- Pocket Flow: A minimalist LLM framework. Let Agents build Agents!☆32Apr 11, 2025Updated last year