Chinese tokens in tiktoken tokenizers.
☆33May 15, 2024Updated 2 years ago
Alternatives and similar repositories for chinese-tokens-in-tiktoken
Users that are interested in chinese-tokens-in-tiktoken are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- code for COLING paper "A Hybrid Model of Classification and Generation for Spatial Relation Extraction"☆10Oct 20, 2022Updated 3 years ago
- Efficient retrieval head analysis with triton flash attention that supports topK probability☆13Jun 15, 2024Updated 2 years ago
- Mini Model Daemon☆13Nov 9, 2024Updated last year
- Training a reward model for RLHF using RWKV.☆15Jun 5, 2023Updated 3 years ago
- Open Source Version of florr.io☆10Jan 16, 2023Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Direct Preference Optimization for RWKV, aiming for RWKV-5 and 6.☆11Mar 1, 2024Updated 2 years ago
- codes for "Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models"☆12Feb 10, 2025Updated last year
- ☆14Jun 14, 2022Updated 4 years ago
- 基于PaddleNLP的对话意图识别☆10Apr 11, 2023Updated 3 years ago
- RWKV Wiki website (archived, please visit official wiki)☆10Mar 26, 2023Updated 3 years ago
- A blazing fast CLIP gRPC service in rust.☆16Aug 9, 2023Updated 3 years ago
- Repository containing the website for the EMNLP 2023 conference☆17Feb 12, 2025Updated last year
- my own paper for history of math and corresponding agentic workflow☆17Apr 12, 2026Updated 4 months ago
- [ACL2023] Official code repository for VLN-Trans☆14Sep 10, 2023Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- A Hello World program in Coq.☆17May 28, 2022Updated 4 years ago
- Official repository for Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning☆12Sep 2, 2024Updated 2 years ago
- A simple Mustache parser/evaluator for Swift☆24Jul 17, 2026Updated last month
- Description for MV-MATH☆15Jul 20, 2025Updated last year
- 清华大学宿舍洗衣机空闲提醒 小程序☆14Feb 4, 2021Updated 5 years ago
- Unofficial implementation for the paper 'Improving Diffusion Models for Inverse Problems using Manifold Constraints'[https://arxiv.org/ab…☆12Aug 21, 2022Updated 4 years ago
- [arxiv: 2604.14142] From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Space☆17Apr 16, 2026Updated 4 months ago
- OpenLLMDE: An open source data engineering framework for LLMs☆18Sep 9, 2023Updated 2 years ago
- [ISSTA'24] A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing☆12Jan 7, 2025Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Improving Transformation Invariance in Contrastive Representation Learning☆12Mar 13, 2021Updated 5 years ago
- ☆10Aug 3, 2023Updated 3 years ago
- VPS inventory monitoring system☆11Jan 11, 2019Updated 7 years ago
- ☆15Sep 25, 2024Updated last year
- ☆19Dec 31, 2025Updated 8 months ago
- 持续追踪ChatGPT相关的技术资料和行业进展。☆12Apr 24, 2023Updated 3 years ago
- Some preliminary explorations of Mamba's context scaling.☆14Dec 18, 2024Updated last year
- 📦Package|GPT3.5 chat async client with support for NLP search context retention and real-time network support, as a pre-release replacem…☆12Apr 2, 2023Updated 3 years ago
- CCKS 2022 通用信息抽取☆13May 1, 2022Updated 4 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Roberta Question Answering using MLX.☆24Feb 22, 2026Updated 6 months ago
- A python command-line tool to download & manage MLX AI models from Hugging Face.☆20Aug 26, 2024Updated 2 years ago
- hybrid representation learning approach for fully automatic and template-free vessel centerline extraction☆12Jul 17, 2023Updated 3 years ago
- AgentsCourt: Building Judicial Decision-Making Agents with Court Debate Simulation and Legal Knowledge Augmentation (EMNLP 2024 Findings)☆19Dec 30, 2024Updated last year
- [ISSTA 2025] A Large-scale Empirical Study on Fine-tuning Large Language Models for Unit Testing☆13Feb 9, 2025Updated last year
- RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment☆18Dec 19, 2024Updated last year
- ☆28Feb 26, 2026Updated 6 months ago