FateScript / token_visualizerLinks
Token level visualization tools for large language models
☆88Updated 8 months ago
Alternatives and similar repositories for token_visualizer
Users that are interested in token_visualizer are comparing it to the libraries listed below
Sorting:
- [ACL 2024 Demo] Official GitHub repo for UltraEval: An open source framework for evaluating foundation models.☆248Updated 10 months ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆137Updated last year
- Scaling Preference Data Curation via Human-AI Synergy☆107Updated 2 months ago
- [ICML 2025] Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale☆263Updated 2 months ago
- ☆165Updated 4 months ago
- Fantastic Data Engineering for Large Language Models☆91Updated 8 months ago
- xVerify: Efficient Answer Verifier for Reasoning Model Evaluations☆130Updated 5 months ago
- [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)☆168Updated 7 months ago
- Counting-Stars (★)☆83Updated 3 months ago
- ☆147Updated last year
- Reformatted Alignment☆113Updated last year
- Feeling confused about super alignment? Here is a reading list☆43Updated last year
- ☆114Updated last year
- On Memorization of Large Language Models in Logical Reasoning☆72Updated 5 months ago
- [ACL 2025] An official pytorch implement of the paper: Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement☆35Updated 3 months ago
- A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.☆214Updated last month
- ☆65Updated 9 months ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆42Updated last year
- This is the repo for our paper "Mr-Ben: A Comprehensive Meta-Reasoning Benchmark for Large Language Models"☆50Updated 10 months ago
- Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning☆190Updated 6 months ago
- Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning☆85Updated last year
- ☆125Updated last year
- Official completion of “Training on the Benchmark Is Not All You Need”.☆35Updated 8 months ago
- We aim to provide the best references to search, select, and synthesize high-quality and large-quantity data for post-training your LLMs.☆59Updated 11 months ago
- WideSearch: Benchmarking Agentic Broad Info-Seeking☆93Updated last month
- [ICML 2025] |TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation☆113Updated 4 months ago
- ☆49Updated last year
- [ICML 2024] Selecting High-Quality Data for Training Language Models☆187Updated last year
- Official github repo for AutoDetect, an automated weakness detection framework for LLMs.☆42Updated last year
- InsTag: A Tool for Data Analysis in LLM Supervised Fine-tuning☆273Updated 2 years ago