FateScript / token_visualizerLinks
Token level visualization tools for large language models
☆88Updated 7 months ago
Alternatives and similar repositories for token_visualizer
Users that are interested in token_visualizer are comparing it to the libraries listed below
Sorting:
- [ACL 2024 Demo] Official GitHub repo for UltraEval: An open source framework for evaluating foundation models.☆247Updated 10 months ago
- [ICML 2025] Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale☆259Updated last month
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆136Updated last year
- ☆161Updated 4 months ago
- Feeling confused about super alignment? Here is a reading list☆43Updated last year
- ☆109Updated last year
- A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.☆209Updated last month
- Fantastic Data Engineering for Large Language Models☆90Updated 8 months ago
- ☆125Updated last year
- ☆65Updated 9 months ago
- ☆74Updated 2 weeks ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆42Updated last year
- ☆50Updated last year
- ACL 2024 | LooGLE: Long Context Evaluation for Long-Context Language Models☆185Updated 10 months ago
- Scaling Preference Data Curation via Human-AI Synergy☆100Updated 2 months ago
- xVerify: Efficient Answer Verifier for Reasoning Model Evaluations☆128Updated 4 months ago
- Counting-Stars (★)☆83Updated 2 months ago
- [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)☆166Updated 6 months ago
- ☆71Updated 7 months ago
- [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs☆253Updated 8 months ago
- Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning☆190Updated 5 months ago
- Pretrain、decay、SFT a CodeLLM from scratch 🧙♂️☆37Updated last year
- [ACL 2025] An official pytorch implement of the paper: Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinement☆34Updated 3 months ago
- SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper☆33Updated last year
- On Memorization of Large Language Models in Logical Reasoning☆71Updated 5 months ago
- We aim to provide the best references to search, select, and synthesize high-quality and large-quantity data for post-training your LLMs.☆57Updated 10 months ago
- A Comprehensive Survey on Long Context Language Modeling☆180Updated last month
- [ACL-25] We introduce ScaleQuest, a scalable, novel and cost-effective data synthesis method to unleash the reasoning capability of LLMs.☆64Updated 10 months ago
- Official completion of “Training on the Benchmark Is Not All You Need”.☆35Updated 8 months ago
- [ICML 2025] |TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation☆113Updated 3 months ago