secsilm / chinese-tokens-in-tiktoken

Related projects: ⓘ

• crazycth / WizardLearner
  Pretrain、decay、SFT a CodeLLM from scratch 🧙‍♂️
  ☆30Updated 4 months ago
• open-compass / GAOKAO-Eval
  ☆88Updated 2 months ago
• 2404589803 / hf-daily-paper-newsletter-chinese
  HF🤗每日简报机器人
  ☆33Updated this week
• ClosedCharacter / Peach
  我们是第一个完全可商用的角色大模型。
  ☆31Updated last month
• Bui1dMySea / MemLong
  ☆57Updated 3 weeks ago
• FateScript / token_visualizer
  Token level visualization tools for large language models
  ☆46Updated last month
• isaacJinyu / Math-Multi-Agent
  2024年阿里全球数学竞赛AI赛道全球第2名项目（特工宇宙）
  ☆45Updated 3 months ago
• RapidAI / Open-Llama
  The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
  ☆70Updated last year
• SmartFlowAI / Hand-on-RAG
  顾名思义：手搓的RAG
  ☆108Updated 6 months ago
• vaew / SkyScript-100M
  SkyScript-100M: 1,000,000,000 Pairs of Scripts and Shooting Scripts for Short Drama: https://arxiv.org/abs/2408.09333v2
  ☆92Updated 3 weeks ago
• ninehills / llm-playground
  Prompt 工程师利器，可同时比较多个 Prompts 在多个 LLM 模型上的效果
  ☆95Updated last year
• jingyaogong / minimind-v
  从头训练一个小参数量的视觉多模态VLM，预计2024年内开源
  ☆23Updated this week
• FunnySaltyFish / Better-Ruozhiba
  【逐条进行中】人为审核+加修改每一条的弱智吧精选问题QA数据集
  ☆80Updated 2 months ago
• LLM-Red-Team / emo-visual-data
  😜 表情包视觉数据集，使用glm-4v、step-1v的图像解析能力标注。
  ☆93Updated 4 months ago
• MoonshotAI / moonpalace
  MoonPalace（月宫）是由 Moonshot AI 月之暗面提供的 API 调试工具。
  ☆54Updated last week
• codefuse-ai / CodeFuse-MFT-VLM
  ☆32Updated 6 months ago
• opendatalab / image-downloader
  ☆25Updated 4 months ago
• SivilTaram / code-html-to-markdown
  A lightweight script for processing HTML page to markdown format with support for code blocks
  ☆70Updated 5 months ago
• mazzzystar / TurtleBenchmark
  Benchmark for LLM Reasoning & Understanding with Challenging Tasks from Real Users.
  ☆103Updated this week
• LightChen233 / Awesome-LLM-for-NLP
  ☆89Updated 3 months ago
• Liuziyu77 / Soda
  Search, organize, discover anything!
  ☆44Updated 5 months ago
• THUDM / CogAgent
  ☆56Updated 8 months ago
• MLGroup-JLU / LLM-data-aug-survey
  The official GitHub page for the survey paper "A Survey on Data Augmentation in Large Model Era"
  ☆101Updated 2 months ago
• paralym / COIG-CQIA
  ☆75Updated 9 months ago
• CyberCommy / baidu-qa-100w
  百度QA100万数据集
  ☆48Updated 9 months ago
• OpenBMB / MobileCPM
  A Toolkit for Running On-device Large Language Models (LLMs) in APP
  ☆53Updated 2 months ago
• ninehills / langeval
  Evaluation for AI apps and agent
  ☆35Updated 8 months ago
• JingYiJun / MOSS_backend
  backend for fastnlp MOSS project
  ☆58Updated 2 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆121Updated 3 months ago
• ssbuild / rwkv_finetuning
  rwkv finetuning
  ☆35Updated 4 months ago