Alternatives and similar repositories for bytepiece-rs

Users that are interested in bytepiece-rs are comparing it to the libraries listed below

• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated last year
• liyucheng09 / llm-compressive
  Longitudinal Evaluation of LLMs via Data Compression
  ☆32Updated last year
• TemporaryLoRA / Temp-LoRA
  ☆110Updated last year
• Jellyfish042 / uncheatable_eval
  Evaluating LLMs with Dynamic Data
  ☆91Updated last month
• RWKV-Wiki / MultilingualShareGPT
  MultilingualShareGPT, the free multi-language corpus for LLM training
  ☆73Updated 2 years ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• codefuse-ai / Collinear-Constrained-Attention
  ☆63Updated last year
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆136Updated last year
• princeton-pli / MeCo
  Code for ICML 25 paper "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆42Updated 2 months ago
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆247Updated last year
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆64Updated last year
• nick7nlp / Counting-Stars
  Counting-Stars (★)
  ☆83Updated 3 months ago
• ExpressAI / AI-Gaokao
  Gaokao Benchmark for AI
  ☆108Updated 3 years ago
• Jacob-Zhou / simple-csc
  This repository provides an implementation of "A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction B…
  ☆75Updated last month
• microsoft / nanovppo
  Nano repo for RL training of LLMs
  ☆63Updated 2 weeks ago
• THUDM / icetk
  A unified tokenization tool for Images, Chinese and English.
  ☆151Updated 2 years ago
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆76Updated last year
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆56Updated 3 weeks ago
• alanshi / charset_mnbvc
  本项目旨在对大量文本文件进行快速编码检测和转换以辅助mnbvc语料集项目的数据清洗工作
  ☆62Updated 10 months ago
• woct0rdho / transformers-qwen3-moe-fused
  Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth
  ☆168Updated last week
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆78Updated last year
• Oneflow-Inc / one-glm
  A more efficient GLM implementation!
  ☆54Updated 2 years ago
• OpenLMLab / MOSS_WebSearchTool
  MOSS 003 WebSearchTool: A simple but reliable implementation
  ☆45Updated 2 years ago
• wangshuai09 / vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆38Updated this week
• OpenLMLab / ChatZoo
  Light local website for displaying performances from different chat models.
  ☆87Updated last year
• rlite-project / RLite
  A lightweight reinforcement learning framework that integrates seamlessly into your codebase, empowering developers to focus on algorithm…
  ☆67Updated last week
• OpenLMLab / GAOKAO-Bench-Updates
  GAOGAO-Bench-Updates is a supplement to the GAOKAO-Bench, a dataset to evaluate large language models.
  ☆34Updated 7 months ago
• cofe-ai / Mu-scaling
  Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales
  ☆32Updated 2 years ago
• multimodal-art-projection / Megatron-LM-NEO
  ☆40Updated last year