Alternatives and similar repositories for bytepiece-rs

Users that are interested in bytepiece-rs are comparing it to the libraries listed below

• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• codefuse-ai / Collinear-Constrained-Attention
  ☆62Updated last year
• Jellyfish042 / uncheatable_eval
  Evaluating LLMs with Dynamic Data
  ☆95Updated 2 months ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated last year
• liyucheng09 / llm-compressive
  Longitudinal Evaluation of LLMs via Data Compression
  ☆33Updated last year
• RWKV-Wiki / MultilingualShareGPT
  MultilingualShareGPT, the free multi-language corpus for LLM training
  ☆73Updated 2 years ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆137Updated last year
• TemporaryLoRA / Temp-LoRA
  ☆116Updated last year
• microsoft / nanovppo
  Nano repo for RL training of LLMs
  ☆66Updated last month
• princeton-pli / MeCo
  Code for ICML 25 paper "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆42Updated 3 months ago
• Jacob-Zhou / simple-csc
  This repository provides an implementation of "A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction B…
  ☆78Updated 2 months ago
• THUDM / icetk
  A unified tokenization tool for Images, Chinese and English.
  ☆151Updated 2 years ago
• inferflow / inferflow
  Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
  ☆248Updated last year
• Infini-AI-Lab / gsm_infinite
  ☆54Updated 3 months ago
• bojone / tiger
  A Tight-fisted Optimizer
  ☆50Updated 2 years ago
• Bui1dMySea / MemLong
  ☆95Updated 10 months ago
• YuchuanTian / DiJiang
  [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…
  ☆102Updated last year
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆64Updated last year
• cofe-ai / Mu-scaling
  Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales
  ☆32Updated 2 years ago
• ninehills / langeval
  Evaluation for AI apps and agent
  ☆43Updated last year
• gabrielolympie / moe-pruner
  A repository aimed at pruning DeepSeek V3, R1 and R1-zero to a usable size
  ☆69Updated last month
• ExpressAI / AI-Gaokao
  Gaokao Benchmark for AI
  ☆108Updated 3 years ago
• woct0rdho / transformers-qwen3-moe-fused
  Fused Qwen3 MoE layer for faster training, compatible with HF Transformers, LoRA, 4-bit quant, Unsloth
  ☆183Updated last week
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆56Updated last week
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆78Updated last year
• FlagOpen / Infinity-Instruct
  ☆49Updated last year
• Oneflow-Inc / one-glm
  A more efficient GLM implementation!
  ☆54Updated 2 years ago
• johanwind / wind_rwkv
  ☆26Updated 2 months ago
• OpenNLPLab / Transnormer
  [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer
  ☆63Updated 2 years ago