Alternatives and similar repositories for bytepiece-rs

Users that are interested in bytepiece-rs are comparing it to the libraries listed below

• vtuber-plan / langport
  Langport is a language model inference service
  ☆94Updated last year
• SunDoge / bytepiece-rs
  更纯粹、更高压缩率的Tokenizer in Rust
  ☆13Updated 10 months ago
• Oneflow-Inc / one-glm
  A more efficient GLM implementation!
  ☆54Updated 2 years ago
• liyucheng09 / llm-compressive
  Longitudinal Evaluation of LLMs via Data Compression
  ☆33Updated last year
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year
• LaurentMazare / glim
  ☆19Updated last year
• mrsteyk / RWKV-LM-deepspeed
  ☆42Updated 2 years ago
• Jacob-Zhou / simple-csc
  This repository provides an implementation of "A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction B…
  ☆83Updated 3 months ago
• vllm-project / vllm-nccl
  Manages vllm-nccl dependency
  ☆17Updated last year
• HillZhang1999 / RobustGEC
  Code & Data for our Paper "RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation" (EMNLP 2023)
  ☆17Updated last year
• HITsz-TMG / KaLM-Embedding
  Code for KaLM-Embedding models
  ☆93Updated 3 months ago
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆64Updated last year
• nkypy / candle-rwkv
  RWKV models and examples powered by candle.
  ☆19Updated 7 months ago
• atoma-network / atoma-infer
  Fast serverless LLM inference, in Rust.
  ☆94Updated 7 months ago
• jshuadvd / LongRoPE
  Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper
  ☆150Updated last year
• Academic-Hammer / HammerLLM
  1.4B sLLM for Chinese and English - HammerLLM🔨
  ☆44Updated last year
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆41Updated last year
• alibaba-damo-academy / SpokenNLP
  A wide variety of research projects developed by the SpokenNLP team of Speech Lab, Alibaba Group.
  ☆118Updated 4 months ago
• the-seeds / imitater
  Imitate OpenAI with Local Models
  ☆88Updated last year
• haixuanTao / bert-onnx-rs-server
  A Demo server serving Bert through ONNX with GPU written in Rust with <3
  ☆41Updated 4 years ago
• beichao1314 / Open-Llama
  The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
  ☆67Updated 2 years ago
• Oxen-AI / GRPO-With-Cargo-Feedback
  This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback
  ☆107Updated 7 months ago
• GeeeekExplorer / transformers-patch
  patches for huggingface transformers to save memory
  ☆30Updated 4 months ago
• alanshi / charset_mnbvc
  本项目旨在对大量文本文件进行快速编码检测和转换以辅助mnbvc语料集项目的数据清洗工作
  ☆65Updated last week
• AI21Labs / Parallel-Context-Windows
  ☆105Updated 2 years ago
• Narsil / bindgen_cuda
  ☆24Updated 6 months ago
• mrsteyk / rwkvk-rs
  ☆32Updated 2 years ago
• dust-tt / llama-ssp
  Experiments on speculative sampling with Llama models
  ☆125Updated 2 years ago
• KerfuffleV2 / smolrsrwkv
  A relatively basic implementation of RWKV in Rust written by someone with very little math and ML knowledge. Supports 32, 8 and 4 bit eva…
  ☆94Updated 2 years ago
• liyucheng09 / LatestEval
  Latest Evaluation Toolkit (LatestEval). Assessing the language models with latest, uncontaminated materials.
  ☆25Updated 8 months ago