hscspring / bytepiece-rsLinks
The Bytepiece Tokenizer Implemented in Rust.
☆14Updated last year
Alternatives and similar repositories for bytepiece-rs
Users that are interested in bytepiece-rs are comparing it to the libraries listed below
Sorting:
- Langport is a language model inference service☆94Updated last year
- 更纯粹、更高压缩率的Tokenizer in Rust☆13Updated 10 months ago
- A more efficient GLM implementation!☆54Updated 2 years ago
- Longitudinal Evaluation of LLMs via Data Compression☆33Updated last year
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆58Updated last year
- ☆19Updated last year
- ☆42Updated 2 years ago
- This repository provides an implementation of "A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction B…☆83Updated 3 months ago
- Manages vllm-nccl dependency☆17Updated last year
- Code & Data for our Paper "RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation" (EMNLP 2023)☆17Updated last year
- Code for KaLM-Embedding models☆93Updated 3 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆64Updated last year
- RWKV models and examples powered by candle.☆19Updated 7 months ago
- Fast serverless LLM inference, in Rust.☆94Updated 7 months ago
- Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper☆150Updated last year
- 1.4B sLLM for Chinese and English - HammerLLM🔨☆44Updated last year
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆41Updated last year
- A wide variety of research projects developed by the SpokenNLP team of Speech Lab, Alibaba Group.☆118Updated 4 months ago
- Imitate OpenAI with Local Models☆88Updated last year
- A Demo server serving Bert through ONNX with GPU written in Rust with <3☆41Updated 4 years ago
- The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.☆67Updated 2 years ago
- This repository has code for fine-tuning LLMs with GRPO specifically for Rust Programming using cargo as feedback☆107Updated 7 months ago
- patches for huggingface transformers to save memory☆30Updated 4 months ago
- 本项目旨在对大量文本文件进行快速编码检测和转换以辅助mnbvc语料集项目的数据清洗工作☆65Updated last week
- ☆105Updated 2 years ago
- ☆24Updated 6 months ago
- ☆32Updated 2 years ago
- Experiments on speculative sampling with Llama models☆125Updated 2 years ago
- A relatively basic implementation of RWKV in Rust written by someone with very little math and ML knowledge. Supports 32, 8 and 4 bit eva…☆94Updated 2 years ago
- Latest Evaluation Toolkit (LatestEval). Assessing the language models with latest, uncontaminated materials.☆25Updated 8 months ago