BBuf / RWKV-World-HF-Tokenizer
☆34Updated 8 months ago
Alternatives and similar repositories for RWKV-World-HF-Tokenizer:
Users that are interested in RWKV-World-HF-Tokenizer are comparing it to the libraries listed below
- A fast RWKV Tokenizer written in Rust☆43Updated this week
- RWKV-7: Surpassing GPT☆82Updated 4 months ago
- A repository for research on medium sized language models.☆76Updated 10 months ago
- ☆30Updated 9 months ago
- ☆31Updated 9 months ago
- GoldFinch and other hybrid transformer components☆45Updated 8 months ago
- ☆18Updated 2 months ago
- Evaluating LLMs with Dynamic Data☆78Updated last month
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated 9 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆62Updated last year
- ☆110Updated last week
- RWKV v5,v6 LoRA Trainer on Cuda and Rocm Platform. RWKV is a RNN with transformer-level LLM performance. It can be directly trained like …☆12Updated last year
- Reinforcement Learning Toolkit for RWKV.(v6,v7,ARWKV) Distillation,SFT,RLHF(DPO,ORPO), infinite context training, Aligning. Exploring the…☆35Updated last week
- DPO, but faster 🚀☆40Updated 3 months ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆97Updated 5 months ago
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆53Updated last month
- A large-scale RWKV v6, v7(World, ARWKV, PRWKV) inference. Capable of inference by combining multiple states(Pseudo MoE). Easy to deploy o…☆32Updated last week
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆44Updated 8 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆26Updated 6 months ago
- imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…☆30Updated 9 months ago
- RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!☆148Updated 7 months ago
- ☆53Updated 9 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆40Updated last year
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆121Updated 2 months ago
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆33Updated last year
- RWKV, in easy to read code☆71Updated 3 months ago
- ☆17Updated 10 months ago
- ☆46Updated last year