riotu-lab / aranizer

Aranizer: A Custom Tokenizer based on SentencePiece and BPE tailored for Arabic Language Modeling
13Updated last month

Related projects: