riotu-lab / aranizer
Aranizer: A Custom Tokenizer based on SentencePiece and BPE tailored for Arabic Language Modeling
☆15Updated 3 months ago
Related projects ⓘ
Alternatives and complementary repositories for aranizer
- Instruction dataset for Arabic with 10,000 instruction and output pairs. CIDAR can be used to fine-tune LLMs to follow instructions.☆32Updated 9 months ago
- Python intefrace for evaluation on chatgpt models☆19Updated 9 months ago
- The official implementation of CATT Arabic diacritization models.☆35Updated 3 months ago
- This is the official repository for Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks.☆19Updated 8 months ago
- TURJUMAN, a neural toolkit for translating from 20 languages into Modern Standard Arabic (MSA).☆52Updated last year
- Seq2Seq-based open domain empathetic conversational model for Arabic: Dataset & Model☆56Updated 4 months ago
- مستودع الأوراق المسحية في معالجة اللغة العربية (أسبر) A Repository for survey and review papers in Arabic Natural Language processing (AN…☆78Updated last week
- ☆114Updated 8 months ago
- ☆40Updated last year
- ☆39Updated 4 months ago
- AraT5: Text-to-Text Transformers for Arabic Language Understanding☆85Updated 6 months ago
- Arabic nested named entity recognition☆32Updated 6 months ago
- A Python implementation of Farasa toolkit☆112Updated last month
- The largest public catalogue for Arabic NLP and speech datasets. There are +500 datasets annotated with more than 25 attributes.☆147Updated 4 months ago
- Arabic cleaning, normalization and segmentation library.☆61Updated last year
- Arabic edition of ALBERT pretrained language models☆16Updated 3 years ago
- ANETAC: Arabic Named Entity Transliteration and Classification Dataset☆34Updated 5 years ago
- This repository contains the Arabic sarcasm dataset (ArSarcasm)☆23Updated 3 years ago
- This is a diacritization model for Arabic language. This model was built/trained using the Tashkeela: the Arabic diacritization corpus on…☆38Updated last year
- ☆21Updated 5 months ago
- UBC ARBERT and MARBERT Deep Bidirectional Transformers for Arabic☆101Updated 3 years ago
- This is a repository of the Multi-dialect Arabic BERT model.☆38Updated 4 years ago
- Benchmark Arabic text diacritization dataset☆71Updated 5 years ago
- hULMonA (حلمنا): tHe first Universal Language MOdel iN Arabic☆46Updated 4 years ago
- Code and models for "The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models". EACL 2021, WANLP.☆43Updated 5 months ago
- ☆24Updated 5 months ago
- Arabic named entity recognition using AnerCorp corpus (location , organisation, person, Miscellaneous Word)☆37Updated 7 years ago
- Arabic Tokenization Library. It provides many tokenization algorithms.☆88Updated 10 months ago
- ☆9Updated 3 years ago
- ArSarcasm-v2 is an extension to the original ArSarcasm dataset. It was used for the shared task on sarcasm detection and sentiment analys…☆11Updated 2 years ago