google-research-datasets / dakshinaView external linksLinks
The Dakshina dataset is a collection of text in both Latin and native scripts for 12 South Asian languages. For each language, the dataset includes a large collection of native script Wikipedia text, a romanization lexicon of words in the native script with attested romanizations, and some full sentence parallel data in both a native script of t…
☆205May 27, 2020Updated 5 years ago
Alternatives and similar repositories for dakshina
Users that are interested in dakshina are comparing it to the libraries listed below
Sorting:
- The project aims on adding a state-of-the-art transliteration module for cross transliterations among all Indian languages including Engl…☆273Oct 28, 2022Updated 3 years ago
- A collaborative catalog of NLP resources for Indic languages☆628Dec 14, 2024Updated last year
- Pre-trained, multilingual sequence-to-sequence models for Indian languages☆51Jul 20, 2022Updated 3 years ago
- Natural Language Toolkit for Indic Languages aims to provide out of the box support for various NLP tasks that an application developer m…☆839Jan 20, 2024Updated 2 years ago
- Description Describes the IndicNLP corpus and associated datasets☆193Apr 16, 2023Updated 2 years ago
- indicTranslate v1 - Machine Translation for 11 Indic languages. For latest v2, check: https://github.com/AI4Bharat/IndicTrans2☆136Jan 2, 2024Updated 2 years ago
- Resources and tools for Indian language Natural Language Processing☆627Jun 7, 2024Updated last year
- Indic-BERT-v1: BERT-based Multilingual Model for 11 Indic Languages and Indian-English. For latest Indic-BERT v2, check: https://github.c…☆292May 11, 2023Updated 2 years ago
- Dataset of sentences from Hindi stories tagged with different emotion tags☆11Nov 26, 2019Updated 6 years ago
- A rule-based lemmatizer for Bengali / Bangla based written in Python. Under active development.☆25Dec 28, 2019Updated 6 years ago
- MIMICS: A Large-Scale Data Collection for Search Clarification☆80Sep 1, 2020Updated 5 years ago
- Aksharamukha Python Library☆56Feb 2, 2025Updated last year
- 🎯 Speech Recognition Challenge by Speech Lab - IIT Madras☆11Nov 5, 2020Updated 5 years ago
- CSCI 544 NLP Research Project - Machine Translation☆11Apr 10, 2017Updated 8 years ago
- Datasets and tools for basic natural language processing.☆388Sep 10, 2021Updated 4 years ago
- Shoonya - Platform to Annotate and label data at scale.☆64Oct 31, 2025Updated 3 months ago
- Python library for converting UTF to WX and vice-versa for Indian languages.☆47May 26, 2022Updated 3 years ago
- ☆47Jan 23, 2020Updated 6 years ago
- A benchmark for code-switched NLP, ACL 2020☆76May 28, 2024Updated last year
- BNLP is a natural language processing toolkit for Bengali Language.☆308Jan 22, 2026Updated 3 weeks ago
- Resources and Tool for Bangla language computation☆14May 5, 2023Updated 2 years ago
- Repository for the English-Hindi Codemixed to Monolingual English Parallel Corpus☆13Feb 17, 2019Updated 6 years ago
- https://sharedtask.duolingo.com☆50Mar 19, 2020Updated 5 years ago
- Open source speech to text models for Indic Languages☆321Sep 16, 2022Updated 3 years ago
- Code from the paper "Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity"☆19May 8, 2020Updated 5 years ago
- The website for the CMU Language Technologies Institute low resource NLP bootcamp 2020☆606Jun 4, 2020Updated 5 years ago
- Xlit-Crowd: Hindi-English Transliteration Corpus☆38Feb 17, 2015Updated 10 years ago
- Viewer for the 🤗 datasets library.☆86Jul 30, 2021Updated 4 years ago
- Code for ACL 2022 paper "Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation"☆30Apr 2, 2022Updated 3 years ago
- Pytorch implementation for paper 'BANNER: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition'☆13Apr 15, 2020Updated 5 years ago
- An LSTM-CRF classifier for NER in Telugu, an Indian language.☆15Sep 4, 2022Updated 3 years ago
- Tokenization across languages. Useful as preprocessing for subword tokenization.☆21Feb 7, 2023Updated 3 years ago
- Various scripts that facilitate the preparation of Automatic Speech Recognition related resources☆17Apr 16, 2020Updated 5 years ago
- Finite-state script normalization and processing utilities☆46Jan 14, 2026Updated last month
- A tutorial on how to implement models for natural language inference using PyTorch and TorchText. [IN PROGRESS]☆26Mar 3, 2020Updated 5 years ago
- This code provides word level language identification tool for identifying language for individual words in Code-Mixed text. e.g. The tex…☆57Aug 11, 2020Updated 5 years ago
- A tool for holistic analysis of language generations systems☆471Sep 22, 2025Updated 4 months ago
- A collection of Bangla newspaper and blog crawlers. Can be used to mine bangla text data for Natural Language Processing tasks.☆18Jan 30, 2023Updated 3 years ago
- ☆22May 5, 2022Updated 3 years ago