The Dakshina dataset is a collection of text in both Latin and native scripts for 12 South Asian languages. For each language, the dataset includes a large collection of native script Wikipedia text, a romanization lexicon of words in the native script with attested romanizations, and some full sentence parallel data in both a native script of t…
☆206May 27, 2020Updated 5 years ago
Alternatives and similar repositories for dakshina
Users that are interested in dakshina are comparing it to the libraries listed below
Sorting:
- The project aims on adding a state-of-the-art transliteration module for cross transliterations among all Indian languages including Engl…☆273Oct 28, 2022Updated 3 years ago
- A collaborative catalog of NLP resources for Indic languages☆627Dec 14, 2024Updated last year
- Natural Language Toolkit for Indic Languages aims to provide out of the box support for various NLP tasks that an application developer m…☆839Jan 20, 2024Updated 2 years ago
- Description Describes the IndicNLP corpus and associated datasets☆195Apr 16, 2023Updated 2 years ago
- Resources and tools for Indian language Natural Language Processing☆629Jun 7, 2024Updated last year
- Indic-BERT-v1: BERT-based Multilingual Model for 11 Indic Languages and Indian-English. For latest Indic-BERT v2, check: https://github.c…☆291May 11, 2023Updated 2 years ago
- Dataset of sentences from Hindi stories tagged with different emotion tags☆11Nov 26, 2019Updated 6 years ago
- A rule-based lemmatizer for Bengali / Bangla based written in Python. Under active development.☆25Dec 28, 2019Updated 6 years ago
- Geometry-aware Multilingual Embeddings☆26Dec 8, 2022Updated 3 years ago
- MIMICS: A Large-Scale Data Collection for Search Clarification☆80Sep 1, 2020Updated 5 years ago
- Aksharamukha Python Library☆58Feb 2, 2025Updated last year
- 🎯 Speech Recognition Challenge by Speech Lab - IIT Madras☆11Nov 5, 2020Updated 5 years ago
- transliterate hindi to english☆15Dec 21, 2025Updated 2 months ago
- CSCI 544 NLP Research Project - Machine Translation☆11Apr 10, 2017Updated 8 years ago
- Datasets and tools for basic natural language processing.☆389Sep 10, 2021Updated 4 years ago
- Shoonya - Platform to Annotate and label data at scale.☆64Oct 31, 2025Updated 4 months ago
- Classification Benchmarks for Under-resourced Bengali Language based on Multichannel Convolutional-LSTM Network☆20Jul 26, 2021Updated 4 years ago
- ☆47Jan 23, 2020Updated 6 years ago
- A benchmark for code-switched NLP, ACL 2020☆76May 28, 2024Updated last year
- BNLP is a natural language processing toolkit for Bengali Language.☆308Jan 22, 2026Updated last month
- Repository for the English-Hindi Codemixed to Monolingual English Parallel Corpus☆13Feb 17, 2019Updated 7 years ago
- ☆38Jan 13, 2024Updated 2 years ago
- https://sharedtask.duolingo.com☆50Mar 19, 2020Updated 5 years ago
- Open source speech to text models for Indic Languages☆324Sep 16, 2022Updated 3 years ago
- The website for the CMU Language Technologies Institute low resource NLP bootcamp 2020☆606Jun 4, 2020Updated 5 years ago
- Xlit-Crowd: Hindi-English Transliteration Corpus☆38Feb 17, 2015Updated 11 years ago
- Viewer for the 🤗 datasets library.☆86Jul 30, 2021Updated 4 years ago
- Code for ACL 2022 paper "Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation"☆30Apr 2, 2022Updated 3 years ago
- Finite-state script normalization and processing utilities☆46Feb 25, 2026Updated last week
- Tokenization across languages. Useful as preprocessing for subword tokenization.☆21Feb 7, 2023Updated 3 years ago
- An LSTM-CRF classifier for NER in Telugu, an Indian language.☆15Sep 4, 2022Updated 3 years ago
- Source code for ACL 2020 paper "Learning Spoken Language Representations with Neural Lattice Language Modeling"☆17Feb 11, 2023Updated 3 years ago
- Pytorch implementation for paper 'BANNER: A Cost-Sensitive Contextualized Model for Bangla Named Entity Recognition'☆13Apr 15, 2020Updated 5 years ago
- Various scripts that facilitate the preparation of Automatic Speech Recognition related resources☆17Apr 16, 2020Updated 5 years ago
- A tutorial on how to implement models for natural language inference using PyTorch and TorchText. [IN PROGRESS]☆26Mar 3, 2020Updated 6 years ago
- This code provides word level language identification tool for identifying language for individual words in Code-Mixed text. e.g. The tex…☆57Aug 11, 2020Updated 5 years ago
- This is a package in Python which implements a tokenizer, stemmer for Hindi language☆95Oct 2, 2020Updated 5 years ago
- A tool for holistic analysis of language generations systems☆471Sep 22, 2025Updated 5 months ago
- A collection of Bangla newspaper and blog crawlers. Can be used to mine bangla text data for Natural Language Processing tasks.☆18Jan 30, 2023Updated 3 years ago