Kensuke-Mitsuzawa / JapaneseTokenizers
aim to use JapaneseTokenizer as easy as possible
โ138Updated 5 years ago
Alternatives and similar repositories for JapaneseTokenizers:
Users that are interested in JapaneseTokenizers are comparing it to the libraries listed below
- Juman++ (a Morphological Analyzer Toolkit)โ385Updated last year
- ๐ฟ An easy-to-use Japanese Text Processing tool, which makes it possible to switch tokenizers with small changes of code.โ241Updated 10 months ago
- Kyoto University Web Document Leads Corpusโ81Updated last year
- Japanese Word Similarity Datasetโ101Updated 3 years ago
- 50k English-Japanese Parallel Corpus for Machine Translation Benchmark.โ95Updated 5 years ago
- japanese sentence segmentation library for pythonโ70Updated last year
- CaboCha wrapper for Python3โ47Updated 6 years ago
- lists of text corpus and more (mainly Japanese)โ116Updated 7 months ago
- Python version of Sudachi, a Japanese tokenizer.โ403Updated 2 years ago
- Sentence boundary disambiguation tool for Japanese texts (ๆฅๆฌ่ชๆๅข็ๅคๅฎๅจ)โ189Updated 11 months ago
- Japanese data from the Google UDT 2.0.โ37Updated 4 months ago
- Japanese text normalizer for mecab-neologdโ278Updated last week
- Yet Another Japanese Dependency Structure Analyzerโ111Updated last month
- A Python Module for JUMAN++/KNPโ89Updated 2 weeks ago
- natto-py combines the Python programming language with MeCab, the part-of-speech and morphological analyzer for the Japanese language.โ93Updated 9 months ago
- hottoSNS-BERT: ๅคง่ฆๆจกSNSใณใผใในใซใใๆๅๆฃ่กจ็พใขใใซโ61Updated 3 months ago
- A large parallel corpus of English and Japaneseโ84Updated 7 years ago
- A Japanese tokenizer based on recurrent neural networksโ396Updated 9 months ago
- The Kyoto Text Analysis Toolkit for word segmentation and pronunciation estimation, etc.โ207Updated 4 years ago
- An example usage of JParaCrawl pre-trained Neural Machine Translation (NMT) models.โ104Updated 3 years ago
- โ96Updated last year
- Distributed representations of words and named entities trained on Wikipedia.โ182Updated 3 years ago
- A comparison tool of Japanese tokenizersโ121Updated 9 months ago
- ๐ A list of pre-trained BERT models for Japanese with word/subword tokenization + vocabulary construction algorithm informationโ130Updated 2 years ago
- Japanese data from the Google UDT 2.0.โ28Updated 2 years ago
- chakki's Aspect-Based Sentiment Analysis datasetโ140Updated 3 years ago
- Python wrapper for KyTeaโ34Updated last year
- Japanese IOB2 tagged corpus for Named Entity Recognition.โ60Updated 5 years ago
- Word2vec (word to vectors) approach for Japanese language using Gensim and Mecab.โ86Updated 3 years ago
- Kyoto University Text Corpusโ62Updated last year