Alternatives and similar repositories for Japanese-BPEEncoder

Users that are interested in Japanese-BPEEncoder are comparing it to the libraries listed below

• tanreinama / Japanese-BPEEncoder_V2

  View on GitHub
  Japanese-BPEEncoder Version 2
  ☆41Jan 15, 2023Updated 3 years ago
• tanreinama / RoBERTa-japanese

  View on GitHub
  Japanese BERT Pretrained Model
  ☆23Nov 13, 2021Updated 4 years ago
• ndl-lab / huriganacorpus-ndlbib

  View on GitHub
  全国書誌データから作成した振り仮名のデータセット
  ☆28Sep 21, 2021Updated 4 years ago
• ndl-lab / huriganacorpus-aozora

  View on GitHub
  青空文庫及びサピエの点字データから作成した振り仮名コーパスのデータセット
  ☆17Jan 17, 2024Updated 2 years ago
• tanreinama / aMLP-japanese

  View on GitHub
  aMLP Transformer Model for Japanese
  ☆16May 10, 2022Updated 3 years ago
• GINK03 / minimal-search-engine

  View on GitHub
  最小のサーチエンジン/PageRank/tf-idf
  ☆19May 22, 2023Updated 2 years ago
• tarao / lambda-scala3

  View on GitHub
  Type-level lambda calculus in Scala 3
  ☆13Oct 11, 2022Updated 3 years ago
• aiishii / JEMHopQA

  View on GitHub
  ☆29Apr 10, 2025Updated 10 months ago
• WorksApplications / SudachiTra

  View on GitHub
  Japanese tokenizer for Transformers
  ☆79Dec 15, 2023Updated 2 years ago
• tanreinama / text2text-japanese

  View on GitHub
  gpt-2 based text2text conversion model
  ☆33Jul 22, 2021Updated 4 years ago
• sbintuitions / flexeval

  View on GitHub
  Flexible evaluation tool for language models
  ☆57Jan 28, 2026Updated 2 weeks ago
• i10416 / laika-laminar

  View on GitHub
  Good-Bye non type-safe JavaScript and CSS😊
  ☆19Feb 2, 2023Updated 3 years ago
• mynlp / niilc-qa

  View on GitHub
  NIILC QA data
  ☆18Nov 20, 2015Updated 10 years ago
• laboroai / Laboro-DistilBERT-Japanese

  View on GitHub
  ☆16Dec 17, 2020Updated 5 years ago
• 1never / open2ch-dialogue-corpus

  View on GitHub
  おーぷん2ちゃんねるをクロールして作成した対話コーパス
  ☆98Jun 6, 2021Updated 4 years ago
• kunishou / oasst1-89k-ja

  View on GitHub
  ☆16Nov 19, 2023Updated 2 years ago
• mizuumi / JDocQA

  View on GitHub
  ☆41Apr 10, 2025Updated 10 months ago
• hottolink / hottoSNS-bert

  View on GitHub
  hottoSNS-BERT: 大規模SNSコーパスによる文分散表現モデル
  ☆62Jan 22, 2026Updated 3 weeks ago
• laboroai / Laboro-ParaCorpus

  View on GitHub
  Scripts for creating a Japanese-English parallel corpus and training NMT models
  ☆18Nov 9, 2021Updated 4 years ago
• kanjirz50 / termextract

  View on GitHub
  専門用語抽出アルゴリズムの実装の練習
  ☆18Sep 26, 2018Updated 7 years ago
• ce-lery / japanese-mistral-300m-recipe

  View on GitHub
  ☆19Sep 26, 2025Updated 4 months ago
• sonoisa / t5-japanese

  View on GitHub
  日本語T5モデル
  ☆116Sep 15, 2025Updated 4 months ago
• akirakubo / bert-japanese-aozora

  View on GitHub
  Japanese BERT trained on Aozora Bunko and Wikipedia, pre-tokenized by MeCab with UniDic & SudachiPy
  ☆40Aug 8, 2020Updated 5 years ago
• BandaiNamcoResearchInc / DistilBERT-base-jp

  View on GitHub
  ☆161Oct 19, 2020Updated 5 years ago
• ku-nlp / bertknp

  View on GitHub
  A Japanese dependency parser based on BERT
  ☆23Oct 26, 2022Updated 3 years ago
• llm-jp / llm-jp-tokenizer

  View on GitHub
  ☆45Sep 6, 2025Updated 5 months ago
• yahoojapan / JGLUE

  View on GitHub
  JGLUE: Japanese General Language Understanding Evaluation
  ☆333Mar 31, 2025Updated 10 months ago
• ku-nlp / jumanpp

  View on GitHub
  Juman++ (a Morphological Analyzer Toolkit)
  ☆407Oct 3, 2023Updated 2 years ago
• pfnet-research / pfgen-bench

  View on GitHub
  Preferred Generation Benchmark
  ☆91Oct 28, 2025Updated 3 months ago
• hppRC / llm-translator

  View on GitHub
  Mixtral-based Ja-En (En-Ja) Translation model
  ☆20Jan 6, 2025Updated last year
• himkt / awesome-bert-japanese

  View on GitHub
  📝 A list of pre-trained BERT models for Japanese with word/subword tokenization + vocabulary construction algorithm information
  ☆131Mar 15, 2023Updated 2 years ago
• Ino-Ichan / GIT-LLM

  View on GitHub
  ☆22Sep 18, 2023Updated 2 years ago
• pityka / lamp

  View on GitHub
  deep learning and scientific computing framework with native CPU and GPU backend for the Scala programming language
  ☆29Apr 22, 2025Updated 9 months ago
• jqk09a / japanese-daily-dialogue

  View on GitHub
  ☆54Mar 17, 2023Updated 2 years ago
• Takeuchi-Lab-LM / python_asa

  View on GitHub
  python版日本語意味役割付与システム（ASA）
  ☆22Nov 11, 2022Updated 3 years ago
• shonosuke / ishiwatari-naacl2019

  View on GitHub
  Learning to Describe Unknown Phrases with Local and Global Contexts
  ☆21Jun 21, 2022Updated 3 years ago
• tanreinama / Japanese-Fakenews-Dataset

  View on GitHub
  日本語フェイクニュースデータセット
  ☆20May 2, 2021Updated 4 years ago
• tmu-nlp / TwitterCorpus

  View on GitHub
  首都大日本語 Twitter コーパス
  ☆21Mar 14, 2016Updated 9 years ago
• ikegami-yukino / neologdn

  View on GitHub
  Japanese text normalizer for mecab-neologd
  ☆286Dec 2, 2025Updated 2 months ago