llm-jp / text2dataset
Easily turn large English text datasets into Japanese text datasets using open LLMs.
☆17Updated 2 months ago
Alternatives and similar repositories for text2dataset:
Users that are interested in text2dataset are comparing it to the libraries listed below
- Preferred Generation Benchmark☆60Updated last week
- 【2024年版】BERTによるテキスト分類☆27Updated 6 months ago
- ☆50Updated last year
- ☆83Updated last year
- ☆34Updated 5 years ago
- Mecab + NEologd + Docker + Python3☆35Updated 2 years ago
- ☆25Updated 2 months ago
- ☆15Updated 2 weeks ago
- ☆33Updated 5 months ago
- JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset, LREC-COLING 2024☆22Updated 9 months ago
- ☆28Updated 7 months ago
- Japanese instruction data (日本語指示データ)☆22Updated last year
- デジタル化資料OCRテキスト化事業において作成されたOCR学習用データセット☆66Updated 6 months ago
- Exploring Japanese SimCSE☆67Updated last year
- JQaRA: Japanese Question Answering with Retrieval Augmentation - 検索拡張(RAG)評価のための日本語Q&Aデータセット☆28Updated this week
- Japanese-BPEEncoder☆41Updated 3 years ago
- Viterbi-based accelerated tokenizer (Python wrapper)☆41Updated 4 months ago
- Mixtral-based Ja-En (En-Ja) Translation model☆16Updated last week
- DeepLearningのAttentionモデルをPytorchの低レベルAPIを使って1から制作しようという試みのリポジトリです。☆46Updated last year
- ☆22Updated last year
- DistilBERT model pre-trained on 131 GB of Japanese web text. The teacher model is BERT-base that built in-house at LINE.☆43Updated last year
- ☆25Updated 3 weeks ago
- Training and evaluation scripts for JGLUE, a Japanese language understanding benchmark☆17Updated last week
- ボケて電笑戦 (bokete DENSHOSEN) Workshop☆41Updated 2 years ago
- ☆15Updated 10 months ago
- 敬語変換タスクにおける評価用データセット☆20Updated 2 years ago
- ☆52Updated 7 months ago
- ☆44Updated last month
- NLP2024 チュートリアル3 作って学ぶ日本語大規模言語モデル - 環境構築手順とソースコード / NLP2024 Tutorial 3: Practicing how to build a Japanese large-scale language model - E…☆108Updated 9 months ago
- Japanese OCR with CenterNet☆38Updated this week