llm-jp / text2dataset
Easily turn large English text datasets into Japanese text datasets using open LLMs.
☆20Updated 2 months ago
Alternatives and similar repositories for text2dataset:
Users that are interested in text2dataset are comparing it to the libraries listed below
- ☆83Updated last year
- 【2024年版】BERTによるテキスト分類☆29Updated 9 months ago
- Swallowプロジェクト 大規模言語モデル 評価スクリプト☆16Updated last week
- ☆16Updated last year
- ☆50Updated last year
- Mixtral-based Ja-En (En-Ja) Translation model☆19Updated 3 months ago
- Preferred Generation Benchmark☆78Updated 3 weeks ago
- ☆26Updated 5 months ago
- Mecab + NEologd + Docker + Python3☆35Updated 2 years ago
- デジタル化資料OCRテキスト化事業において作成されたOCR学習用データセット☆70Updated 9 months ago
- ☆48Updated 3 months ago
- JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset, LREC-COLING 2024☆23Updated last year
- ☆51Updated 10 months ago
- ☆34Updated 5 years ago
- Japanese-BPEEncoder☆41Updated 3 years ago
- ☆22Updated last year
- ☆39Updated last month
- Viterbi-based accelerated tokenizer (Python wrapper)☆41Updated 7 months ago
- 敬語変換タスクにおける評価用データセット☆21Updated 2 years ago
- Accommodation Search Dialog Corpus (宿泊施設探索対話 コーパス)☆25Updated last year
- ☆127Updated this week
- ☆16Updated 3 months ago
- ☆14Updated 7 months ago
- ☆60Updated 10 months ago
- ☆21Updated 2 years ago
- The evaluation scripts of JMTEB (Japanese Massive Text Embedding Benchmark)☆53Updated 2 weeks ago
- Exploring Japanese SimCSE☆69Updated last year
- ☆55Updated 2 months ago
- JQaRA: Japanese Question Answering with Retrieval Augmentation - 検索拡張(RAG)評価のための日本語Q&Aデータセット☆30Updated 2 months ago
- ☆20Updated 2 months ago