speed1313 / jax-llmLinks
JAX implementation of Large Language Models. You can train GPT-2-like model with 青空文庫 (aozora bunko-clean dataset) or any other text dataset.
☆13Updated last year
Alternatives and similar repositories for jax-llm
Users that are interested in jax-llm are comparing it to the libraries listed below
Sorting:
- ☆27Updated 11 months ago
- JMultiWOZ: A Large-Scale Japanese Multi-Domain Task-Oriented Dialogue Dataset, LREC-COLING 2024☆25Updated last year
- Swallowプロジェクト 事後学習済み大規模言語モデル 評価フレームワーク☆21Updated 2 weeks ago
- Preferred Generation Benchmark☆85Updated this week
- Easily turn large English text datasets into Japanese text datasets using open LLMs.☆22Updated 9 months ago
- ☆25Updated 5 months ago
- The evaluation scripts of JMTEB (Japanese Massive Text Embedding Benchmark)☆75Updated last month
- ☆34Updated 6 years ago
- ☆19Updated last year
- Japanese-BPEEncoder☆41Updated 4 years ago
- ☆51Updated 2 years ago
- Viterbi-based accelerated tokenizer (Python wrapper)☆43Updated last year
- Japanese Language Model Financial Evaluation Harness☆75Updated last week
- Mixtral-based Ja-En (En-Ja) Translation model☆19Updated 9 months ago
- 🛥 Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.☆20Updated 5 months ago
- ☆139Updated this week
- NLP 100 Exercise 2025☆33Updated 6 months ago
- 【2024年版】BERTによるテキスト分類☆29Updated last year
- ☆49Updated 10 months ago
- Exploring Japanese SimCSE☆69Updated 2 years ago
- ☆86Updated 2 years ago
- おーぷん2ちゃんねるをクロールして作成した対話コーパス☆99Updated 4 years ago
- Training and evaluation scripts for JGLUE, a Japanese language understanding benchmark☆17Updated 2 weeks ago
- This repository has implementations of data augmentation for NLP for Japanese.☆64Updated 2 years ago
- Japanese Realistic Textual Entailment Corpus (NLP 2020, LREC 2020)☆77Updated 2 years ago
- 敬語変換タスクにおける評価用データセット☆21Updated 2 years ago
- Funer is Rule based Named Entity Recognition tool.☆22Updated 3 years ago
- 原論文から解き明かす生成AI(技術評論社)のサポートページです☆71Updated last week
- Show notes for https://anchor.fm/yoheikikuta.☆15Updated 3 years ago
- Japanese instruction data (日本語指示データ)☆24Updated 2 years ago