monologg/ko_lm_dataformat external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

monologg/ko_lm_dataformat

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/monologg/ko_lm_dataformat)

Close

monologg / ko_lm_dataformatView on GitHubMore

• External links
• Readme badge

A utility for storing and reading files for Korean LM training 💾

☆35Jul 18, 2026Updated 3 months ago

Alternatives and similar repositories for ko_lm_dataformat

Users that are interested in ko_lm_dataformat are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.

• korean-named-entity / konec

  View on GitHub
  Korean Named Entity Corpus
  ☆26May 12, 2023Updated 3 years ago
• hamanlp / hama-py

  View on GitHub
  🦛 파이썬 한글 처리 라이브러리. Python Korean Morphological Analyzer
  ☆19Feb 4, 2025Updated last year
• passing2961 / EmoNSMC

  View on GitHub
  Korean large emotion labeled dataset (EmoNSMC)
  ☆14Mar 5, 2020Updated 6 years ago
• songys / 2021Langcon

  View on GitHub
  ☆11Oct 3, 2021Updated 5 years ago
• hyunwoongko / python-mecab-kor

  View on GitHub
  Yet another python binding for mecab-ko
  ☆88May 16, 2023Updated 3 years ago
• GPUs on demand by Runpod - Special Offer Available • Ad
  Run AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
• ko-nlp / moducorpus-sanitizer

  View on GitHub
  모두의 말뭉치 데이터를 분석에 편리한 형태로 변환하는 기능을 제공합니다.
  ☆11Mar 2, 2022Updated 4 years ago
• tlkh / t2t-tuner

  View on GitHub
  Convenient Text-to-Text Training for Transformers
  ☆18Dec 10, 2021Updated 4 years ago
• seopbo / nlp_tutorials

  View on GitHub
  huggingface를 이용하여 downstream task 수행하기
  ☆62Dec 28, 2021Updated 4 years ago
• jason9693 / oslo-kogpt-finetunig

  View on GitHub
  kogpt를 oslo로 파인튜닝하는 예제.
  ☆23Aug 26, 2022Updated 4 years ago
• lassl / lassl

  View on GitHub
  Easy Language Model Pretraining leveraging Huggingface's Transformers and Datasets
  ☆131Nov 12, 2022Updated 3 years ago
• naver-ai / carecall-corpus

  View on GitHub
  CareCall for Seniors: Role Specified Open-Domain Dialogue dataset generated by leveraging LLMs (NAACL 2022).
  ☆64May 3, 2022Updated 4 years ago
• codertimo / jax-lm-training

  View on GitHub
  generative language model training on top of the JAX and Huggingface 🤗
  ☆10May 11, 2022Updated 4 years ago
• SKplanet / Dialog-KoELECTRA

  View on GitHub
  ELECTRA기반 한국어 대화체 언어모델
  ☆54Aug 4, 2021Updated 5 years ago
• openkorpos / model-mecab

  View on GitHub
  MeCab model trained with OpenKorPos.
  ☆23Jun 19, 2022Updated 4 years ago
• Deploy on Railway without the complexity - Free Credits Offer • Ad
  Connect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
• monologg / KoCharELECTRA

  View on GitHub
  Character-level Korean ELECTRA Model (음절 단위 한국어 ELECTRA)
  ☆55Jun 12, 2023Updated 3 years ago
• JoungheeKim / kor-spacing

  View on GitHub
  This is project for korean auto spacing
  ☆12Aug 3, 2020Updated 6 years ago
• korean-named-entity / konne-prep

  View on GitHub
  ☆19Jan 29, 2023Updated 3 years ago
• monologg / KoBigBird

  View on GitHub
  🦅 Pretrained BigBird Model for Korean (up to 4096 tokens)
  ☆201Dec 28, 2023Updated 2 years ago
• snunlp / KR-ELECTRA

  View on GitHub
  KoRean based ELECTRA pre-trained models (KR-ELECTRA) for Tensorflow and PyTorch
  ☆15Feb 13, 2022Updated 4 years ago
• nayohan / SimKoR

  View on GitHub
  [HCLT 2022] Korean sentence text similarity dataset using naver shopping review
  ☆25Oct 20, 2022Updated 4 years ago
• Beomi / transformers-language-modeling

  View on GitHub
  Train 🤗transformers with DeepSpeed: ZeRO-2, ZeRO-3
  ☆23May 20, 2021Updated 5 years ago
• haven-jeon / KoGPT2-subtasks

  View on GitHub
  NSMC, KorSTS ... fine-tunings
  ☆18Feb 23, 2022Updated 4 years ago
• KLUE-benchmark / KLUE-baseline

  View on GitHub
  Finetuning Pipeline
  ☆89Feb 25, 2022Updated 4 years ago
• 1-Click AI Models by DigitalOcean Gradient • Ad
  Deploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
• tunib-ai / artwork_captions

  View on GitHub
  Machine Generated Captions for Best Artworks
  ☆22Sep 21, 2022Updated 4 years ago
• baikalai / baikal-bert

  View on GitHub
  baikal.ai's pre-trained BERT models: descriptions and sample codes
  ☆12Jun 24, 2021Updated 5 years ago
• paust-team / pko-t5

  View on GitHub
  bpe based korean t5 model for text-to-text unified framework
  ☆63Apr 17, 2024Updated 2 years ago
• hyunwoongko / pecab

  View on GitHub
  Pecab: Pure python Korean morpheme analyzer based on Mecab
  ☆172Apr 27, 2024Updated 2 years ago
• taeminlee / KoGPT2-Transformers

  View on GitHub
  KoGPT2 on Huggingface Transformers
  ☆33May 4, 2021Updated 5 years ago
• boychaboy / KOLD

  View on GitHub
  KOLD: Korean Offensive Language Dataset
  ☆84Nov 13, 2022Updated 3 years ago
• kakaobrain / kortok

  View on GitHub
  The code and models for "An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks" (AACL-IJCNLP 2020)
  ☆119Oct 8, 2020Updated 6 years ago
• nlpai-lab / Korean-CommonGen

  View on GitHub
  [Findings of NAACL2022] A Dog Is Passing Over The Jet? A Text-Generation Dataset for Korean Commonsense Reasoning and Evaluation
  ☆11May 27, 2022Updated 4 years ago
• ModuNLP / weekly-meeting

  View on GitHub
  매주 목요일, 20:00 모임
  ☆16Jul 24, 2020Updated 6 years ago
• Managed Database hosting by DigitalOcean • Ad
  PostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
• seujung / kolang-t5-base

  View on GitHub
  T5-base model for Korean
  ☆27May 20, 2021Updated 5 years ago
• monologg / EncT5

  View on GitHub
  Pytorch Implementation of EncT5: Fine-tuning T5 Encoder for Non-autoregressive Tasks
  ☆62Jan 22, 2022Updated 4 years ago
• cosmoquester / 2021-dialogue-summary-competition

  View on GitHub
  [2021 훈민정음 한국어 음성•자연어 인공지능 경진대회] 대화요약 부문 알라꿍달라꿍 팀의 대화요약 학습 및 추론 코드를 공유하기 위한 레포입니다.
  ☆127Jul 11, 2022Updated 4 years ago
• Huffon / klue-transformers-tutorial

  View on GitHub
  KLUE 데이터를 활용한 HuggingFace Transformers 튜토리얼
  ☆129Jun 28, 2021Updated 5 years ago
• jeongukjae / nori-clone

  View on GitHub
  Standalone Nori (Korean Morphological Analyzer)
  ☆42Sep 20, 2023Updated 3 years ago
• snoop2head / KLUE-RBERT

  View on GitHub
  ↔️ Utilizing RBERT model structure for KLUE Relation Extraction task
  ☆15Nov 15, 2022Updated 3 years ago
• Beomi / exbert-transformers

  View on GitHub
  exBERT on Transformers🤗
  ☆10Jun 14, 2021Updated 5 years ago