EleutherAI/dps external links

---

GitHub - View repository

GitHub.dev - Open in online code editor

Star history - Growth chart over time

deps.dev - Dependencies and security vulnerabilities

Gitingest - Export source code for LLM usage

Repomix - Export source code for LLM usage

uithub - Export source code for LLM usage

Close

EleutherAI/dps

Readme badge preview -

If you own this repo, copy the snippet below and add it to your README.md

---

Copy

[![RelatedRepos](https://img.shields.io/badge/related-repos-yellow)](https://relatedrepos.com/gh/EleutherAI/dps)

Close

EleutherAI / dpsView on GitHubMore

• External links
• Readme badge

Data processing system for polyglot

☆93Sep 5, 2023Updated 2 years ago

Alternatives and similar repositories for dps

Users that are interested in dps are comparing it to the libraries listed below

• jason9693 / polyglot-finetuning-oslo

  View on GitHub
  ☆19Sep 20, 2022Updated 3 years ago
• hyunwoongko / beyond-lm

  View on GitHub
  Beyond LM: How can language model go forward in the future?
  ☆15Apr 30, 2023Updated 2 years ago
• jason9693 / oslo-kogpt-finetunig

  View on GitHub
  kogpt를 oslo로 파인튜닝하는 예제.
  ☆23Aug 26, 2022Updated 3 years ago
• MrBananaHuman / PangyoCorpora

  View on GitHub
  ☆36Oct 4, 2023Updated 2 years ago
• Beomi / easy-lm-trainer

  View on GitHub
  🤗 최소한의 세팅으로 LM을 학습하기 위한 샘플코드
  ☆59May 23, 2023Updated 2 years ago
• korean-named-entity / konec

  View on GitHub
  Korean Named Entity Corpus
  ☆25May 12, 2023Updated 2 years ago
• human-rights-corpus / HRC

  View on GitHub
  #인권코퍼스
  ☆31Oct 6, 2023Updated 2 years ago
• EleutherAI / oslo

  View on GitHub
  OSLO: Open Source for Large-scale Optimization
  ☆175Sep 9, 2023Updated 2 years ago
• hyunwoongko / python-mecab-kor

  View on GitHub
  Yet another python binding for mecab-ko
  ☆88May 16, 2023Updated 2 years ago
• hyunwoongko / pecab

  View on GitHub
  Pecab: Pure python Korean morpheme analyzer based on Mecab
  ☆172Apr 27, 2024Updated last year
• EleutherAI / polyglot

  View on GitHub
  Polyglot: Large Language Models of Well-balanced Competence in Multi-languages
  ☆484Aug 22, 2023Updated 2 years ago
• KU-HIAI / Ko-Gemma

  View on GitHub
  ☆34Feb 27, 2024Updated 2 years ago
• openkorpos / model-mecab

  View on GitHub
  MeCab model trained with OpenKorPos.
  ☆23Jun 19, 2022Updated 3 years ago
• BM-K / KoDiffCSE

  View on GitHub
  Difference-based Contrastive Learning for Korean Sentence Embeddings
  ☆23Updated this week
• MrBananaHuman / open-korean-instructions

  View on GitHub
  언어모델을 학습하기 위한 공개 한국어 instruction dataset들을 모아두었습니다.
  ☆19Jul 16, 2023Updated 2 years ago
• naver-ai / carecall-corpus

  View on GitHub
  CareCall for Seniors: Role Specified Open-Domain Dialogue dataset generated by leveraging LLMs (NAACL 2022).
  ☆60May 3, 2022Updated 3 years ago
• EleutherAI / polyglot-data

  View on GitHub
  data related codebase for polyglot project
  ☆19Mar 30, 2023Updated 2 years ago
• boychaboy / KOLD

  View on GitHub
  KOLD: Korean Offensive Language Dataset
  ☆81Nov 13, 2022Updated 3 years ago
• AIRC-KETI / ke-t5

  View on GitHub
  ☆197May 22, 2023Updated 2 years ago
• monologg / ko_lm_dataformat

  View on GitHub
  A utility for storing and reading files for Korean LM training 💾
  ☆35Oct 15, 2025Updated 4 months ago
• 42dot / 42dot_LLM

  View on GitHub
  42dot LLM consists of a pre-trained language model, 42dot LLM-PLM, and a fine-tuned model, 42dot LLM-SFT, which is trained to respond to …
  ☆131Mar 7, 2024Updated last year
• hyunwoongko / megatron-11b

  View on GitHub
  Megatron LM 11B on Huggingface Transformers
  ☆27Jul 11, 2021Updated 4 years ago
• hyunwoongko / bert2bert-summarization

  View on GitHub
  Abstractive summarization using Bert2Bert framework.
  ☆31Dec 5, 2020Updated 5 years ago
• Data-Intelligence-Lab / DEFT-korean-alpaca

  View on GitHub
  ☆23Oct 30, 2023Updated 2 years ago
• LG-NLP / KorWikiTableQuestions

  View on GitHub
  This repo is for Korean wiki table question answering datasets described in the paper of Korean-Specific Dataset for Table Question Answe…
  ☆91Oct 22, 2024Updated last year
• JoungheeKim / kor-spacing

  View on GitHub
  This is project for korean auto spacing
  ☆12Aug 3, 2020Updated 5 years ago
• monologg / korean-hate-speech-koelectra

  View on GitHub
  Bias, Hate classification with KoELECTRA 👿
  ☆27Jun 12, 2023Updated 2 years ago
• tunib-ai / KMWP

  View on GitHub
  Korean Math Word Problems
  ☆59Jan 14, 2022Updated 4 years ago
• Marker-Inc-Korea / KO-Platypus

  View on GitHub
  [KO-Platy🥮] Korean-Open-platypus를 활용하여 llama-2-ko를 fine-tuning한 KO-platypus model
  ☆73Aug 24, 2025Updated 6 months ago
• HAE-RAE / HAE-RAE-BENCH

  View on GitHub
  Benchmark in Korean Context
  ☆138Sep 26, 2023Updated 2 years ago
• ko-nlp / moducorpus-sanitizer

  View on GitHub
  모두의 말뭉치 데이터를 분석에 편리한 형태로 변환하는 기능을 제공합니다.
  ☆11Mar 2, 2022Updated 3 years ago
• songys / 2021Langcon

  View on GitHub
  ☆11Oct 3, 2021Updated 4 years ago
• MrBananaHuman / UnethicalQuestionsKor

  View on GitHub
  ☆19Oct 24, 2023Updated 2 years ago
• lassl / lassl

  View on GitHub
  Easy Language Model Pretraining leveraging Huggingface's Transformers and Datasets
  ☆130Nov 12, 2022Updated 3 years ago
• tunib-ai / DKTC

  View on GitHub
  Dataset of Korean Threatening Conversations
  ☆72Nov 1, 2022Updated 3 years ago
• sionic-ai / Data_KoSuperNI

  View on GitHub
  StrategyQA 데이터 세트 번역
  ☆23Apr 12, 2024Updated last year
• tunib-ai / artwork_captions

  View on GitHub
  Machine Generated Captions for Best Artworks
  ☆22Sep 21, 2022Updated 3 years ago
• passing2961 / KMRE

  View on GitHub
  Korean Moview Review Emotion (KMRE) Dataset
  ☆21Sep 7, 2020Updated 5 years ago
• hyunwoongko / kss

  View on GitHub
  KSS: Korean String processing Suite
  ☆468Nov 13, 2025Updated 3 months ago