EleutherAI / dps
Data processing system for polyglot
☆92Updated last year
Alternatives and similar repositories for dps:
Users that are interested in dps are comparing it to the libraries listed below
- Official repository for KoMT-Bench built by LG AI Research☆51Updated 5 months ago
- bpe based korean t5 model for text-to-text unified framework☆63Updated 9 months ago
- [Google Meet] MLLM Arxiv Casual Talk☆55Updated last year
- Easy Language Model Pretraining leveraging Huggingface's Transformers and Datasets☆127Updated 2 years ago
- KoCommonGEN v2: A Benchmark for Navigating Korean Commonsense Reasoning Challenges in Large Language Models☆25Updated 5 months ago
- 🤗 최소한의 세팅으로 LM을 학습하기 위한 샘플코드☆57Updated last year
- 한국어 LLM 리더보드 및 모델 성능/안전성 관리☆22Updated last year
- Korean Math Word Problems☆57Updated 3 years ago
- Korean-English Bilingual Electra Models☆109Updated 3 years ago
- CareCall for Seniors: Role Specified Open-Domain Dialogue dataset generated by leveraging LLMs (NAACL 2022).☆60Updated 2 years ago
- ☆32Updated last year
- 한국어 언어 모델 학습을 위한 프로젝트(Flax, Pytorch with Huggingface Accelerate)☆31Updated last year
- Adversarial Test Dataset for Korean Multi-turn Response Selection☆34Updated 3 years ago
- The code and models for "An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks" (AACL-IJCNLP 2020)☆116Updated 4 years ago
- Dataset of Korean Threatening Conversations☆69Updated 2 years ago
- ☆19Updated last year
- [Findings of NAACL2022] A Dog Is Passing Over The Jet? A Text-Generation Dataset for Korean Commonsense Reasoning and Evaluation☆28Updated 2 years ago
- Character-level Korean ELECTRA Model (음절 단위 한국어 ELECTRA)☆53Updated last year
- CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean☆42Updated last month
- ☆34Updated last year
- Kobart model on Huggingface transformers☆63Updated 2 years ago
- Performs benchmarking on two Korean datasets with minimal time and effort.☆27Updated 5 months ago
- Finetuning Pipeline☆90Updated 2 years ago
- BERTScore for Korean☆73Updated 11 months ago
- A utility for storing and reading files for Korean LM training 💾☆37Updated last year
- 자체 구축한 한국어 평가 데이터셋을 이용한 한국어 모델 평가☆31Updated 8 months ago
- ☆102Updated last year
- Code and data for "KoDialogBench: Evaluating Conversational Understanding of Language Models with Korean Dialogue Benchmark" (LREC-COLING…☆15Updated 10 months ago
- ☆87Updated 2 years ago