onesuper / HuggingFace-Datasets-Text-Quality-Analysis
Retrieves parquet files from Hugging Face, identifies and quantifies junky data, duplication, contamination, and biased content in dataset using pandas
☆54Updated last year
Alternatives and similar repositories for HuggingFace-Datasets-Text-Quality-Analysis:
Users that are interested in HuggingFace-Datasets-Text-Quality-Analysis are comparing it to the libraries listed below
- Light local website for displaying performances from different chat models.☆86Updated last year
- A Multi-Turn Dialogue Corpus based on Alpaca Instructions☆169Updated last year
- MultilingualShareGPT, the free multi-language corpus for LLM training☆73Updated 2 years ago
- ☆128Updated last year
- 中文大语言模型评测第二期☆70Updated last year
- make LLM easier to use☆59Updated last year
- Summarize all open source Large Languages Models and low-cost replication methods for Chatgpt.☆137Updated last year
- ☆31Updated last year
- [EMNLP 2023 Demo] CLEVA: Chinese Language Models EVAluation Platform☆62Updated 3 weeks ago
- ☆172Updated 2 years ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆41Updated last year
- Leveraging passage embeddings for efficient listwise reranking with large language models.☆40Updated 4 months ago
- MEASURING MASSIVE MULTITASK CHINESE UNDERSTANDING☆87Updated last year
- 1.4B sLLM for Chinese and English - HammerLLM🔨☆44Updated last year
- Unofficial implementation of AlpaGasus☆90Updated last year
- ⏳ ChatLog: Recording and Analysing ChatGPT Across Time☆97Updated 10 months ago
- ☆23Updated last year
- Gaokao Benchmark for AI☆108Updated 2 years ago
- Counting-Stars (★)☆82Updated 7 months ago
- ☆97Updated last year
- Source code for ACL 2023 paper Decoder Tuning: Efficient Language Understanding as Decoding☆49Updated last year
- OpenLLMDE: An open source data engineering framework for LLMs☆17Updated last year
- This repository contains the code to train flan t5 with alpaca instructions and low rank adaptation.☆51Updated last year
- ☆143Updated 9 months ago
- LLaMa Tuning with Stanford Alpaca Dataset using Deepspeed and Transformers☆51Updated 2 years ago
- ☆104Updated last year
- ☆124Updated last year
- 中文大语言模型评测第一期☆108Updated last year
- AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark☆137Updated 4 months ago
- YuLan-IR: Information Retrieval Boosted LMs☆218Updated last year