PKU-ONELab / Themis
The official repository for our NLG evaluation LLM Themis and the paper Themis: Towards Flexible and Interpretable NLG Evaluation.
☆17Updated 4 months ago
Related projects ⓘ
Alternatives and complementary repositories for Themis
- GSM-Plus: Data, Code, and Evaluation for Enhancing Robust Mathematical Reasoning in Math Word Problems.☆47Updated 4 months ago
- Towards Systematic Measurement for Long Text Quality☆28Updated 2 months ago
- First explanation metric (diagnostic report) for text generation evaluation☆61Updated 4 months ago
- The code implementation of the EMNLP2022 paper: DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Gene…☆25Updated last year
- 🩺 A collection of ChatGPT evaluation reports on various bechmarks.☆48Updated last year
- ☆36Updated 10 months ago
- Code and data accompanying the paper "TRUE: Re-evaluating Factual Consistency Evaluation".☆71Updated this week
- Do Large Language Models Know What They Don’t Know?☆85Updated 2 weeks ago
- ☆35Updated last year
- ACL'23: Unified Demonstration Retriever for In-Context Learning☆33Updated 11 months ago
- [ICLR'24 spotlight] Tool-Augmented Reward Modeling☆36Updated 8 months ago
- This project maintains a reading list for general text generation tasks☆65Updated 2 years ago
- ☆31Updated 7 months ago
- Technical Report: Is ChatGPT a Good NLG Evaluator? A Preliminary Study☆42Updated last year
- Github repository for "FELM: Benchmarking Factuality Evaluation of Large Language Models" (NeurIPS 2023)☆54Updated 10 months ago
- Code base of In-Context Learning for Dialogue State tracking☆44Updated last year
- [ICML'2024] Can AI Assistants Know What They Don't Know?☆70Updated 9 months ago
- Calculate the probability of a paper being accepted by EMNLP2023 based on score distribution of ACL2023.☆14Updated last year
- An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs).☆43Updated 3 months ago
- ☆16Updated 8 months ago
- Code and data for the paper "Can Large Language Models Understand Real-World Complex Instructions?"(AAAI2024)☆44Updated 7 months ago
- Evaluate the Quality of Critique☆35Updated 5 months ago
- ☆47Updated 3 months ago
- [ACL 2024] MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues☆51Updated 3 months ago
- ☆39Updated 7 months ago
- This is the repository for paper "CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models"☆22Updated last year
- [EMNLP 2023] ALCUNA: Large Language Models Meet New Knowledge☆25Updated last year
- Resources for our ACL 2023 paper: Distilling Script Knowledge from Large Language Models for Constrained Language Planning☆35Updated last year
- ☆65Updated 6 months ago
- Code for M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models☆22Updated 3 months ago