AkariAsai / ScholarQABench
This repository contains ScholarQABench data and evaluation pipeline.
☆66Updated last month
Alternatives and similar repositories for ScholarQABench:
Users that are interested in ScholarQABench are comparing it to the libraries listed below
- [EMNLP 2024] A Retrieval Benchmark for Scientific Literature Search☆70Updated 3 months ago
- Official implementation of the ACL 2024: Scientific Inspiration Machines Optimized for Novelty☆77Updated 11 months ago
- [ICLR'25] ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery☆67Updated 2 weeks ago
- Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data☆35Updated 3 weeks ago
- Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments (EMNLP'2024)☆35Updated 2 months ago
- Codebase accompanying the Summary of a Haystack paper.☆75Updated 5 months ago
- [arXiv preprint] Official Repository for "Evaluating Language Models as Synthetic Data Generators"☆34Updated 3 months ago
- This repository contains expert evaluation interface and data evaluation script for the OpenScholar project.☆23Updated 3 months ago
- This is the code repo for our paper "Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents".☆104Updated 4 months ago
- [ACL 2024] <Large Language Models for Automated Open-domain Scientific Hypotheses Discovery>. It has also received the best poster award …☆39Updated 4 months ago
- ☆67Updated last year
- ☆119Updated 5 months ago
- Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]☆136Updated 4 months ago
- BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval☆85Updated last month
- Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators (Liu et al.; COLM 2024)☆44Updated last month
- Framework and toolkits for building and evaluating collaborative agents that can work together with humans.☆68Updated 3 weeks ago
- A package to generate summaries of long-form text and evaluate the coherence of these summaries. Official package for our ICLR 2024 paper…☆115Updated 5 months ago
- This repository contains the joint use of CPO and SimPO method for better reference-free preference learning methods.☆51Updated 7 months ago
- ReBase: Training Task Experts through Retrieval Based Distillation☆28Updated last month
- Code/data for MARG (multi-agent review generation)☆41Updated 3 months ago
- [ICLR 2025] InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales☆77Updated last month
- Code release for "SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers" [NeurIPS D&B, 2024]☆54Updated 2 months ago
- The code for the paper: "Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models"☆55Updated 8 months ago
- ☆39Updated 7 months ago
- [Data + code] ExpertQA : Expert-Curated Questions and Attributed Answers☆125Updated 11 months ago
- [ACL'24] Code and data of paper "When is Tree Search Useful for LLM Planning? It Depends on the Discriminator"☆54Updated last year
- 🚢 Data Toolkit for Sailor Language Models☆87Updated 2 weeks ago
- Dataset and evaluation suite enabling LLM instruction-following for scientific literature understanding.☆35Updated this week