OpenMatch / NeuScraper
[ACL 2024] This is the code repo for our ACL’24 paper "Cleaner Pretraining Corpus Curation with Neural Web Scraping".
☆224Updated 6 months ago
Alternatives and similar repositories for NeuScraper:
Users that are interested in NeuScraper are comparing it to the libraries listed below
- A lightweight script for processing HTML page to markdown format with support for code blocks☆79Updated 11 months ago
- Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"☆226Updated 3 weeks ago
- Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]☆81Updated last month
- Meta-Chunking: Learning Efficient Text Segmentation via Logical Perception☆126Updated this week
- [ICLR 2025] The official implementation of paper "ToolGen: Unified Tool Retrieval and Calling via Generation"☆130Updated 2 weeks ago
- This is the code repo for our paper "Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents".☆104Updated 4 months ago
- [NeurlPS D&B 2024] Generative AI for Math: MathPile☆409Updated 4 months ago
- [Preprint] Learning to Filter Context for Retrieval-Augmented Generaton☆190Updated 11 months ago
- OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation☆68Updated this week
- Code implement reposity of Paper HiQA☆98Updated last week
- ☆51Updated 7 months ago
- Repo for NAACL 2025 Paper "Unfolding the Headline: Iterative Self-Questioning for News Retrieval and Timeline Summarization"☆256Updated last month
- DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought☆208Updated 2 months ago
- Codebase accompanying the Summary of a Haystack paper.☆75Updated 5 months ago
- [ACL24] Official repo for "Synthesizing Text-to-SQL Data from Weak and Strong LLMs"☆65Updated 7 months ago
- Dense X Retrieval: What Retrieval Granularity Should We Use?☆151Updated last year
- The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.☆69Updated last year
- ☆122Updated last year
- conversion doc(pdf/html/doc/docx/ppt/pptx)to markdown☆38Updated 7 months ago
- Mixture-of-Experts (MoE) Language Model☆185Updated 6 months ago
- LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA☆479Updated 2 months ago
- The official repository for the paper: Evaluation of Retrieval-Augmented Generation: A Survey.☆135Updated 5 months ago
- [EMNLP 2024: Demo Oral] RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation☆293Updated 4 months ago
- Source code of the paper: RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering [F…☆62Updated 9 months ago
- ☆309Updated 5 months ago
- Official repo for "LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs".☆227Updated 6 months ago
- Reformatted Alignment☆114Updated 5 months ago
- Scholar Copilot is an intelligent academic writing assistant that enhances the research writing process through AI-powered text completio…☆81Updated this week
- Evaluation for AI apps and agent☆36Updated last year