OpenMatch / NeuScraper
[ACL 2024] This is the code repo for our ACL’24 paper "Cleaner Pretraining Corpus Curation with Neural Web Scraping".
☆221Updated 5 months ago
Alternatives and similar repositories for NeuScraper:
Users that are interested in NeuScraper are comparing it to the libraries listed below
- Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"☆209Updated 3 months ago
- A lightweight script for processing HTML page to markdown format with support for code blocks☆78Updated 9 months ago
- AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark☆123Updated last month
- [ICLR 2025] The official implementation of paper "ToolGen: Unified Tool Retrieval and Calling via Generation"☆121Updated last month
- Meta-Chunking: Learning Efficient Text Segmentation via Logical Perception☆107Updated last week
- [EMNLP 2024: Demo Oral] RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation☆283Updated 3 months ago
- This is the code repo for our paper "Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents".☆102Updated 3 months ago
- ☆51Updated 6 months ago
- ☆116Updated 2 months ago
- Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]☆76Updated last week
- 🚢 Data Toolkit for Sailor Language Models☆85Updated last month
- A Comprehensive Benchmark for Document Parsing and Evaluation☆212Updated 2 weeks ago
- Dense X Retrieval: What Retrieval Granularity Should We Use?☆143Updated last year
- DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought☆201Updated last month
- Reformatted Alignment☆113Updated 4 months ago
- Repository for “PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers”, NAACL24☆132Updated 7 months ago
- Mixture-of-Experts (MoE) Language Model☆183Updated 4 months ago
- This repository contains the paper list for the paper: Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reaso…☆347Updated last year
- The official repository for the paper: Evaluation of Retrieval-Augmented Generation: A Survey.☆118Updated 3 months ago
- Evaluation tools for Retrieval-augmented Generation (RAG) methods.☆145Updated 2 months ago
- [Preprint] Learning to Filter Context for Retrieval-Augmented Generaton☆188Updated 9 months ago
- Comprehensive benchmark for RAG☆103Updated 2 months ago
- This repository presents the original implementation of LumberChunker: Long-Form Narrative Document Segmentation by André V. Duarte, João…☆50Updated 3 months ago
- Official repo for "LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs".☆213Updated 5 months ago
- ☆301Updated 4 months ago
- ☆165Updated last month
- Codebase accompanying the Summary of a Haystack paper.☆74Updated 4 months ago
- Build & Optimize your RAG.☆136Updated this week
- Official Implementation of "Multi-Head RAG: Solving Multi-Aspect Problems with LLMs"☆194Updated 2 months ago
- LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA☆455Updated last month