OpenMatch / NeuScraper
[ACL 2024] This is the code repo for our ACL’24 paper "Cleaner Pretraining Corpus Curation with Neural Web Scraping".
☆224Updated 6 months ago
Alternatives and similar repositories for NeuScraper:
Users that are interested in NeuScraper are comparing it to the libraries listed below
- A lightweight script for processing HTML page to markdown format with support for code blocks☆79Updated 10 months ago
- Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]☆79Updated last month
- Meta-Chunking: Learning Efficient Text Segmentation via Logical Perception☆116Updated last month
- Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"☆221Updated 2 weeks ago
- 🚢 Data Toolkit for Sailor Language Models☆85Updated last week
- This is the code repo for our paper "Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents".☆104Updated 4 months ago
- ☆128Updated last month
- This repository presents the original implementation of LumberChunker: Long-Form Narrative Document Segmentation by André V. Duarte, João…☆56Updated 5 months ago
- Reformatted Alignment☆114Updated 5 months ago
- Mixture-of-Experts (MoE) Language Model☆184Updated 5 months ago
- Repo for NAACL 2025 Paper "Unfolding the Headline: Iterative Self-Questioning for News Retrieval and Timeline Summarization"☆249Updated last month
- [ICLR 2025] The official implementation of paper "ToolGen: Unified Tool Retrieval and Calling via Generation"☆126Updated last week
- Dense X Retrieval: What Retrieval Granularity Should We Use?☆149Updated last year
- Official repo for "Make Your LLM Fully Utilize the Context"☆251Updated 9 months ago
- ☆51Updated 7 months ago
- ☆107Updated 6 months ago
- LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA☆474Updated 2 months ago
- Evaluation for AI apps and agent☆36Updated last year
- This is a repository of RALM surveys containing a summary of state-of-the-art RAG and other technologies☆194Updated 8 months ago
- [NAACL 2024] Visually Guided Generative Text-Layout Pre-training for Document Intelligence☆140Updated 5 months ago
- Codebase accompanying the Summary of a Haystack paper.☆75Updated 5 months ago
- The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.☆69Updated last year
- a curated list of the role of small models in the LLM era☆91Updated 5 months ago
- Large Language Models Are Reasoning Teachers (ACL 2023)☆323Updated last year
- The official repository for the paper: Evaluation of Retrieval-Augmented Generation: A Survey.☆133Updated 4 months ago
- This repository contains the paper list for the paper: Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reaso…☆349Updated last year
- AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark☆127Updated 2 months ago
- [NeurlPS D&B 2024] Generative AI for Math: MathPile☆409Updated 4 months ago
- ☆120Updated 8 months ago