OpenMatch / NeuScraperLinks
[ACL 2024] This is the code repo for our ACL’24 paper "Cleaner Pretraining Corpus Curation with Neural Web Scraping".
☆228Updated last year
Alternatives and similar repositories for NeuScraper
Users that are interested in NeuScraper are comparing it to the libraries listed below
Sorting:
- A lightweight script for processing HTML page to markdown format with support for code blocks☆80Updated last year
- Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]☆85Updated 7 months ago
- [ICML 2025] Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale☆262Updated 2 months ago
- Official repo for "LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs".☆240Updated last year
- Official repo for "Make Your LLM Fully Utilize the Context"☆253Updated last year
- ☆66Updated last year
- Deep Reasoning Translation (DRT) Project☆231Updated 2 weeks ago
- [Preprint] Learning to Filter Context for Retrieval-Augmented Generaton☆194Updated last year
- This is the code repo for our paper "Autonomously Knowledge Assimilation and Accommodation through Retrieval-Augmented Agents".☆107Updated 10 months ago
- [NeurlPS D&B 2024] Generative AI for Math: MathPile☆417Updated 5 months ago
- [ACL 2024] LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement☆189Updated last year
- This is the repository for our paper "INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning"☆204Updated 9 months ago
- Reformatted Alignment☆113Updated 11 months ago
- Evaluation tools for Retrieval-augmented Generation (RAG) methods.☆165Updated 9 months ago
- [ACL 2025] AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark☆155Updated last month
- Dense X Retrieval: What Retrieval Granularity Should We Use?☆161Updated last year
- XVERSE-65B: A multilingual large language model developed by XVERSE Technology Inc.☆140Updated last year
- 🚢 Data Toolkit for Sailor Language Models☆94Updated 6 months ago
- ☆95Updated 9 months ago
- ☆62Updated last year
- Benchmark baseline for retrieval qa applications☆116Updated last year
- [ACL24] Official repo for "Synthesizing Text-to-SQL Data from Weak and Strong LLMs"☆67Updated last year
- This is a repository of RALM surveys containing a summary of state-of-the-art RAG and other technologies☆201Updated last year
- ☆154Updated last year
- Code repo for "Agent Instructs Large Language Models to be General Zero-Shot Reasoners"☆116Updated last year
- Compress your input to ChatGPT or other LLMs, to let them process 2x more content and save 40% memory and GPU time.☆396Updated last year
- ☆319Updated 11 months ago
- Repository for “PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers”, NAACL24☆145Updated last year
- This repository presents the original implementation of LumberChunker: Long-Form Narrative Document Segmentation by André V. Duarte, João…☆76Updated 11 months ago
- LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA☆504Updated 8 months ago