chatnoir-eu / chatnoir-resiliparseLinks
A robust web archive analytics toolkit
☆119Updated 2 weeks ago
Alternatives and similar repositories for chatnoir-resiliparse
Users that are interested in chatnoir-resiliparse are comparing it to the libraries listed below
Sorting:
- Pretraining Efficiently on S2ORC!☆171Updated last year
- The pipeline for the OSCAR corpus☆173Updated last year
- This project studies the performance and robustness of language models and task-adaptation methods.☆154Updated last year
- Python tools for processing the stackexchange data dumps into a text dataset for Language Models☆82Updated last year
- What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets☆223Updated 11 months ago
- ☆200Updated 4 months ago
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆188Updated 3 months ago
- minimal pytorch implementation of bm25 (with sparse tensors)☆104Updated last year
- Inspired by google c4, here is a series of colossal clean data cleaning scripts focused on CommonCrawl data processing. Including Chinese…☆131Updated 2 years ago
- ☆39Updated last year
- ☆16Updated 10 months ago
- Baguetter is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, imp…☆190Updated last year
- ☆78Updated 10 months ago
- A large-scale information-rich web dataset, featuring millions of real clicked query-document labels☆345Updated 10 months ago
- 💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆163Updated 4 months ago
- XTR: Rethinking the Role of Token Retrieval in Multi-Vector Retrieval☆58Updated last year
- ☆52Updated 3 months ago
- [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.☆110Updated last year
- ☆62Updated last year
- ☆91Updated 3 years ago
- Model implementation for the contextual embeddings project☆36Updated 4 months ago
- ☆72Updated 2 months ago
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆49Updated last year
- Manage scalable open LLM inference endpoints in Slurm clusters☆273Updated last year
- CLIR version of ColBERT☆74Updated 4 months ago
- One-stop shop for running and fine-tuning transformer-based language models for retrieval☆59Updated last week
- Pipeline for pulling and processing online language model pretraining data from the web☆178Updated 2 years ago
- multimodal document analysis☆166Updated last year
- Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking☆25Updated 6 months ago
- ☆46Updated 3 years ago