Alternatives and similar repositories for chatnoir-resiliparse

Users that are interested in chatnoir-resiliparse are comparing it to the libraries listed below

• allenai / peS2o
  Pretraining Efficiently on S2ORC!
  ☆164Updated 8 months ago
• oscar-project / ungoliant
  The pipeline for the OSCAR corpus
  ☆171Updated last year
• lemurproject / ClueWeb22
  ☆16Updated 7 months ago
• EleutherAI / stackexchange-dataset
  Python tools for processing the stackexchange data dumps into a text dataset for Language Models
  ☆83Updated last year
• allenai / wimbd
  What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets
  ☆221Updated 8 months ago
• castorini / umbrela
  ☆51Updated 5 months ago
• allenai / catwalk
  This project studies the performance and robustness of language models and task-adaptation methods.
  ☆150Updated last year
• sileod / tasksource
  Datasets collection and preprocessings framework for NLP extreme multitask learning
  ☆184Updated last week
• vespa-engine / pyvespa
  Python API for https://vespa.ai, the open big data serving engine
  ☆130Updated last week
• webis-de / lightning-ir
  One-stop shop for running and fine-tuning transformer-based language models for retrieval
  ☆57Updated last week
• EleutherAI / openwebtext2
  ☆90Updated 3 years ago
• bloomberg / minilmv2.bb
  Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)
  ☆61Updated 2 years ago
• Snowflake-Labs / arctic-embed
  ☆74Updated 7 months ago
• jxmorris12 / bm25_pt
  minimal pytorch implementation of bm25 (with sparse tensors)
  ☆103Updated last year
• sebastian-hofstaetter / colberter
  ☆47Updated 3 years ago
• allenai / bff
  ☆38Updated last year
• illuin-tech / contextual-embeddings
  Model implementation for the contextual embeddings project
  ☆33Updated last month
• terrierteam / pyterrier_colbert
  ☆86Updated 3 months ago
• webis-de / rank-distillm
  Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking
  ☆23Updated 3 months ago
• shjwudp / c4-dataset-script
  Inspired by google c4, here is a series of colossal clean data cleaning scripts focused on CommonCrawl data processing. Including Chinese…
  ☆127Updated 2 years ago
• google-deepmind / xtr
  XTR: Rethinking the Role of Token Retrieval in Multi-Vector Retrieval
  ☆52Updated last year
• allenai / mmda
  multimodal document analysis
  ☆166Updated last year
• terrierteam / ir_measures
  provides a common interface to many IR measure tools
  ☆87Updated 2 months ago
• hltcoe / ColBERT-X
  CLIR version of ColBERT
  ☆70Updated 3 weeks ago
• mixedbread-ai / baguetter
  Baguetter is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, imp…
  ☆185Updated 10 months ago
• kddubey / cappr
  Completion After Prompt Probability. Make your LLM make a choice
  ☆79Updated 8 months ago
• AmenRa / indxr
  A Python utility for indexing file lines. Best demo honourable mention at ECIR 2024.
  ☆23Updated last year
• huggingface / llm-swarm
  Manage scalable open LLM inference endpoints in Slurm clusters
  ☆265Updated last year
• microsoft / MS-MARCO-Web-Search
  A large-scale information-rich web dataset, featuring millions of real clicked query-document labels
  ☆337Updated 7 months ago
• flairNLP / fabricator
  [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.
  ☆108Updated last year