Alternatives and similar repositories for trafilatura

Users that are interested in trafilatura are comparing it to the libraries listed below

• pemistahl / lingua-py
  The most accurate natural language detection library for Python, suitable for short text and mixed-language text
  ☆1,625Updated 2 months ago
• matthewwithanm / python-markdownify
  Convert HTML to Markdown
  ☆2,045Updated 2 months ago
• deedy5 / ddgs
  DDGS | Dux Distributed Global Search. A metasearch library that aggregates results from diverse web search services
  ☆2,101Updated last month
• neuml / txtai
  💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
  ☆12,071Updated last week
• explosion / spacy-llm
  🦙 Integrating LLMs into structured NLP pipelines
  ☆1,362Updated last year
• scrapinghub / article-extraction-benchmark
  Article extraction benchmark: dataset and evaluation scripts
  ☆351Updated 4 months ago
• AndyTheFactory / newspaper4k
  📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
  ☆992Updated 2 weeks ago
• qdrant / fastembed
  Fast, Accurate, Lightweight Python library to make State of the Art Embedding
  ☆2,665Updated 3 weeks ago
• eth-sri / lmql
  A language for constraint-guided and efficient LLM programming.
  ☆4,139Updated 8 months ago
• eyurtsev / kor
  LLM(😽)
  ☆1,698Updated 11 months ago
• xhluca / bm25s
  Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy
  ☆1,465Updated last month
• Filimoa / open-parse
  Improved file parsing for LLM’s
  ☆3,151Updated last year
• miso-belica / jusText
  Heuristic based boilerplate removal tool
  ☆811Updated 11 months ago
• deepdoctection / deepdoctection
  A Repo For Document AI
  ☆3,127Updated 2 weeks ago
• stanford-futuredata / ColBERT
  ColBERT: state-of-the-art neural search (SIGIR'20, TACL'21, NeurIPS'21, NAACL'22, CIKM'22, ACL'23, EMNLP'23)
  ☆3,765Updated 3 months ago
• huggingface / setfit
  Efficient few-shot learning with Sentence Transformers
  ☆2,673Updated last month
• embeddings-benchmark / mteb
  MTEB: Massive Text Embedding Benchmark
  ☆3,095Updated this week
• buriy / python-readability
  fast python port of arc90's readability tool, updated to match latest readability.js!
  ☆2,884Updated this week
• segment-any-text / wtpsplit
  Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.
  ☆1,232Updated last week
• urchade / GLiNER
  Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts) @ NAACL 2024
  ☆2,737Updated this week
• jackmpcollins / magentic
  Seamlessly integrate LLMs as Python functions
  ☆2,386Updated 2 months ago
• dottxt-ai / outlines
  Structured Outputs
  ☆13,322Updated this week
• Unstructured-IO / unstructured
  Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean…
  ☆13,715Updated this week
• michaelfeil / infinity
  Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali
  ☆2,642Updated last month
• Alir3z4 / html2text
  Convert HTML to Markdown-formatted text.
  ☆2,120Updated 3 months ago
• AnswerDotAI / RAGatouille
  Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…
  ☆3,839Updated 8 months ago
• aurelio-labs / semantic-router
  Superfast AI decision making and intelligent processing of multi-modal data.
  ☆3,210Updated 2 months ago
• xlang-ai / instructor-embedding
  [ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings
  ☆2,022Updated last year
• microsoft / LLMLingua
  [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which ach…
  ☆5,788Updated 3 months ago
• dragnet-org / dragnet
  Just the facts -- web page content extraction
  ☆1,279Updated 6 months ago