segment-any-text / wtpsplitLinks
Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.
β1,159Updated 3 months ago
Alternatives and similar repositories for wtpsplit
Users that are interested in wtpsplit are comparing it to the libraries listed below
Sorting:
- ππ―pySBD (Python Sentence Boundary Disambiguation) is a rule-based sentence boundary detection that works out-of-the-box.β878Updated last year
- Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipyβ1,346Updated last month
- The most accurate natural language detection library for Python, suitable for short text and mixed-language textβ1,510Updated 4 months ago
- π¦ Integrating LLMs into structured NLP pipelinesβ1,319Updated 9 months ago
- Bringing BERT into modernity via both architecture changes and scalingβ1,537Updated 3 months ago
- NeuSpell: A Neural Spelling Correction Toolkitβ695Updated 2 years ago
- SGPT: GPT Sentence Embeddings for Semantic Searchβ874Updated last year
- Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts) @ NAACL 2024β2,399Updated this week
- Open neural machine translation models and web servicesβ735Updated 3 months ago
- State-of-the-art LLM-based translation models.β557Updated 6 months ago
- Training open neural machine translation modelsβ379Updated 6 months ago
- SpanMarker for Named Entity Recognitionβ453Updated 9 months ago
- Efficient few-shot learning with Sentence Transformersβ2,574Updated 2 months ago
- A Collection of BM25 Algorithms in Pythonβ1,250Updated last year
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.β1,547Updated 4 months ago
- SPLADE: sparse neural search (SIGIR21, SIGIR22)β919Updated last year
- Late Interaction Models Training & Retrievalβ610Updated 2 weeks ago
- β543Updated last year
- Fast Semantic Text Deduplication & Filteringβ814Updated last week
- String-to-String Algorithms for Natural Language Processingβ555Updated last year
- Easily embed, cluster and semantically label text datasetsβ578Updated last year
- β‘ boost inference speed of T5 models by 5x & reduce the model size by 3x.β587Updated 2 years ago
- π Process PDFs, Word documents and more with spaCyβ767Updated 7 months ago
- Tools to download and cleanup Common Crawl dataβ1,026Updated 2 years ago
- FastFit β‘ When LLMs are Unfit Use FastFit β‘ Fast and Effective Text Classification with Many Classesβ211Updated 3 weeks ago
- β369Updated last year
- Lite & Super-fast re-ranking for your search & retrieval pipelines. Supports SoTA Listwise and Pairwise reranking based on LLMs and croβ¦β864Updated 3 weeks ago
- Train and Infer Powerful Sentence Embeddings with AnglE | π₯ SOTA on STS and MTEB Leaderboardβ555Updated 6 months ago
- A Neural Framework for MT Evaluationβ670Updated last month
- π¬ Language Identification with Support for More Than 2000 Labels -- EMNLP 2023β162Updated 4 months ago