segment-any-text / wtpsplitLinks
Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.
☆1,109Updated last month
Alternatives and similar repositories for wtpsplit
Users that are interested in wtpsplit are comparing it to the libraries listed below
Sorting:
- Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy☆1,267Updated 2 months ago
- 🐍💯pySBD (Python Sentence Boundary Disambiguation) is a rule-based sentence boundary detection that works out-of-the-box.☆864Updated 11 months ago
- The most accurate natural language detection library for Python, suitable for short text and mixed-language text☆1,450Updated last month
- Open neural machine translation models and web services☆713Updated last month
- Bringing BERT into modernity via both architecture changes and scaling☆1,473Updated last month
- NeuSpell: A Neural Spelling Correction Toolkit☆696Updated 2 years ago
- SPLADE: sparse neural search (SIGIR21, SIGIR22)☆884Updated last year
- Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts) @ NAACL 2024☆2,206Updated this week
- Training open neural machine translation models☆369Updated 4 months ago
- State-of-the-art LLM-based translation models.☆548Updated 3 months ago
- A Collection of BM25 Algorithms in Python☆1,217Updated 10 months ago
- 🦙 Integrating LLMs into structured NLP pipelines☆1,289Updated 7 months ago
- SpanMarker for Named Entity Recognition☆444Updated 7 months ago
- SGPT: GPT Sentence Embeddings for Semantic Search☆869Updated last year
- Easily embed, cluster and semantically label text datasets☆560Updated last year
- Train and Infer Powerful Sentence Embeddings with AnglE | 🔥 SOTA on STS and MTEB Leaderboard☆550Updated 4 months ago
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.☆1,505Updated 2 months ago
- Late Interaction Models Training & Retrieval☆521Updated 3 weeks ago
- Efficient few-shot learning with Sentence Transformers☆2,534Updated 3 months ago
- String-to-String Algorithms for Natural Language Processing☆551Updated last year
- A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.☆1,908Updated 2 months ago
- ⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍☆579Updated 2 weeks ago
- A Neural Framework for MT Evaluation☆642Updated this week
- ☆1,232Updated last year
- Lite & Super-fast re-ranking for your search & retrieval pipelines. Supports SoTA Listwise and Pairwise reranking based on LLMs and cro…☆842Updated last month
- 80x faster and 95% accurate language identification with Fasttext☆160Updated last year
- Easy to use, state-of-the-art Neural Machine Translation for 100+ languages☆1,240Updated last year
- All-in-one text de-duplication☆706Updated 2 weeks ago
- Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from R…☆461Updated this week
- ☆535Updated last year