A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.
☆570Feb 14, 2026Updated 3 weeks ago
Alternatives and similar repositories for semchunk
Users that are interested in semchunk are comparing it to the libraries listed below
Sorting:
- Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from R…☆571Updated this week
- Tool to apply Legal Matter Specification Standard (LMSS) to documents☆12Aug 15, 2024Updated last year
- The code used to create and update the Open Australian Legal Embeddings, the first open-source embeddings of Australian legislative and j…☆13Feb 17, 2024Updated 2 years ago
- KL3M training data collection and preprocessing☆20Apr 14, 2025Updated 10 months ago
- High-performance retrieval engine for unstructured data☆1,562Nov 10, 2025Updated 3 months ago
- Check for data drift between two OpenAI multi-turn chat jsonl files.☆39Apr 11, 2024Updated last year
- Fast lexical search implementing BM25 in Python using Numpy, Numba and Scipy☆1,507Feb 17, 2026Updated 2 weeks ago
- Code for explaining and evaluating late chunking (chunked pooling)☆490Dec 23, 2024Updated last year
- A Python tool that uses AI to generate well-structured technical and educational articles from any topic. Features transparent reasoning,…☆17Apr 19, 2025Updated 10 months ago
- Fast, Accurate, Lightweight Python library to make State of the Art Embedding☆2,744Jan 9, 2026Updated last month
- Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-…☆3,868May 17, 2025Updated 9 months ago
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.☆1,602Dec 20, 2025Updated 2 months ago
- I will be adding different kind of opensource data extraction tools code using python☆10Nov 15, 2024Updated last year
- Fast Multimodal Semantic Deduplication & Filtering☆892Jan 20, 2026Updated last month
- 🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines☆3,809Updated this week
- Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali☆2,688Feb 5, 2026Updated last month
- Fast State-of-the-Art Static Embeddings☆2,007Feb 28, 2026Updated last week
- A Python library to chunk/group your texts based on semantic similarity.☆103Jul 11, 2024Updated last year
- ☆249Jun 10, 2025Updated 8 months ago
- A curated list of materials on AI guardrails☆46Jun 3, 2025Updated 9 months ago
- Use late-interaction multi-modal models such as ColPali in just a few lines of code.☆845Jan 28, 2025Updated last year
- Distilabel is a framework for synthetic data and AI feedback for engineers who need fast, reliable and scalable pipelines based on verifi…☆3,108Feb 23, 2026Updated last week
- A system for agentic LLM-powered data processing and ETL☆3,669Feb 2, 2026Updated last month
- chunkipy is an extremely useful tool for segmenting long texts into smaller chunks, based on either a character or token count. With cust…☆37Dec 18, 2025Updated 2 months ago
- [ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings☆2,023Jan 15, 2025Updated last year
- Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean…☆14,135Updated this week
- A framework for evaluating function calls made by LLMs☆40Jul 23, 2024Updated last year
- Extract structured text from pdfs quickly☆670Jun 11, 2025Updated 8 months ago
- 💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows☆12,247Feb 25, 2026Updated last week
- Structured Outputs☆13,488Updated this week
- The Batched API provides a flexible and efficient way to process multiple requests in a batch, with a primary focus on dynamic batching o…☆159Jul 14, 2025Updated 7 months ago
- Efficient few-shot learning with cross-encoders.☆63Feb 16, 2024Updated 2 years ago
- A dataset for pretraining language models targeted for legal tasks.☆143Jun 30, 2022Updated 3 years ago
- ExtractThinker is a Document Intelligence library for LLMs, offering ORM-style interaction for flexible and powerful document workflows.☆1,489Aug 27, 2025Updated 6 months ago
- [NeurIPS 2024] The official implementation of "Image Copy Detection for Diffusion Models"☆18Oct 1, 2024Updated last year
- 🔌 Want one client library for all your embeddings? 💙 Choose Catsu! 🐱☆60Feb 20, 2026Updated 2 weeks ago
- The code used to create and update the Open Australian Legal Corpus, the first and only multijurisdictional open corpus of Australian leg…☆106May 26, 2025Updated 9 months ago
- Dataset Viber is your chill repo for data collection, annotation and vibe checks.☆47Sep 5, 2024Updated last year
- Retrieval and Retrieval-augmented LLMs☆11,352Dec 15, 2025Updated 2 months ago