benbrandt / text-splitter
Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.
☆245Updated this week
Related projects: ⓘ
- Neural search for web-sites, docs, articles - online!☆124Updated last year
- A Python library to chunk/group your texts based on semantic similarity.☆77Updated 2 months ago
- Library for generating vector embeddings, reranking in Rust☆250Updated 3 weeks ago
- ☆160Updated 2 months ago
- In-Context Learning for eXtreme Multi-Label Classification (XMC) using only a handful of examples.☆362Updated 7 months ago
- A fast and lightweight pure Python library for splitting text into semantically meaningful chunks.☆142Updated 2 months ago
- A minimalist yet highly performant, lightweight, lightning fast, multisource, multimodal and local embedding solution, built in rust.☆220Updated this week
- ☆150Updated this week
- Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langchain, LlamaIndex, Fructose, Marvin, Outlines, etc on task…☆117Updated 3 weeks ago
- A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.☆845Updated last week
- Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.☆250Updated this week
- Formatron empowers everyone to control the format of language models' output with minimal overhead.☆116Updated this week
- Fast lexical search library implementing BM25 in Python using Numpy and Scipy☆770Updated this week
- ☆134Updated 6 months ago
- This package, developed as part of our research detailed in the Chroma Technical Report, provides tools for text chunking and evaluation.…☆91Updated 2 months ago
- Use late-interaction multi-modal models such as ColPali in just a few lines of code.☆330Updated this week
- The code used to train and run inference with the ColPali architecture.☆512Updated this week
- Efficient vector database for hundred millions of embeddings.☆196Updated 4 months ago
- LLM Orchestrator built in Rust☆261Updated 6 months ago
- Building AI agents, atomically☆344Updated this week
- ☆478Updated 3 weeks ago
- FastAPI wrapper around DSPy☆201Updated 6 months ago
- A simple Python sandbox for helpful LLM data agents☆143Updated 3 months ago
- Fast, Accurate, Lightweight Python library to make State of the Art Embedding☆1,371Updated this week
- Self-hosted web UI for Qdrant☆210Updated this week
- AgentSearch is a framework for powering search agents and enabling customizable local search.☆432Updated 4 months ago
- SUQL: Conversational Search over Structured and Unstructured Data with LLMs☆194Updated 3 weeks ago
- Lite & Super-fast re-ranking for your search & retrieval pipelines. Supports SoTA Listwise and Pairwise reranking based on LLMs and cro…☆571Updated 2 weeks ago
- Super performant RAG pipelines for AI apps. Summarization, Retrieve/Rerank and Code Interpreters in one simple API.☆335Updated 4 months ago
- High-performance retrieval engine for unstructured data☆778Updated this week