gioelecrispo / chunkipy
chunkipy is an extremely useful tool for segmenting long texts into smaller chunks, based on either a character or token count. With customizable chunk sizes and splitting strategies, chunkipy provides flexibility and control for various text processing tasks.
☆34Updated last year
Alternatives and similar repositories for chunkipy:
Users that are interested in chunkipy are comparing it to the libraries listed below
- 🧡 Hacker News summaries☆19Updated 10 months ago
- Mistral + Haystack: build RAG pipelines that rock 🤘☆100Updated last year
- Convert a web page to markdown☆63Updated 5 months ago
- ☆45Updated 10 months ago
- Chunk your text using gpt4o-mini more accurately☆43Updated 6 months ago
- Data extraction with LLM on CPU☆112Updated last year
- A framework that uses multi-agents to enable users to perform a systematic data science pipeline with just two inputs.☆38Updated 6 months ago
- High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡☆64Updated 3 months ago
- Writing Blog Posts with Generative Feedback Loops!☆47Updated 11 months ago
- 📝 Reference-Free automatic summarization evaluation with potential hallucination detection☆101Updated last year
- Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…☆48Updated 7 months ago
- ☆58Updated 3 months ago
- Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafte…☆60Updated 3 months ago
- Table detection with Florence.☆13Updated 7 months ago
- Data extraction with LLM on CPU☆67Updated last year
- A reimplementation of langgraph's customer support example in Rasa's CALM paradigm and a quantiative evaluation of the 2 approaches☆74Updated last week
- ☆18Updated 4 months ago
- Example demonstrating how to use gpt-4o-mini for fine-tuning☆25Updated 5 months ago
- ☆58Updated last year
- Dataset Viber is your chill repo for data collection, annotation and vibe checks.☆44Updated 5 months ago
- ☆65Updated 8 months ago
- ☆29Updated 11 months ago
- Explore the use of DSPy for extracting features from PDFs 🔎☆38Updated 11 months ago
- ☆20Updated last year
- A stable, fast and easy-to-use inference library with a focus on a sync-to-async API☆45Updated 4 months ago
- A tutorial on DSPy and whether automated prompt engineering lives up to the hype☆22Updated 9 months ago
- Using open source LLMs to build synthetic datasets for direct preference optimization☆57Updated 11 months ago
- A personal knowledge base that I can dump information to and help me learn☆24Updated 8 months ago
- langchain-streamlit demo with streaming llm, memory, and langsmith feedback☆18Updated last week
- ☆47Updated last year