IBM / data-prep-kit
Open source project for data preparation of LLM application builders
☆264Updated this week
Related projects ⓘ
Alternatives and complementary repositories for data-prep-kit
- 🦄 Unitxt: a python library for getting data fired up and set for training and evaluation☆159Updated this week
- ☆196Updated this week
- Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs☆160Updated this week
- IBM development fork of https://github.com/huggingface/text-generation-inference☆57Updated last month
- Taxonomy tree that will allow you to create models tuned with your data☆196Updated this week
- Granite Snack Cookbook -- easily consumable recipes (python notebooks) that showcase the capabilities of the Granite models☆36Updated this week
- ☆131Updated 3 months ago
- Build Enterprise RAG (Retriver Augmented Generation) Pipelines to tackle various Generative AI use cases with LLM's by simply plugging co…☆108Updated 3 months ago
- Prompt Declaration Language (PDL) is a declarative prompt programming language.☆65Updated this week
- Let's build better datasets, together!☆202Updated 3 months ago
- awesome synthetic (text) datasets☆239Updated last week
- This project showcases an LLMOps pipeline that fine-tunes a small-size LLM model to prepare for the outage of the service LLM.☆288Updated 2 months ago
- Build document-native LLM applications☆50Updated last month
- RAGElo is a set of tools that helps you selecting the best RAG-based LLM agents by using an Elo ranker☆105Updated last week
- Python library for Synthetic Data Generation☆20Updated this week
- Automated knowledge graph creation SDK☆109Updated 4 months ago
- Tutorial for building LLM router☆157Updated 3 months ago
- Benchmark various LLM Structured Output frameworks: Instructor, Mirascope, Langchain, LlamaIndex, Fructose, Marvin, Outlines, etc on task…☆129Updated last month
- 🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.☆26Updated this week
- TapeAgents is a framework that facilitates all stages of the LLM Agent development lifecycle☆115Updated this week
- 🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.☆132Updated 3 months ago
- Model, Code & Data for the EMNLP'23 paper "Making Large Language Models Better Data Creators"☆115Updated last year
- This is the reproduction repository for my 🤗 Hugging Face blog post on synthetic data☆61Updated 8 months ago
- End-to-End LLM Guide☆97Updated 4 months ago
- ARAGOG- Advanced RAG Output Grading. Exploring and comparing various Retrieval-Augmented Generation (RAG) techniques on AI research paper…☆97Updated 6 months ago
- Manage scalable open LLM inference endpoints in Slurm clusters☆237Updated 3 months ago
- codebase release for EMNLP2023 paper publication☆19Updated 8 months ago
- Granite Code Cookbook☆13Updated this week
- Additional packages (components, document stores and the likes) to extend the capabilities of Haystack version 2.0 and onwards☆112Updated this week
- Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard a…☆764Updated this week