Alternatives and similar repositories for redpajama

Users that are interested in redpajama are comparing it to the libraries listed below

• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆30Updated 9 months ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆47Updated last year
• johnrobinsn / alpaca_lora_30b_4bit
  ☆18Updated 2 years ago
• pacman100 / peft-codegen-25
  ☆23Updated last year
• explodinggradients / Funtuner
  Supervised instruction finetuning for LLM with HF trainer and Deepspeed
  ☆35Updated last year
• ConiferLabsWA / flan-ul2-dolly
  ☆34Updated 2 years ago
• hamelsmu / ft-drift
  Check for data drift between two OpenAI multi-turn chat jsonl files.
  ☆37Updated last year
• deep-diver / TFX-WandB
  ☆14Updated last year
• CarperAI / treasure_trove
  ☆22Updated last year
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• NirantK / agentai
  Text to Python Objects via a LLM Function Call
  ☆58Updated last year
• modal-labs / ci-on-modal
  A sample pattern for running CI tests on Modal
  ☆18Updated 2 months ago
• philschmid / optimum-static-quantization
  ☆28Updated 2 years ago
• dm4ml / motion
  Framework for building and maintaining self-updating prompts for LLMs
  ☆63Updated last year
• hundredblocks / large-model-parallelism
  Functional local implementations of main model parallelism approaches
  ☆95Updated 2 years ago
• mlabonne / chessllm
  ☆38Updated last year
• ChrisHayduk / QLoRA-for-MLM
  QLoRA for Masked Language Modeling
  ☆22Updated last year
• nateraw / huggingface-datasets-converter
  Scripts to convert datasets from various sources to Hugging Face Datasets.
  ☆57Updated 2 years ago
• nateraw / lambdacloud
  An unofficial Python client library for Lambda Lab's Cloud Computing Platform
  ☆12Updated 2 years ago
• allenai / EmbeddingRecycling
  Embedding Recycling for Language models
  ☆38Updated last year
• bhavsarpratik / semantic-search
  [WIP] Behold, semantic-search, built over sentence-transformers to make it easy for search engineers to evaluate, optimise and deploy mod…
  ☆15Updated 2 years ago
• deep-diver / Continuous-Adaptation-for-Machine-Learning-System-to-Data-Changes
  https://blog.tensorflow.org/2021/12/continuous-adaptation-for-machine.html
  ☆29Updated 3 years ago
• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆33Updated last month
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated last year
• kaiokendev / cutoff-len-is-context-len
  Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit
  ☆63Updated 2 years ago
• CarperAI / decontamination
  This repository contains code for cleaning your training data of benchmark data to help combat data snooping.
  ☆25Updated 2 years ago
• weaviate-tutorials / Hurricane
  Writing Blog Posts with Generative Feedback Loops!
  ☆48Updated last year
• Upaya07 / NeurIPS-llm-efficiency-challenge
  Code for NeurIPS LLM Efficiency Challenge
  ☆59Updated last year
• deep-diver / LLM-Pref-Mark-UI
  ☆37Updated 2 years ago
• deep-diver / complete-mlops-system-workflow
  ☆17Updated 2 years ago