Alternatives and similar repositories for redpajama:

Users that are interested in redpajama are comparing it to the libraries listed below

• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆29Updated 4 months ago
• hamelsmu / ft-drift
  Check for data drift between two OpenAI multi-turn chat jsonl files.
  ☆37Updated 10 months ago
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated last year
• Upaya07 / NeurIPS-llm-efficiency-challenge
  Code for NeurIPS LLM Efficiency Challenge
  ☆55Updated 10 months ago
• NirantK / agentai
  Text to Python Objects via a LLM Function Call
  ☆56Updated 10 months ago
• ChrisHayduk / QLoRA-for-MLM
  QLoRA for Masked Language Modeling
  ☆21Updated last year
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆45Updated last year
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• modal-labs / ci-on-modal
  A sample pattern for running CI tests on Modal
  ☆14Updated 4 months ago
• pacman100 / peft-codegen-25
  ☆24Updated last year
• philschmid / optimum-static-quantization
  ☆28Updated last year
• CarperAI / treasure_trove
  ☆22Updated last year
• ConiferLabsWA / flan-ul2-dolly
  ☆34Updated last year
• ashvardanian / usearch-binary
  Binary vector search example using Unum's USearch engine and pre-computed Wikipedia embeddings from Co:here and MixedBread
  ☆18Updated 10 months ago
• johnrobinsn / alpaca_lora_30b_4bit
  ☆18Updated last year
• davanstrien / data-for-fine-tuning-llms
  ☆76Updated 8 months ago
• weaviate-tutorials / Hurricane
  Writing Blog Posts with Generative Feedback Loops!
  ☆47Updated 10 months ago
• sgugger / hf_examples
  NLP Examples using the 🤗 libraries
  ☆41Updated 3 years ago
• drisspg / transformer_nuggets
  A place to store reusable transformer components of my own creation or found on the interwebs
  ☆44Updated this week
• nateraw / lambdacloud
  An unofficial Python client library for Lambda Lab's Cloud Computing Platform
  ☆12Updated last year
• nateraw / huggingface-datasets-converter
  Scripts to convert datasets from various sources to Hugging Face Datasets.
  ☆58Updated 2 years ago
• tcapelle / mixtral
  Mixtral finetuning
  ☆19Updated last year
• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆34Updated 2 months ago
• NielsRogge / awesome-huggingface
  Repository containing awesome resources regarding Hugging Face tooling.
  ☆46Updated last year
• explodinggradients / Funtuner
  Supervised instruction finetuning for LLM with HF trainer and Deepspeed
  ☆34Updated last year
• AnswerDotAI / toolslm
  Tools to make language models a bit easier to use
  ☆34Updated 3 weeks ago
• bhavsarpratik / semantic-search
  [WIP] Behold, semantic-search, built over sentence-transformers to make it easy for search engineers to evaluate, optimise and deploy mod…
  ☆15Updated last year
• LAION-AI / blade2blade
  Adversarial Training and SFT for Bot Safety Models
  ☆39Updated last year
• lamini-ai / lamini-earnings-calls
  ☆41Updated 8 months ago