Alternatives and similar repositories for redpajama

Users that are interested in redpajama are comparing it to the libraries listed below

• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆29Updated 8 months ago
• pacman100 / peft-codegen-25
  ☆23Updated last year
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆46Updated last year
• philschmid / optimum-static-quantization
  ☆28Updated 2 years ago
• hamelsmu / ft-drift
  Check for data drift between two OpenAI multi-turn chat jsonl files.
  ☆37Updated last year
• nbroad1881 / strideformer
  Using short models to classify long texts
  ☆21Updated 2 years ago
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated last year
• bhavsarpratik / semantic-search
  [WIP] Behold, semantic-search, built over sentence-transformers to make it easy for search engineers to evaluate, optimise and deploy mod…
  ☆15Updated 2 years ago
• explodinggradients / Funtuner
  Supervised instruction finetuning for LLM with HF trainer and Deepspeed
  ☆35Updated last year
• IBM / model-recycling
  Ranking of fine-tuned HF models as base models.
  ☆35Updated 3 weeks ago
• PrithivirajDamodaran / vision-language-modelling-series
  Companion Repo for the Vision Language Modelling YouTube series - https://bit.ly/3PsbsC2 - by Prithivi Da. Open to PRs and collaborations
  ☆14Updated 2 years ago
• deep-diver / complete-mlops-system-workflow
  ☆17Updated 2 years ago
• ChrisHayduk / QLoRA-for-MLM
  QLoRA for Masked Language Modeling
  ☆22Updated last year
• januverma / transformers-stuff
  Codes, scripts, and notebooks on various aspects of transformer models.
  ☆27Updated 2 years ago
• NirantK / agentai
  Text to Python Objects via a LLM Function Call
  ☆57Updated last year
• yiyixuxu / n-grammer-flax
  Implementation of N-Grammer in Flax
  ☆17Updated 2 years ago
• sgugger / hf_examples
  NLP Examples using the 🤗 libraries
  ☆41Updated 4 years ago
• davanstrien / data-for-fine-tuning-llms
  ☆77Updated 11 months ago
• nateraw / huggingface-datasets-converter
  Scripts to convert datasets from various sources to Hugging Face Datasets.
  ☆57Updated 2 years ago
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated last year
• modal-labs / ci-on-modal
  A sample pattern for running CI tests on Modal
  ☆18Updated last month
• dm4ml / motion
  Framework for building and maintaining self-updating prompts for LLMs
  ☆63Updated 11 months ago
• sayakpaul / GCP-ML-API-Demos
  Contains Colab Notebooks show cool use-cases of different GCP ML APIs.
  ☆10Updated 4 years ago
• krypticmouse / matryoshka-representation-learning
  PyTorch implementation for MRL
  ☆18Updated last year
• deep-diver / TFX-WandB
  ☆14Updated last year
• facebookarchive / fronni
  machine learning model performance metrics & charts with confidence intervals, optimized with numba to be fast
  ☆16Updated 3 years ago
• tanyuqian / cappy
  NeurIPS 2023 - Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer
  ☆43Updated last year
• deep-diver / Continuous-Adaptation-for-Machine-Learning-System-to-Data-Changes
  https://blog.tensorflow.org/2021/12/continuous-adaptation-for-machine.html
  ☆29Updated 3 years ago
• shuttie / embed-benchmark
  ☆16Updated last year
• stephantul / unitoken
  Tokenization across languages. Useful as preprocessing for subword tokenization.
  ☆22Updated 2 years ago