Alternatives and similar repositories for redpajama

Users that are interested in redpajama are comparing it to the libraries listed below

• hamelsmu / ft-drift
  Check for data drift between two OpenAI multi-turn chat jsonl files.
  ☆37Updated last year
• google-research-datasets / QAmeleon
  QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…
  ☆34Updated 2 years ago
• abacaj / train-with-fsdp
  ☆94Updated last year
• sgugger / hf_examples
  NLP Examples using the 🤗 libraries
  ☆40Updated 4 years ago
• hundredblocks / large-model-parallelism
  Functional local implementations of main model parallelism approaches
  ☆96Updated 2 years ago
• google-research-datasets / seahorse
  Seahorse is a dataset for multilingual, multi-faceted summarization evaluation. It consists of 96K summaries with human ratings along 6 q…
  ☆89Updated last year
• pacman100 / peft-codegen-25
  ☆23Updated 2 years ago
• Upaya07 / NeurIPS-llm-efficiency-challenge
  Code for NeurIPS LLM Efficiency Challenge
  ☆59Updated last year
• muellerzr / minimal-trainer-zoo
  Minimal example scripts of the Hugging Face Trainer, focused on staying under 150 lines
  ☆197Updated last year
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆47Updated 2 years ago
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆104Updated last year
• explodinggradients / Funtuner
  Supervised instruction finetuning for LLM with HF trainer and Deepspeed
  ☆35Updated 2 years ago
• mobarski / alpaca-libre
  Reimplementation of the task generation part from the Alpaca paper
  ☆119Updated 2 years ago
• Rallio67 / language-model-agents
  Experiments with generating opensource language model assistants
  ☆97Updated 2 years ago
• januverma / transformers-stuff
  Codes, scripts, and notebooks on various aspects of transformer models.
  ☆27Updated 2 years ago
• xaiguy / chippy
  ☆13Updated 2 years ago
• HendrikStrobelt / LMdiff
  A diff tool for language models
  ☆44Updated last year
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated last year
• ConiferLabsWA / flan-ul2-dolly
  ☆34Updated 2 years ago
• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆33Updated this week
• stas00 / porting
  Helper scripts and notes that were used while porting various nlp models
  ☆47Updated 3 years ago
• NirantK / agentai
  Text to Python Objects via a LLM Function Call
  ☆58Updated last year
• geronimi73 / phi2-finetune
  ☆88Updated last year
• tcapelle / llm_recipes
  A set of scripts and notebooks on LLM finetunning and dataset creation
  ☆110Updated 11 months ago
• mrmps / ai-chunker
  Chunk your text using gpt4o-mini more accurately
  ☆44Updated last year
• rasbt / nn_plus_gzip
  Gzip and nearest neighbors for text classification
  ☆57Updated 2 years ago
• keerthanpg / DadJokeGenerator
  ☆46Updated 2 years ago
• AI21Labs / lm-evaluation
  Evaluation suite for large-scale language models.
  ☆128Updated 4 years ago
• CarperAI / cheese
  Used for adaptive human in the loop evaluation of language and embedding models.
  ☆311Updated 2 years ago
• IBM / model-recycling
  Ranking of fine-tuned HF models as base models.
  ☆36Updated 4 months ago