Alternatives and similar repositories for wikipedia-downloader

Users that are interested in wikipedia-downloader are comparing it to the libraries listed below

• EleutherAI / openwebtext2
  ☆91Updated 3 years ago
• EleutherAI / stackexchange-dataset
  Python tools for processing the stackexchange data dumps into a text dataset for Language Models
  ☆83Updated last year
• young-geng / koala_data_pipeline
  The data processing pipeline for the Koala chatbot language model
  ☆118Updated 2 years ago
• facebookresearch / lss_eval
  This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…
  ☆31Updated 2 years ago
• EleutherAI / github-downloader
  Script for downloading GitHub.
  ☆97Updated last year
• LLM360 / crystalcoder-train
  Pre-training code for CrystalCoder 7B LLM
  ☆55Updated last year
• microsoft / SafeNLP
  Safety Score for Pre-Trained Language Models
  ☆96Updated 2 years ago
• allenai / olmo-cookbook
  OLMost every training recipe you need to perform data interventions with the OLMo family of models.
  ☆50Updated last week
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆47Updated 2 years ago
• castorini / hf-spacerini
  Plug-and-play Search Interfaces with Pyserini and Hugging Face
  ☆32Updated 2 years ago
• umass-ml4ed / mathGPT
  A GPT-based generative LM for combined text and math formulas, leveraging tree-based formula encoding. Published as "Tree-Based Represent…
  ☆40Updated 2 years ago
• choosewhatulike / case2code
  ☆16Updated 6 months ago
• leogao2 / lm_dataformat
  ☆79Updated last year
• huggingface / data-measurements-tool
  Developing tools to automatically analyze datasets
  ☆75Updated 11 months ago
• Agora-Lab-AI / Orca
  An Implementation of "Orca: Progressive Learning from Complex Explanation Traces of GPT-4"
  ☆42Updated last year
• explodinggradients / nemesis
  Reward Model framework for LLM RLHF
  ☆61Updated 2 years ago
• bigcode-project / bigcode-analysis
  Repository for analysis and experiments in the BigCode project.
  ☆124Updated last year
• argilla-io / distilabel-spin-dibt
  Repository containing the SPIN experiments on the DIBT 10k ranked prompts
  ☆24Updated last year
• leogao2 / commoncrawl_downloader
  ☆33Updated 2 years ago
• leehanchung / lora-instruct
  Finetune Falcon, LLaMA, MPT, and RedPajama on consumer hardware using PEFT LoRA
  ☆103Updated 5 months ago
• mungg / FABLES
  ☆57Updated last year
• EleutherAI / DeeperSpeed
  DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
  ☆169Updated 3 weeks ago
• allenai / CommonGen-Eval
  Evaluating LLMs with CommonGen-Lite
  ☆91Updated last year
• EQ-bench / eqbench3
  ☆26Updated 2 months ago
• jina-ai / textbook
  distill chatGPT coding ability into small model (1b)
  ☆30Updated 2 years ago
• akjindal53244 / Arithmo
  Small and Efficient Mathematical Reasoning LLMs
  ☆72Updated last year
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆107Updated last year
• finetunej / transformers
  🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.
  ☆55Updated 3 years ago
• xlang-ai / batch-prompting
  [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.
  ☆76Updated last year
• EleutherAI / lm_perplexity
  ☆158Updated 4 years ago