Alternatives and similar repositories for wikipedia-downloader

Users that are interested in wikipedia-downloader are comparing it to the libraries listed below

• EleutherAI / openwebtext2
  ☆92Updated 3 years ago
• EleutherAI / stackexchange-dataset
  Python tools for processing the stackexchange data dumps into a text dataset for Language Models
  ☆86Updated 2 years ago
• umass-ml4ed / mathGPT
  A GPT-based generative LM for combined text and math formulas, leveraging tree-based formula encoding. Published as "Tree-Based Represent…
  ☆41Updated 2 years ago
• EleutherAI / lm_perplexity
  ☆160Updated 4 years ago
• young-geng / koala_data_pipeline
  The data processing pipeline for the Koala chatbot language model
  ☆118Updated 2 years ago
• choosewhatulike / case2code
  ☆17Updated 9 months ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆49Updated 2 years ago
• Agora-Lab-AI / Orca
  An Implementation of "Orca: Progressive Learning from Complex Explanation Traces of GPT-4"
  ☆43Updated last year
• LLM360 / crystalcoder-train
  Pre-training code for CrystalCoder 7B LLM
  ☆56Updated last year
• vibrantlabsai / nemesis
  Reward Model framework for LLM RLHF
  ☆62Updated 2 years ago
• leogao2 / commoncrawl_downloader
  ☆32Updated 2 years ago
• togethercomputer / OpenDataHub
  ☆128Updated 2 years ago
• EleutherAI / DeeperSpeed
  DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.
  ☆171Updated 3 months ago
• akjindal53244 / Arithmo
  Small and Efficient Mathematical Reasoning LLMs
  ☆73Updated last year
• microsoft / SafeNLP
  Safety Score for Pre-Trained Language Models
  ☆96Updated 2 years ago
• bigcode-project / bigcode-analysis
  Repository for analysis and experiments in the BigCode project.
  ☆128Updated last year
• LLM360 / Analysis360
  Open Implementations of LLM Analyses
  ☆107Updated last year
• leogao2 / lm_dataformat
  ☆78Updated 2 years ago
• EleutherAI / best-download
  URL downloader supporting checkpointing and continuous checksumming.
  ☆19Updated 2 years ago
• huggingface / api-inference-community
  ☆172Updated 10 months ago
• thoppe / The-Pile-FreeLaw
  Download, parse, and filter data from Court Listener, part of the FreeLaw projects. Data-ready for The-Pile.
  ☆15Updated 2 years ago
• allenai / datamap-rs
  Data mapping framework for rust stuff
  ☆42Updated this week
• facebookresearch / lss_eval
  This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…
  ☆31Updated 2 years ago
• daniel-furman / sft-demos
  Lightweight demos for finetuning LLMs. Powered by 🤗 transformers and open-source datasets.
  ☆78Updated last year
• kyegomez / Finetuning-Suite
  Finetune any model on HF in less than 30 seconds
  ☆56Updated 2 months ago
• EleutherAI / github-downloader
  Script for downloading GitHub.
  ☆98Updated last year
• huggingface / disaggregators
  🤗 Disaggregators: Curated data labelers for in-depth analysis.
  ☆67Updated 2 years ago
• huggingface / data-measurements-tool
  Developing tools to automatically analyze datasets
  ☆75Updated last year
• togethercomputer / Llama-2-7B-32K-Instruct
  ☆85Updated 2 years ago
• official-elinas / zeus-llm-trainer
  Zeus LLM Trainer is a rewrite of Stanford Alpaca aiming to be the trainer for all Large Language Models
  ☆70Updated 2 years ago