Alternatives and similar repositories for cc_net

Users that are interested in cc_net are comparing it to the libraries listed below

• google-research / deduplicate-text-datasets
  ☆1,218Updated 10 months ago
• allenai / natural-instructions
  Expanding natural instructions
  ☆995Updated last year
• ChenghaoMou / text-dedup
  All-in-one text de-duplication
  ☆679Updated last week
• facebookresearch / DPR
  Dense Passage Retriever - is a set of tools and models for open domain Q&A task.
  ☆1,801Updated 2 years ago
• bigscience-workshop / bigscience
  Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
  ☆997Updated 10 months ago
• google-research / byt5
  ☆507Updated last year
• bigscience-workshop / data-preparation
  Code used for sourcing and cleaning the BigScience ROOTS corpus
  ☆312Updated 2 years ago
• google-research / bleurt
  BLEURT is a metric for Natural Language Generation based on transfer learning.
  ☆733Updated last year
• facebookresearch / GENRE
  Autoregressive Entity Retrieval
  ☆788Updated last year
• facebookresearch / contriever
  Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning
  ☆737Updated 2 years ago
• THUDM / P-tuning
  A novel method to tune language models. Codes and datasets for paper ``GPT understands, too''.
  ☆933Updated 2 years ago
• google-research / FLAN
  ☆1,520Updated last month
• XiangLi1999 / PrefixTuning
  Prefix-Tuning: Optimizing Continuous Prompts for Generation
  ☆930Updated last year
• facebookresearch / SpanBERT
  Code for using and evaluating SpanBERT.
  ☆899Updated last year
• huggingface / transformers-bloom-inference
  Fast Inference Solutions for BLOOM
  ☆564Updated 7 months ago
• bigscience-workshop / xmtf
  Crosslingual Generalization through Multitask Finetuning
  ☆535Updated 8 months ago
• google-research / multilingual-t5
  ☆1,275Updated 2 years ago
• facebookresearch / KILT
  Library for Knowledge Intensive Language Tasks
  ☆944Updated 3 years ago
• princeton-nlp / LM-BFF
  [ACL 2021] LM-BFF: Better Few-shot Fine-tuning of Language Models https://arxiv.org/abs/2012.15723
  ☆730Updated 2 years ago
• Ki6an / fastT5
  ⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.
  ☆578Updated 2 years ago
• microsoft / ProphetNet
  A research project for natural language generation, containing the official implementations by MSRA NLC team.
  ☆726Updated 10 months ago
• bigscience-workshop / Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆1,391Updated last year
• microsoft / fastseq
  An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…
  ☆433Updated 2 years ago
• google-research / xtreme
  XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 ty…
  ☆644Updated 2 years ago
• google-research / prompt-tuning
  Original Implementation of Prompt Tuning from Lester, et al, 2021
  ☆681Updated 2 months ago
• princeton-nlp / DensePhrases
  [ACL 2021] Learning Dense Representations of Phrases at Scale; EMNLP'2021: Phrase Retrieval Learns Passage Retrieval, Too https://arxiv.o…
  ☆604Updated 2 years ago
• allenai / RL4LMs
  A modular RL library to fine-tune language models to human preferences
  ☆2,308Updated last year
• Tiiiger / bert_score
  BERT score for text generation
  ☆1,746Updated 10 months ago
• nyu-mll / GLUE-baselines
  [DEPRECATED] Repo for exploring multi-task learning approaches to learning sentence representations
  ☆799Updated 3 years ago
• EleutherAI / the-pile
  ☆1,572Updated 2 years ago