facebookresearch / cc_net
Tools to download and cleanup Common Crawl data
☆971Updated last year
Related projects ⓘ
Alternatives and complementary repositories for cc_net
- All-in-one text de-duplication☆622Updated 5 months ago
- BLEURT is a metric for Natural Language Generation based on transfer learning.☆697Updated last year
- Dense Passage Retriever - is a set of tools and models for open domain Q&A task.☆1,726Updated last year
- ☆1,124Updated 3 months ago
- Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.☆980Updated 3 months ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,338Updated 8 months ago
- Expanding natural instructions☆959Updated 11 months ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆306Updated last year
- Library for Knowledge Intensive Language Tasks☆916Updated 2 years ago
- Crosslingual Generalization through Multitask Finetuning☆516Updated last month
- ☆1,474Updated 3 weeks ago
- Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning☆685Updated last year
- A research project for natural language generation, containing the official implementations by MSRA NLC team.☆692Updated 3 months ago
- Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons☆1,068Updated 3 months ago
- SGPT: GPT Sentence Embeddings for Semantic Search☆852Updated 9 months ago
- A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.☆1,620Updated 3 months ago
- An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…☆433Updated 2 years ago
- Natural Questions (NQ) contains real user questions issued to Google search, and answers found from Wikipedia by annotators. NQ is design…☆940Updated 3 years ago
- [ACL 2021] Learning Dense Representations of Phrases at Scale; EMNLP'2021: Phrase Retrieval Learns Passage Retrieval, Too https://arxiv.o…☆605Updated 2 years ago
- Fast BPE☆656Updated 5 months ago
- Fast Inference Solutions for BLOOM☆560Updated last month
- Plug and Play Language Model implementation. Allows to steer topic and attributes of GPT-2 models.☆1,132Updated 9 months ago
- Prefix-Tuning: Optimizing Continuous Prompts for Generation☆896Updated 6 months ago
- [ACL 2021] LM-BFF: Better Few-shot Fine-tuning of Language Models https://arxiv.org/abs/2012.15723☆721Updated 2 years ago
- XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 ty…☆631Updated last year
- Officially supported AllenNLP models☆528Updated last year
- Autoregressive Entity Retrieval☆764Updated last year
- Code for using and evaluating SpanBERT.☆891Updated last year
- Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03…☆517Updated 11 months ago
- FastFormers - highly efficient transformer models for NLU☆701Updated 10 months ago