leogao2 / commoncrawl_downloaderLinks
☆33Updated 2 years ago
Alternatives and similar repositories for commoncrawl_downloader
Users that are interested in commoncrawl_downloader are comparing it to the libraries listed below
Sorting:
- ☆91Updated 3 years ago
- ☆78Updated last year
- ☆50Updated 2 years ago
- ☆159Updated 4 years ago
- Evaluation suite for large-scale language models.☆128Updated 4 years ago
- Python tools for processing the stackexchange data dumps into a text dataset for Language Models☆82Updated last year
- Open source library for few shot NLP☆78Updated 2 years ago
- Experiments with generating opensource language model assistants☆97Updated 2 years ago
- YT_subtitles - extracts subtitles from YouTube videos to raw text for Language Model training☆45Updated 5 years ago
- Script for downloading GitHub.☆97Updated last year
- Source codes for the paper "Bounding the Capabilities of Large Language Models in Open Text Generation with Prompt Constraints"☆27Updated 2 years ago
- DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.☆169Updated last month
- ☆131Updated 3 years ago
- Simple Annotated implementation of GPT-NeoX in PyTorch☆110Updated 3 years ago
- Pipeline for pulling and processing online language model pretraining data from the web☆178Updated 2 years ago
- ☆114Updated 3 years ago
- For experiments involving instruct gpt. Currently used for documenting open research questions.☆70Updated 2 years ago
- This repository contains all the code for collecting large scale amounts of code from GitHub.☆109Updated 2 years ago
- Create soft prompts for fairseq 13B dense, GPT-J-6B and GPT-Neo-2.7B for free in a Google Colab TPU instance☆28Updated 2 years ago
- Documentation effort for the BookCorpus dataset☆34Updated 4 years ago
- A library for squeakily cleaning and filtering language datasets.☆47Updated 2 years ago
- GenieNLP: A versatile codebase for any NLP task☆88Updated last year
- Used for adaptive human in the loop evaluation of language and embedding models.☆307Updated 2 years ago
- ☆97Updated 3 years ago
- ☆95Updated 10 months ago
- Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)☆61Updated 2 years ago
- One stop shop for all things carp☆59Updated 3 years ago
- XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale☆156Updated last year
- Code for Relevance-guided Supervision for OpenQA with ColBERT (TACL'21)☆41Updated 4 years ago
- An experimental implementation of the retrieval-enhanced language model☆75Updated 2 years ago