sayakpaul / count-tokens-hf-datasets
This project shows how to derive the total number of training tokens from a large text dataset from 🤗 datasets with Apache Beam and Dataflow.
☆24Updated 2 years ago
Related projects ⓘ
Alternatives and complementary repositories for count-tokens-hf-datasets
- Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.☆92Updated last year
- Embedding Recycling for Language models☆38Updated last year
- A multi-label text classifier to predict the subject areas of arXiv papers from their abstract bodies.☆17Updated 3 years ago
- PyTorch implementation of GLOM☆21Updated 2 years ago
- Google's BigBird (Jax/Flax & PyTorch) @ 🤗Transformers☆47Updated last year
- Companion Repo for the Vision Language Modelling YouTube series - https://bit.ly/3PsbsC2 - by Prithivi Da. Open to PRs and collaborations☆14Updated 2 years ago
- Official code release for the paper Coder Reviewer Reranking for Code Generation.☆42Updated last year
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆36Updated 2 years ago
- This repository contains example code to build models on TPUs☆30Updated last year
- Tutorial to pretrain & fine-tune a 🤗 Flax T5 model on a TPUv3-8 with GCP☆58Updated 2 years ago
- An implementation of Transformer with Expire-Span, a circuit for learning which memories to retain☆33Updated 4 years ago
- NLP Examples using the 🤗 libraries☆42Updated 3 years ago
- This repository hosts the code to port NumPy model weights of BiT-ResNets to TensorFlow SavedModel format.☆14Updated 2 years ago
- Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing☆47Updated 2 years ago
- ☆21Updated 3 years ago
- Alternate Implementation for Zero Shot Text Classification: Instead of reframing NLI/XNLI, this reframes the text backbone of CLIP models…☆37Updated 2 years ago
- 🛠️ Tools for Transformers compression using PyTorch Lightning ⚡☆79Updated last week
- ☆16Updated last year
- BERT Probe: A python package for probing attention based robustness to character and word based adversarial evaluation. Also, with recipe…☆18Updated 2 years ago
- A library to create and manage configuration files, especially for machine learning projects.☆77Updated 2 years ago
- KitanaQA: Adversarial training and data augmentation for neural question-answering models☆57Updated last year
- Code for the paper: Saying No is An Art: Contextualized Fallback Responses for Unanswerable Dialogue Queries☆19Updated 2 years ago
- Code for scaling Transformers☆26Updated 3 years ago
- Fastai community entry to 2020 Reproducibility Challenge☆17Updated 2 years ago
- Minimal implementation of Denoised Smoothing (https://arxiv.org/abs/2003.01908) in TensorFlow.☆20Updated 3 years ago
- A collection of Models, Datasets, DataModules, Callbacks, Metrics, Losses and Loggers to better integrate pytorch-lightning with transfor…☆47Updated last year
- Shows how to do parameter ensembling using differential evolution.☆10Updated 2 years ago
- Consists of the largest (10K) human annotated code-switched semantic parsing dataset & 170K generated utterance using the CST5 augmentati…☆33Updated last year
- Helper scripts and notes that were used while porting various nlp models☆44Updated 2 years ago