timoschick / one-token-approximation
This repository contains the code for applying One-Token Approximation to a pretrained language model using subword-level tokenization.
☆11Updated 4 years ago
Alternatives and similar repositories for one-token-approximation:
Users that are interested in one-token-approximation are comparing it to the libraries listed below
- Research code for the paper "How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models"☆26Updated 3 years ago
- Analyzing mBERT's multilinguality in a small laboratory setting☆13Updated last year
- Repository with code for MaChAmp: https://aclanthology.org/2021.eacl-demos.22/☆83Updated last week
- ☆46Updated 5 years ago
- RelEx - A simple framework for Relation Extraction built on AllenNLP☆16Updated 4 years ago
- This repository contains the code for "BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Representations".☆63Updated 4 years ago
- This repository contains the WordNet Language Model Probing (WNLaMPro) dataset introduced in "Rare Words: A Major Problem for Contextuali…☆14Updated 5 years ago
- ☆24Updated last year
- Tool to perform paired evaluation of automatic systems☆12Updated 3 years ago
- A software for transferring pre-trained English models to foreign languages☆18Updated last year
- ☆25Updated last year
- EMNLP 2021 Tutorial: Multi-Domain Multilingual Question Answering☆38Updated 3 years ago
- Statistics on multilingual datasets☆17Updated 2 years ago
- ☆11Updated 2 years ago
- Code and datasets for the EMNLP 2020 paper "Calibration of Pre-trained Transformers"☆58Updated last year
- A template for starting a new allennlp project using config files and `allennlp train`☆38Updated 11 months ago
- Automatically harvested multilingual contrastive word sense disambiguation test sets for machine translation☆17Updated 4 years ago
- ☆25Updated 2 years ago
- This data release is meant to accompany and document the paper: https://arxiv.org/abs/2004.11997 Collecting Entailment Data for Pretrain…☆14Updated 4 years ago
- Implementation of the paper "Fine-Tuning Transformers: Vocabulary Transfer" https://arxiv.org/pdf/2112.14569.pdf☆20Updated 3 years ago
- We are creating a challenging new benchmark MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models. Retrieval quest…☆31Updated 4 years ago
- ☆24Updated 5 years ago
- ☆13Updated 3 years ago
- A template for starting an allennlp project using a python script instead of config files☆27Updated 11 months ago
- The Referential Reader: A Recurrent Entity Network for Anaphora Resolution, published at ACL 2019☆19Updated 5 years ago
- Pretraining scripts for BART transformer model☆11Updated last year
- Implementation of the paper 'Sentence Bottleneck Autoencoders from Transformer Language Models'☆17Updated 3 years ago
- Hyperparameter Search for AllenNLP☆137Updated last week
- Codebase for probing and visualizing multilingual models.☆47Updated 4 years ago
- ☆15Updated 3 years ago