Alternatives and similar repositories for tokenizations:

Users that are interested in tokenizations are comparing it to the libraries listed below

• neulab / wikiasp
  Code for WikiAsp: Multi-document aspect-based summarization.
  ☆41Updated 4 years ago
• atcbosselut / scs-baselines
  Baseline models for the paper: "Modeling Naive Psychology of Characters in Simple Commonsense Stories" by Hannah Rashkin, Antoine Bosselu…
  ☆16Updated 4 years ago
• ruiqi-zhong / Meta-tuning
  EMNLP 2021 Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections
  ☆50Updated 3 years ago
• google-research-datasets / distribution-over-quantities
  ☆19Updated 5 years ago
• asahi417 / relbert
  The official implementation of "Distilling Relation Embeddings from Pre-trained Language Models, EMNLP 2021 main conference", a high-qual…
  ☆46Updated 3 months ago
• marvl-challenge / marvl-code
  [EMNLP 2021] Code and data for our paper "Visually Grounded Reasoning across Languages and Cultures"
  ☆30Updated 3 years ago
• coastalcph / seq2sparql
  Multilingual Compositional Wikidata Questions (MCWQ)
  ☆18Updated last year
• DevSinghSachan / syntax-augmented-bert
  Source code of the paper "Do Syntax Trees Help Pre-trained Transformers Extract Information?" (EACL 2021)
  ☆74Updated 3 years ago
• yumoxu / marge
  Code for ACL 21: Generating Query Focused Summaries from Query-Free Resources
  ☆33Updated 2 years ago
• qkaren / unsup_gen_for_cms_reasoning
  ☆49Updated last year
• INK-USC / CommonGen
  A Constrained Text Generation Challenge Towards Generative Commonsense Reasoning
  ☆140Updated last year
• thunlp / Knowledge-Inheritance
  Source code for paper: Knowledge Inheritance for Pre-trained Language Models
  ☆38Updated 2 years ago
• MurtyShikhar / ExpBERT
  Code for our ACL '20 paper "Representation Engineering with Natural Language Explanations"
  ☆29Updated 4 years ago
• cambridgeltl / xcopa
  XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
  ☆101Updated 4 years ago
• IBM / graph_ensemble_learning
  Graph Ensemble Learning
  ☆38Updated last year
• google-research / dialog-inpainting
  ☆97Updated 2 years ago
• amazon-science / fact-check-summarization
  ☆77Updated 10 months ago
• ghomasHudson / muld
  The Multitask Long Document Benchmark
  ☆38Updated 2 years ago
• facebookresearch / asset
  A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations
  ☆55Updated 2 years ago
• mainlp / CrossRE
  CrossRE: A Cross-Domain Dataset for Relation Extraction (Findings of EMNLP 2022)
  ☆47Updated 7 months ago
• facebookresearch / ketod
  KETOD Knowledge-Enriched Task-Oriented Dialogue
  ☆32Updated 2 years ago
• vered1986 / self_talk
  Code and data for the paper: "Unsupervised Common Sense Question Answering with Self-Talk"
  ☆78Updated 3 years ago
• katzurik / NERetrieve
  ☆28Updated last year
• TalSchuster / VitaminC
  Contrastive Fact Verification
  ☆71Updated 2 years ago
• teacherpeterpan / Zero-shot-Fact-Verification
  Codes for ACL-IJCNLP 2021 Paper "Zero-shot Fact Verification by Claim Generation"
  ☆64Updated 3 years ago
• allenai / flex
  Few-shot NLP benchmark for unified, rigorous eval
  ☆91Updated 2 years ago
• CogComp / faithful_summarization
  ☆15Updated 3 years ago
• allenai / qasc
  Repository for the Question Answering via Sentence Composition (QASC) dataset
  ☆53Updated last year
• dqxiu / ParaSCI
  a large scientific paraphrase dataset for longer paraphrase generation
  ☆38Updated 2 years ago
• ZHAOTING / dialog-processing
  NLG and NLU for dialogue processing
  ☆42Updated last year