Alternatives and similar repositories for tokenizations:

Users that are interested in tokenizations are comparing it to the libraries listed below

• google-research-datasets / distribution-over-quantities
  ☆19Updated 5 years ago
• facebookresearch / asset
  A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations
  ☆55Updated 2 years ago
• beerqa / IRRR
  ☆31Updated 3 years ago
• asahi417 / relbert
  The official implementation of "Distilling Relation Embeddings from Pre-trained Language Models, EMNLP 2021 main conference", a high-qual…
  ☆46Updated 4 months ago
• cambridgeltl / xcopa
  XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
  ☆102Updated 4 years ago
• google-research / dialog-inpainting
  ☆97Updated 2 years ago
• qkaren / unsup_gen_for_cms_reasoning
  ☆49Updated last year
• danieldeutsch / qaeval
  ☆15Updated 3 years ago
• yumoxu / marge
  Code for ACL 21: Generating Query Focused Summaries from Query-Free Resources
  ☆33Updated 2 years ago
• mainlp / CrossRE
  CrossRE: A Cross-Domain Dataset for Relation Extraction (Findings of EMNLP 2022)
  ☆48Updated 8 months ago
• microsoft / AMOS
  [ICLR 2022] Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
  ☆24Updated last year
• ART-Group-it / KERMIT
  🐸 KERMIT - A lightweight library to encode and interpret Universal Syntactic Embeddings
  ☆58Updated 2 years ago
• hiroki13 / span-based-srl
  ☆46Updated 5 years ago
• IBM / graph_ensemble_learning
  Graph Ensemble Learning
  ☆38Updated last year
• allenai / qasc
  Repository for the Question Answering via Sentence Composition (QASC) dataset
  ☆54Updated last year
• Yale-LILY / ROSE
  ☆38Updated last year
• yuchenlin / awesome-commonsense
  [Work in progress] A reading list for machine commonsense reasoning
  ☆35Updated 5 years ago
• jzbjyb / LPAQA
  Language model Prompt And Query Archive
  ☆158Updated 3 years ago
• martiansideofthemoon / hurdles-longform-qa
  Official repository with code and data accompanying the NAACL 2021 paper "Hurdles to Progress in Long-form Question Answering" (https://a…
  ☆46Updated 2 years ago
• facebookresearch / ketod
  KETOD Knowledge-Enriched Task-Oriented Dialogue
  ☆32Updated 2 years ago
• declare-lab / CIDER
  This repository contains the dataset and the pytorch implementations of the models from the paper CIDER: Commonsense Inference for Dialog…
  ☆27Updated 2 years ago
• cindyxinyiwang / expand-via-lexicon-based-adaptation
  Code for ACL 2022 paper "Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation"
  ☆30Updated 3 years ago
• sunlab-osu / ReasonBERT
  Code and pre-trained models for "ReasonBert: Pre-trained to Reason with Distant Supervision", EMNLP'2021
  ☆29Updated 2 years ago
• neulab / cmu-multinlp
  Generalizing Natural Language Analysis through Span-relation Representations
  ☆91Updated 2 years ago
• facebookresearch / online_dialog_eval
  Code for the paper "Learning an Unreferenced Metric for Online Dialogue Evaluation", ACL 2020
  ☆28Updated last year
• allenai / few_shot_explanations
  Code for NAACL 2022 paper "Reframing Human-AI Collaboration for Generating Free-Text Explanations"
  ☆31Updated last year
• salesforce / QGen
  ☆33Updated last year
• HanNight / RE-T5
  Code and data for "Retrieval Enhanced Model for Commonsense Generation" (ACL-IJCNLP 2021).
  ☆28Updated 3 years ago
• google-deepmind / streamingqa
  ☆46Updated last year
• amazon-science / fact-check-summarization
  ☆77Updated 11 months ago