Alternatives and similar repositories for tokenizer:

Users that are interested in tokenizer are comparing it to the libraries listed below

• ikekonglp / TweeboParser
  A Dependency Parser for Tweets
  ☆78Updated 5 years ago
• sean-chester / generalised-brown
  C++ implementation of Generalised Brown clustering and python scripts for feature generation
  ☆41Updated 9 years ago
• jacobeisenstein / SAGE
  Sparse Additive Generative Model of Text
  ☆87Updated 8 years ago
• Mrezvan94 / Harassment-Corpus
  Harassment Lexicon and Corpus
  ☆30Updated 6 years ago
• composes-toolkit / dissect
  ☆54Updated 9 years ago
• Noahs-ARK / idea_relations
  A framework to identify relations between ideas in temporal text corpora.
  ☆28Updated 7 years ago
• dirkhovy / MACE
  Multi-Annotator Competence Estimation tool
  ☆63Updated 5 years ago
• myleott / ark-twokenize-py
  Python port of the Twokenize class of ark-tweet-nlp
  ☆142Updated 6 years ago
• vered1986 / LexNET
  LexNET: Integrated Path-based and Distributional Method for Lexical Semantic Relation Classification
  ☆62Updated 6 years ago
• laurejt / authorless-tms
  Repository for code and metadata to support work described in "Authorless Topic Models: Biasing Models Away from Known Structure"
  ☆28Updated 4 years ago
• UKPLab / argument-reasoning-comprehension-task
  The Argument Reasoning Comprehension Task: Source codes & Datasets
  ☆73Updated 3 years ago
• vered1986 / PythonUtils
  Utility scripts in Python
  ☆37Updated 8 months ago
• GateNLP / broad_twitter_corpus
  The Broad Twitter Corpus, an NER dataset in English stratified for time, location, social media genre, socioeconomic factors (COLING 2016…
  ☆67Updated 2 years ago
• tdszyman / twapy
  Temporal Word Analogies in Python
  ☆18Updated 7 years ago
• UCSM-DUE / IWG_hatespeech_public
  public repository of the interdisciplinary working group 'Hatespeech' of the research training group UCSM
  ☆17Updated 6 years ago
• mlukasik / rumour-classification
  Code to reproduce experiments from the EMNLP 2015 paper about Rumour Stance Classification with Gaussian Processes.
  ☆37Updated 8 years ago
• kj2013 / claff-happydb
  Corpus and annotations for the CL-Aff Shared Task from the University of Pennsylvania
  ☆19Updated 3 years ago
• UKPLab / acl2016-convincing-arguments
  Code and data for ACL2016 article "Which argument is more convincing? Analyzing and predicting convincingness of Web arguments using bidi…
  ☆28Updated 8 years ago
• marcocor / bat-framework
  A framework to compare entity linking systems.
  ☆37Updated 6 years ago
• dallascard / media_frames_corpus
  A set of media framing annotations, along with scripts for obtaining the corresponding news articles
  ☆50Updated 5 years ago
• jhlau / topic-coherence-sensitivity
  Code to compute topic coherence for several topic cardinalities and aggregate scores across them
  ☆21Updated last month
• epfl-dlab / quootstrap
  Unsupervised method for extracting quotation-speaker pairs from large news corpora.
  ☆29Updated 6 years ago
• jmhessel / FightingWords
  Quick implementation of Monroe et al.'s algorithm for comparing languages
  ☆53Updated 4 years ago
• xiaohan2012 / chowmein
  Automatic labeling for topic model
  ☆57Updated 9 years ago
• jacobeisenstein / probabilistic-lexicon-classification
  ☆41Updated 8 years ago
• jjessyli / speciteller
  Sentence specificity prediction
  ☆25Updated 6 years ago
• vered1986 / Chirps
  A Large Automatically-Constructed Resource of Predicate Paraphrases
  ☆45Updated 5 years ago
• cnap / ynacc
  The Yahoo News Annotated Comments Corpus (YNACC)
  ☆18Updated 6 years ago
• harkous / embeddingsviz
  Visualize word embeddings of a vocabulary in TensorBoard, including the neighbors
  ☆46Updated 7 years ago
• cocoxu / SemEval-PIT2015
  data and scripts for the shared task "Task 1: Paraphrase and Semantic Similarity in Twitter (PIT)" at SemEval 2015
  ☆43Updated 4 years ago