Alternatives and similar repositories for tokenizer:

Users that are interested in tokenizer are comparing it to the libraries listed below

• ikekonglp / TweeboParser
  A Dependency Parser for Tweets
  ☆79Updated 5 years ago
• UCSM-DUE / IWG_hatespeech_public
  public repository of the interdisciplinary working group 'Hatespeech' of the research training group UCSM
  ☆17Updated 5 years ago
• jacobeisenstein / SAGE
  Sparse Additive Generative Model of Text
  ☆87Updated 8 years ago
• mlukasik / rumour-classification
  Code to reproduce experiments from the EMNLP 2015 paper about Rumour Stance Classification with Gaussian Processes.
  ☆36Updated 8 years ago
• myleott / ark-twokenize-py
  Python port of the Twokenize class of ark-tweet-nlp
  ☆141Updated 6 years ago
• GateNLP / broad_twitter_corpus
  The Broad Twitter Corpus, an NER dataset in English stratified for time, location, social media genre, socioeconomic factors (COLING 2016…
  ☆66Updated 2 years ago
• tdszyman / twapy
  Temporal Word Analogies in Python
  ☆18Updated 7 years ago
• Noahs-ARK / idea_relations
  A framework to identify relations between ideas in temporal text corpora.
  ☆29Updated 6 years ago
• jjessyli / speciteller
  Sentence specificity prediction
  ☆25Updated 5 years ago
• sean-chester / generalised-brown
  C++ implementation of Generalised Brown clustering and python scripts for feature generation
  ☆42Updated 8 years ago
• kj2013 / claff-happydb
  Corpus and annotations for the CL-Aff Shared Task from the University of Pennsylvania
  ☆19Updated 3 years ago
• afshinrahimi / pigeo
  An Easy to Use, Accurate Python Geolocation Library
  ☆40Updated last year
• diyiy / Wiki_Semantic_Intention
  Predict edit intentions on Wikipedia
  ☆19Updated 5 years ago
• adalmia96 / Cluster-Analysis
  ☆54Updated 3 years ago
• epfl-dlab / quootstrap
  Unsupervised method for extracting quotation-speaker pairs from large news corpora.
  ☆29Updated 6 years ago
• dirkhovy / MACE
  Multi-Annotator Competence Estimation tool
  ☆63Updated 5 years ago
• dallascard / media_frames_corpus
  A set of media framing annotations, along with scripts for obtaining the corresponding news articles
  ☆49Updated 5 years ago
• harkous / embeddingsviz
  Visualize word embeddings of a vocabulary in TensorBoard, including the neighbors
  ☆46Updated 7 years ago
• NLPrinceton / ALaCarte
  ☆104Updated 6 years ago
• vered1986 / Chirps
  A Large Automatically-Constructed Resource of Predicate Paraphrases
  ☆43Updated 4 years ago
• vered1986 / PythonUtils
  Utility scripts in Python
  ☆37Updated 4 months ago
• cocoxu / SemEval-PIT2015
  data and scripts for the shared task "Task 1: Paraphrase and Semantic Similarity in Twitter (PIT)" at SemEval 2015
  ☆44Updated 4 years ago
• sronnqvist / doc2topic
  Neural topic modeling
  ☆29Updated 4 years ago
• jmhessel / FightingWords
  Quick implementation of Monroe et al.'s algorithm for comparing languages
  ☆53Updated 4 years ago
• afshinrahimi / acl2017
  A Neural Model for User Geolocation and Lexical Dialectology
  ☆16Updated 6 years ago
• cedricdeboom / RepresentationLearning
  Code base for representation learning of very short texts, such as tweets. By Cedric De Boom, IBCN, Ghent University, Belgium.
  ☆37Updated 8 years ago
• HKUST-KnowComp / MLMA_hate_speech
  Dataset and code of our EMNLP 2019 paper "Multilingual and Multi-Aspect Hate Speech Analysis"
  ☆56Updated last month
• dallascard / scholar
  Neural models for documents with metadata
  ☆103Updated 4 years ago