ropensci / tokenizers

Fast, Consistent Tokenization of Natural Language Text
186Updated last year

Alternatives and similar repositories for tokenizers:

Users that are interested in tokenizers are comparing it to the libraries listed below