kootenpv / tok
Fast and customizable tokenization
☆64Updated 5 years ago
Related projects ⓘ
Alternatives and complementary repositories for tok
- Find strings/words in text; convenience and C speed☆126Updated 2 years ago
- An easy to use open-source library for advanced Deep Learning and Natural Language Processing☆112Updated 3 months ago
- ALMa (Active Learning Manager) Keeps track of labeled and unlabeled data for active learning☆42Updated 4 years ago
- Code and data accompanying the paper "Approaching nested named entity recognition with parallel LSTM-CRFs."☆26Updated last year
- spaCy + UDPipe☆161Updated 2 years ago
- Misspelling Oblivious Word Embeddings☆202Updated 5 years ago
- Textpipe: clean and extract metadata from text☆299Updated 3 years ago
- A Lightweight NLP Data Loader for All Deep Learning Frameworks in Python☆180Updated 10 months ago
- A lightweight but powerful library to build token indices for NLP tasks, compatible with major Deep Learning frameworks like PyTorch and …☆49Updated 4 years ago
- Python stream processing for humans☆184Updated 3 weeks ago
- An asynchronous concurrent pipeline for classifying Common Crawl based on fastText's pipeline.☆86Updated 3 years ago
- Enso: An Open Source Library for Benchmarking Embeddings + Transfer Learning Methods☆96Updated 3 years ago
- Lightning Fast Language Prediction 🚀☆165Updated 5 years ago
- Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings☆76Updated 2 years ago
- ☆29Updated 2 years ago
- A python module for word inflections designed for use with spaCy.☆92Updated 4 years ago
- Example using Polyaxon to experiment with pre-training spaCy☆65Updated 3 years ago
- Visualization of filters in convolutional neural networks☆30Updated 8 months ago
- Segtok v2 is here: https://github.com/fnl/syntok -- A rule-based sentence segmenter (splitter) and a word tokenizer using orthographic fe…☆170Updated 2 years ago
- Polyglot skipgram embeddings, and their many health benefits☆11Updated 4 years ago
- A fast and memory-optimized string library for heavy-text manipulation in Python☆250Updated 4 years ago
- Text tokenization and sentence segmentation (segtok v2)☆203Updated 2 years ago
- interactive explorer for language models☆132Updated 2 years ago
- My most frequently used learning-to-rank algorithms ported to rust for efficiency. Try it: "pip install fastrank".☆48Updated last year
- Text Mining and Topic Modeling Toolkit for Python with parallel processing power☆193Updated last year
- A Domain Specific Language (DSL) for building language patterns. These can be later compiled into spaCy patterns, pure regex, or any othe…☆65Updated 2 years ago
- jiant-dev☆28Updated 3 years ago
- Python 3 library for reading and writing warc files☆21Updated 6 years ago