Benchmark scripts for comparing different tokenizers and sentence segmenters of German
☆12Feb 27, 2023Updated 3 years ago
Alternatives and similar repositories for Tokenizer-Evaluation
Users that are interested in Tokenizer-Evaluation are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Translation of query languages to serialized KoralQuery protocol☆14Updated this week
- A Corpus Data Retrieval Index using Lucene for Look-Ups☆20Updated this week
- ☆13Apr 10, 2026Updated 3 weeks ago
- Paris multilayer transport network☆11Sep 13, 2021Updated 4 years ago
- Searching in-memory corpus with Corpus Query Language (CQL)☆19Dec 2, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Coquery is a free corpus query tool for linguists, lexicographers, translators, and anybody who wishes to search and analyse a text corpu…☆19Jan 7, 2026Updated 3 months ago
- ☆11Feb 13, 2026Updated 2 months ago
- Multi Tier Annotation Search☆12May 13, 2024Updated last year
- Course materials for the MVA course "algorithms for speech and language processing"☆12Mar 29, 2023Updated 3 years ago
- natural language processing on german texts☆16Mar 20, 2018Updated 8 years ago
- Layout Analysis Dataset with Segmonto (LADaS)☆25Jul 12, 2025Updated 9 months ago
- ☆16Aug 22, 2023Updated 2 years ago
- This is a new backend implementation of the ANNIS linguistic search and visualization system.☆18Apr 18, 2026Updated 2 weeks ago
- Chapel-based Optimization☆15Apr 24, 2026Updated last week
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Collection de romans français du dix-huitième siècle (1751-1800) / Collection of Eighteenth-Century French Novels (1751-1800)☆23Apr 23, 2024Updated 2 years ago
- ☆20Apr 8, 2026Updated 3 weeks ago
- Korpuslinguistik war noch nie so einfach...☆25Feb 18, 2026Updated 2 months ago
- shoco is a compressor for small text strings. [Not maintained].☆11Sep 4, 2019Updated 6 years ago
- code and data used to build a training dataset for dragnet models☆10Nov 29, 2020Updated 5 years ago
- Hexatomic is an extensible software for deep multi-layer annotation of linguistic corpora☆17Nov 27, 2024Updated last year
- A fully-fledge PyTorch package for Morphological Analysis, tailored to morphologically rich and historical languages.☆24Oct 27, 2023Updated 2 years ago
- Basis of FragDenStaat.de's „Koalitionstracker“☆15Jul 14, 2025Updated 9 months ago
- ☆10Dec 21, 2024Updated last year
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Small string compression using smaz compression algorithm. Fast, because it's in C. Supports Python 3+☆13Oct 18, 2025Updated 6 months ago
- [WWW 2026] 🕸 GlotWeb: Web Indexing for Minority Languages☆17Apr 14, 2026Updated 2 weeks ago
- The Wikinflection Corpus, from the paper "Wikinflection Corpus: A (Better) Multilingual, Morpheme-Annotated Inflectional Corpus" (Metheni…☆12Dec 15, 2023Updated 2 years ago
- Benson turns a list of URLs into mp3s of the contents of each web page - take control over your reading backlog!☆16Oct 30, 2024Updated last year
- Camel Morph’s goal is to build large open-source morphological models for Arabic and its dialects across many genres and domains.☆18Dec 8, 2024Updated last year
- DroiD64 is a graphical file manager for D64, D67, D71, D80, D81, D82, D88, T64 and LNX files.☆13Feb 3, 2024Updated 2 years ago
- Automated Safety Testing of Large Language Models☆18Jan 31, 2025Updated last year
- Get started writing a Chrome extension in TypeScript!☆18Aug 3, 2023Updated 2 years ago
- A reddit bot that finds original publish dates on linked articles.☆10Nov 30, 2024Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Static Huffman coding☆10Apr 3, 2017Updated 9 years ago
- SMOR (Stuttgart Morphology) with alternative lemmatization component☆13Aug 10, 2023Updated 2 years ago
- Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki☆28Jul 31, 2024Updated last year
- A set of utilities to turn Dataclasses into useful configuration managers.☆11Mar 27, 2024Updated 2 years ago
- Produces a serialized hardware report of the physical infrastructure for automation☆26Mar 25, 2026Updated last month
- Supervised Stylometry☆27Mar 4, 2026Updated 2 months ago
- ☆19Mar 13, 2025Updated last year