Benchmark scripts for comparing different tokenizers and sentence segmenters of German
☆12Feb 27, 2023Updated 3 years ago
Alternatives and similar repositories for Tokenizer-Evaluation
Users that are interested in Tokenizer-Evaluation are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Translation of query languages to serialized KoralQuery protocol☆15Updated this week
- A Corpus Data Retrieval Index using Lucene for Look-Ups☆20Updated this week
- ☆14May 29, 2026Updated 3 months ago
- Paris multilayer transport network☆11Sep 13, 2021Updated 5 years ago
- Searching in-memory corpus with Corpus Query Language (CQL)☆19Dec 2, 2024Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Coquery is a free corpus query tool for linguists, lexicographers, translators, and anybody who wishes to search and analyse a text corpu…☆19Updated this week
- Multi Tier Annotation Search☆12Jul 10, 2026Updated 2 months ago
- ☆11Feb 13, 2026Updated 7 months ago
- Course materials for the MVA course "algorithms for speech and language processing"☆13Mar 29, 2023Updated 3 years ago
- natural language processing on german texts☆16Mar 20, 2018Updated 8 years ago
- Collection de romans français du dix-huitième siècle (1751-1800) / Collection of Eighteenth-Century French Novels (1751-1800)☆23Apr 23, 2024Updated 2 years ago
- ☆16Aug 22, 2023Updated 3 years ago
- ☆20May 6, 2026Updated 4 months ago
- This is a new backend implementation of the ANNIS linguistic search and visualization system.☆20Aug 11, 2026Updated last month
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A fully-fledge PyTorch package for Morphological Analysis, tailored to morphologically rich and historical languages.☆25Oct 27, 2023Updated 2 years ago
- Korpuslinguistik war noch nie so einfach...☆26Jul 2, 2026Updated 2 months ago
- Chapel-based Optimization☆19Jul 21, 2026Updated 2 months ago
- shoco is a compressor for small text strings. [Not maintained].☆11Sep 4, 2019Updated 7 years ago
- code and data used to build a training dataset for dragnet models☆10Nov 29, 2020Updated 5 years ago
- Small string compression using smaz compression algorithm. Fast, because it's in C. Supports Python 3+☆13Oct 18, 2025Updated 11 months ago
- Basis of FragDenStaat.de's „Koalitionstracker“☆16Sep 4, 2026Updated 2 weeks ago
- Hexatomic is an extensible software for deep multi-layer annotation of linguistic corpora☆18Nov 27, 2024Updated last year
- ☆10Dec 21, 2024Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Automated Safety Testing of Large Language Models☆17Jan 31, 2025Updated last year
- DroiD64 is a graphical file manager for D64, D67, D71, D80, D81, D82, D88, T64 and LNX files.☆12Feb 3, 2024Updated 2 years ago
- The Wikinflection Corpus, from the paper "Wikinflection Corpus: A (Better) Multilingual, Morpheme-Annotated Inflectional Corpus" (Metheni…☆12Dec 15, 2023Updated 2 years ago
- Benson turns a list of URLs into mp3s of the contents of each web page - take control over your reading backlog!☆16Oct 30, 2024Updated last year
- Layout Analysis Dataset with Segmonto (LADaS)☆25May 29, 2026Updated 3 months ago
- Supervised Stylometry☆27Mar 4, 2026Updated 6 months ago
- 📅 Employee Shift Management App – A full-stack application built with MongoDB, Express.js, React, and Node.js (MERN) to manage employee …☆19Aug 14, 2026Updated last month
- Streamlit app that allows users to search for similar artworks based on an uploaded image or a text description☆13Jun 20, 2024Updated 2 years ago
- A reddit bot that finds original publish dates on linked articles.☆10Nov 30, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- A Python scraping module, that extracts text from articles found in RSS feeds. Uses SQLite as database.☆21Jul 5, 2024Updated 2 years ago
- Static Huffman coding☆10Apr 3, 2017Updated 9 years ago
- Get started writing a Chrome extension in TypeScript!☆18Aug 3, 2023Updated 3 years ago
- Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki☆28Jul 31, 2024Updated 2 years ago
- A tiny graph database engine written in C☆10May 9, 2014Updated 12 years ago
- [WWW 2026] 🕸 GlotWeb: Web Indexing for Minority Languages☆17Apr 14, 2026Updated 5 months ago
- A set of utilities to turn Dataclasses into useful configuration managers.☆11Mar 27, 2024Updated 2 years ago