zafercavdar / fasttext-langdetectLinks
80x faster and 95% accurate language identification with Fasttext
☆162Updated last year
Alternatives and similar repositories for fasttext-langdetect
Users that are interested in fasttext-langdetect are comparing it to the libraries listed below
Sorting:
- 💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆148Updated 2 months ago
- Simply, faster, sentence-transformers☆143Updated last year
- ⚡️ 80x faster Fasttext language detection out of the box | Split text by language☆228Updated 4 months ago
- FastFit ⚡ When LLMs are Unfit Use FastFit ⚡ Fast and Effective Text Classification with Many Classes☆211Updated 3 months ago
- Python API for https://vespa.ai, the open big data serving engine☆137Updated this week
- A Python Search Engine for Humans 🥸☆231Updated last year
- Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.☆252Updated 2 years ago
- Efficient few-shot learning with cross-encoders.☆56Updated last year
- Python3 bindings for the Compact Language Detector v3 (CLD3)☆154Updated 2 years ago
- The pipeline for the OSCAR corpus☆171Updated last year
- [EMNLP 2023 Demo] fabricator - annotating and generating datasets with large language models.☆110Updated last year
- Simple multilingual lemmatizer for Python, especially useful for speed and efficiency☆170Updated 2 months ago
- Official implementation of the paper "CoEdIT: Text Editing by Task-Specific Instruction Tuning" (EMNLP 2023)☆129Updated 11 months ago
- Targetted language identifier, based on FastText and Hunspell.☆37Updated 6 months ago
- Baguetter is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, imp…☆188Updated 11 months ago
- Completion After Prompt Probability. Make your LLM make a choice☆80Updated 9 months ago
- A language detection software☆56Updated 7 years ago
- ☆106Updated 8 months ago
- A multilingual version of MS MARCO passage ranking dataset☆144Updated last year
- ☆367Updated last year
- 🔢 Work with static vector models☆28Updated 4 months ago
- A large-scale multilingual dataset for Information Retrieval. Thorough human-annotations across 18 diverse languages.☆194Updated last year
- A robust web archive analytics toolkit☆114Updated 5 months ago
- KeyPhraseTransformer lets you quickly extract key phrases, topics, themes from your text data with T5 transformer | Keyphrase extraction…☆104Updated last year
- ☆172Updated 5 months ago
- Generalist and Lightweight Model for Text Classification☆156Updated 2 months ago
- PyTorch-IE: State-of-the-art Information Extraction in PyTorch☆78Updated 3 weeks ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆74Updated 4 months ago
- Incorporating VIsual LAyout Structures for Scientific Text Classification☆179Updated 2 years ago
- Faster, modernized fork of the language identification tool langid.py☆56Updated 9 months ago