LlmKira / fast-langdetectLinks
⚡️ 80x faster Fasttext language detection out of the box | Split text by language
☆228Updated 5 months ago
Alternatives and similar repositories for fast-langdetect
Users that are interested in fast-langdetect are comparing it to the libraries listed below
Sorting:
- 80x faster and 95% accurate language identification with Fasttext☆162Updated last year
- A streamlined, user-friendly JSON streaming preprocessor, crafted in Python.☆106Updated 11 months ago
- [ACL 2024] This is the code repo for our ACL’24 paper "Cleaner Pretraining Corpus Curation with Neural Web Scraping".☆228Updated last year
- Open Source Text Embedding Models with OpenAI Compatible API☆159Updated last year
- Evaluation for AI apps and agent☆43Updated last year
- 🔧 Repair JSON!Solution for JSON Anomalies from LLMs.☆273Updated last week
- Unattended Lightweight Text Classifiers with LLM Embeddings☆184Updated 11 months ago
- ☆191Updated last week
- A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.☆362Updated 2 weeks ago
- Speech Diarization for scrum automation☆111Updated 2 years ago
- ☆234Updated 2 months ago
- TF-ID: Table/Figure IDentifier for academic papers☆238Updated last year
- Sentence Transformers API: An OpenAI compatible embedding API server☆65Updated 11 months ago
- Extract structured text from pdfs quickly☆585Updated 2 months ago
- Query Expension for Better Query Embedding using LLMs☆56Updated 6 months ago
- Deep Reasoning Translation (DRT) Project☆227Updated 3 months ago
- A very simple news crawler with a funny name☆398Updated this week
- Whisper realtime streaming for long speech-to-text transcription and translation☆121Updated last year
- A lightweight script for processing HTML page to markdown format with support for code blocks☆79Updated last year
- Baguetter is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, imp…☆189Updated 11 months ago
- Code for explaining and evaluating late chunking (chunked pooling)☆443Updated 8 months ago
- Enhancing Translation with RAG-Powered Large Language Models☆81Updated last week
- Lightweight, performant, deep table extraction☆503Updated 3 weeks ago
- Python API for https://vespa.ai, the open big data serving engine☆137Updated this week
- A Python Package to Access World-Class Generative Models☆129Updated last year
- Scrape the webpage convert it into Markdown, and enhance AI search applications.☆252Updated last year
- A simple tool that let's you explore different possible paths that an LLM might sample.☆185Updated 3 months ago
- Formatron empowers everyone to control the format of language models' output with minimal overhead.☆223Updated 2 months ago
- Multi-threaded matrix multiplication and cosine similarity calculations for dense and sparse matrices. Appropriate for calculating the K …☆83Updated 8 months ago
- ☆489Updated 5 months ago