LlmKira / fast-langdetectLinks
⚡️ 80x faster Fasttext language detection out of the box | Split text by language
☆223Updated 4 months ago
Alternatives and similar repositories for fast-langdetect
Users that are interested in fast-langdetect are comparing it to the libraries listed below
Sorting:
- 80x faster and 95% accurate language identification with Fasttext☆160Updated last year
- ✨ Split text by languages (e.g. 你喜欢看アニメ吗 -> 你喜欢看 | アニメ | 吗) for NLP tasks (e.g. parse, TTS). Powered by fasttext and budoux☆62Updated 5 months ago
- [ACL 2024] This is the code repo for our ACL’24 paper "Cleaner Pretraining Corpus Curation with Neural Web Scraping".☆226Updated 11 months ago
- A streamlined, user-friendly JSON streaming preprocessor, crafted in Python.☆103Updated 10 months ago
- Open Source Text Embedding Models with OpenAI Compatible API☆157Updated last year
- Unattended Lightweight Text Classifiers with LLM Embeddings☆185Updated 11 months ago
- A Faster LayoutReader Model based on LayoutLMv3, Sort OCR bboxes to reading order.☆267Updated last month
- Whisper realtime streaming for long speech-to-text transcription and translation☆120Updated last year
- Turn any OCR models into online inference API endpoint 🚀 🌖☆57Updated 4 months ago
- Extract structured text from pdfs quickly☆523Updated last month
- 🔧 Repair JSON!Solution for JSON Anomalies from LLMs.☆270Updated last month
- A lightweight script for processing HTML page to markdown format with support for code blocks☆79Updated last year
- Evaluation for AI apps and agent☆42Updated last year
- Python API for https://vespa.ai, the open big data serving engine☆135Updated this week
- ☆191Updated last month
- Speech Diarization for scrum automation☆110Updated 2 years ago
- A very simple news crawler with a funny name☆395Updated this week
- A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.☆348Updated 2 months ago
- Conversational Retrieval Evaluation Dataset☆101Updated 5 months ago
- Gemma 2 optimized for your local machine.☆376Updated last year
- 🎧 Pod-Helper: Real-time audio transcription and repair on consumer hardware☆77Updated last year
- ☆231Updated last month
- An enterprise-grade AI retriever designed to streamline AI integration into your applications, ensuring cutting-edge accuracy.☆287Updated last month
- A library to extract the main content from html. Developed for information on LLM and for feeding data into LangChain and LlamaIndex.☆45Updated last year
- Deep Reasoning Translation (DRT) Project☆227Updated 2 months ago
- ☆482Updated 4 months ago
- Sentence Transformers API: An OpenAI compatible embedding API server☆64Updated 11 months ago
- Python Implementation of MUVERA (Multi-Vector Retrieval via Fixed Dimensional Encodings)☆278Updated last month
- FineTune LLMs in few lines of code (Text2Text, Text2Speech, Speech2Text)☆240Updated last year
- Baguetter is a flexible, efficient, and hackable search engine library implemented in Python. It's designed for quickly benchmarking, imp…☆186Updated 11 months ago