🌐 OpenCrawl: An ethical, high-performance web crawler built for scale A powerful web crawling library that respects robots.txt and rate limits while leveraging Kafka for high-throughput data processing. Built with ethics and efficiency in mind.
☆28Apr 3, 2025Updated last year
Alternatives and similar repositories for OpenCrawl
Users that are interested in OpenCrawl are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Ernie-AI-Windows is an innovative artificial intelligence software tailored specifically for Windows operating systems, delivering cuttin…☆56Apr 5, 2024Updated 2 years ago
- The fastest FlashText library for Python☆26Jul 4, 2024Updated 2 years ago
- ⚡ Bhumi – The fastest AI inference client for Python, built with Rust for unmatched speed, efficiency, and scalability 🚀☆68Jan 22, 2026Updated 8 months ago
- Welcome to Kokoro, a high-quality text-to-speech synthesis program powered by deep learning. This tool converts any text into high-fideli…☆15Jan 5, 2026Updated 9 months ago
- Public results and task definitions for FrontierHarness Eval☆304Sep 8, 2026Updated last month
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Standalone, local-runnable binaries of popular linux distributions☆10Dec 15, 2021Updated 4 years ago
- ☆15Jan 17, 2025Updated last year
- Infrastructure useful to create natural language processing systems based on transformer networks☆12Sep 26, 2019Updated 7 years ago
- A convenient way to write text and track versions.☆13Feb 15, 2025Updated last year
- ☆27Aug 5, 2026Updated 2 months ago
- A UI designer for constructing AI applications with OpenSearch☆17Sep 30, 2026Updated last week
- Official Python SDK for WaveSpeed — image, video, audio and 3D generation via the wavespeed.ai API☆28Sep 21, 2026Updated 2 weeks ago
- A plugin to encrypt Opensearch data at rest (on linux distributions). A key can be defined for each index level granularity and data at r…☆17Aug 6, 2026Updated 2 months ago
- An offline CPU-first low-resource chat application to perform RAG on your corpus of data. Powered by OpenChat and CTranslate2.☆15Sep 28, 2026Updated last week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- A beautiful desktop application for Local-NotebookLM that provides an intuitive UI for converting PDFs to engaging audio content.☆29Apr 1, 2026Updated 6 months ago
- ☆16May 28, 2025Updated last year
- Library for Object Linking and Embedding (OLE) data types☆13Updated this week
- qt page router☆13May 7, 2024Updated 2 years ago
- ☆24Jul 2, 2025Updated last year
- Trigram tokenizer module for SQLite FTS5☆14Feb 22, 2021Updated 5 years ago
- tools for creating, inspecting and modifying torrent files☆14Jun 1, 2022Updated 4 years ago
- 10 years after the downfall of the TRiAS GmbH in Germany it's definitely time to release the sources of the TRiAS Spatial Information Sys…☆11Nov 11, 2012Updated 13 years ago
- Oracle Berkeley DB 18.x (Private copy of official distribution)☆11Jul 23, 2020Updated 6 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- ☆15Dec 12, 2022Updated 3 years ago
- Dependency-free and header-only C++11 XGen XPD cache I/O library.☆13Jun 23, 2019Updated 7 years ago
- RadioLLM: Introducing Large Language Model into Cognitive Radio via Hybrid Prompt and Token Reprogrammings☆19Jun 14, 2025Updated last year
- Mirror of git://git.code.sf.net/p/libwpd/librevenge☆13Feb 2, 2016Updated 10 years ago
- An invisible AI assistant for Linux that responds to your key combos, bringing thoughts to life — one keystroke at a time.☆20Jun 27, 2025Updated last year
- ☆23Jul 15, 2025Updated last year
- 📖 Convert a yaml file to bib file with the correct journal abbreviations.☆17Updated this week
- Search Relevance Workbench for OpenSearch.☆16Updated this week
- API specification for OpenSearch☆63Sep 24, 2026Updated 2 weeks ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A simple process query/manipulation tool using driver hooked system call. (2019)☆10Aug 30, 2021Updated 5 years ago
- Python module to interact with THOR Thunderstorm service☆18Dec 5, 2025Updated 10 months ago
- Scrapy framework based software to scrape the LinkedIn directory and company profiles.☆51Jul 31, 2026Updated 2 months ago
- From Doxygen documentation, create a Docset for use in Dash or Zeal.☆28Aug 1, 2026Updated 2 months ago
- A small cli tool to listen to dark mode changes on macOS Mojave☆13Jan 18, 2020Updated 6 years ago
- Another character management interface for SillyTavern.☆27Aug 16, 2026Updated last month
- SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data☆26Jan 24, 2026Updated 8 months ago