API definition, resources and reference implementation of URL Frontiers
☆68Nov 13, 2026Updated this week
Alternatives and similar repositories for url-frontier
Users that are interested in url-frontier are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A small, memory efficient crawler written in Rust.☆10Oct 15, 2024Updated 2 years ago
- Resources for running StormCrawler with Docker services☆10Nov 10, 2024Updated 2 years ago
- A scalable, mature and versatile web crawler based on Apache Storm☆1,015Updated this week
- A set of reusable Java components that implement functionality common to any web crawler☆260Nov 12, 2026Updated last week
- A Text Classification API in Java originally developed by DigitalPebble Ltd. The API is independent from the ML implementations used and …☆48Sep 24, 2021Updated 5 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Onyx plugin for ElasticSearch☆18Feb 13, 2018Updated 8 years ago
- A Nutch 2.2.1 plugin which allows users to shuffle off the responsibility for retrieving pages to a selenium hub/node spoke system. This …☆16Jun 9, 2016Updated 10 years ago
- Index Common Crawl archives in tabular format☆132Nov 12, 2026Updated last week
- CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop☆38Oct 1, 2026Updated last month
- Codes and dashboards for 4th place solution for Kaggle's Home Credit Default Risk competition☆34Sep 9, 2018Updated 8 years ago
- The tiniest web-framework, ever.☆12Jan 6, 2009Updated 17 years ago
- ☆12Jun 10, 2025Updated last year
- Boss people around. No suit required.☆13Jun 26, 2017Updated 9 years ago
- Domain Ontology to model web archiving artefacts as Semantic objects and describe web archiving and web preservation activities.☆12Sep 3, 2026Updated 2 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- An asynchronus connection adapter for RSolr☆13Mar 3, 2010Updated 16 years ago
- Trough: Big data, small databases.☆43Jul 25, 2024Updated 2 years ago
- demos using the OpenRNDR framework☆13Mar 27, 2020Updated 6 years ago
- Download GitHub repositories☆13May 10, 2025Updated last year
- VLM pipeline to extract executive compensation data from SEC DEF 14A proxy statements. Uses VLM for table classification and LLM for stru…☆11Feb 4, 2026Updated 9 months ago
- Neural Solr = Solr 9 + Mighty Inference + Node☆18Jun 9, 2022Updated 4 years ago
- Kontrol is a small web framework written in Ruby, which runs directly on Rack.☆19Jan 26, 2013Updated 13 years ago
- A search and presentation system for various data formats (ead, tei, images etc.)☆14Dec 21, 2009Updated 16 years ago
- visualizations/charts for media collections, based on mediainfo☆14Sep 15, 2022Updated 4 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Storm / Solr Integration☆19Feb 2, 2024Updated 2 years ago
- A Maven plugin for inspecting, decompiling, and instrumenting Java jar dependencies☆23Mar 27, 2026Updated 7 months ago
- [ABANDONED] A drop-in replacement for Builder::XmlMarkup which uses libxml for speed and security.☆15Aug 5, 2008Updated 18 years ago
- A library to mutate parquet files☆20May 9, 2023Updated 3 years ago
- 🔗 An Arweave tag specification for cross-chain wallet linking☆11Jul 27, 2023Updated 3 years ago
- Tools for preservation of floppy disks☆15Mar 25, 2026Updated 7 months ago
- The fingerprinting library for Rust☆14Mar 30, 2026Updated 7 months ago
- Original GOKb repo - Moving to https://github.com/openlibraryenvironment/gokb☆11Jan 23, 2018Updated 8 years ago
- domain name ownership verification library written in Go ✔️☆29Sep 6, 2023Updated 3 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Java library for reading and writing WARC files with a typed API☆64Sep 1, 2026Updated 2 months ago
- A tool for collection archival slivers of the web and web archives☆20Jun 1, 2026Updated 5 months ago
- A data management system for electronic tags on marine animals☆13Mar 31, 2025Updated last year
- Log4j2 Appender that ships logs using HTTPs bulk☆11Nov 9, 2026Updated last week
- code and data used to build a training dataset for dragnet models☆10Nov 29, 2020Updated 5 years ago
- GPU-accelerated Python implementation of Stanford OpenIE☆11Jun 12, 2026Updated 5 months ago
- This repository contains the good practice to implement shipment tracking with ONE Record☆12Mar 8, 2024Updated 2 years ago
Are these results useful?