API definition, resources and reference implementation of URL Frontiers
☆67Sep 22, 2026Updated 2 weeks ago
Alternatives and similar repositories for url-frontier
Users that are interested in url-frontier are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A scalable, mature and versatile web crawler based on Apache Storm☆1,007Updated this week
- Internet Archive's Sparkling Data Processing Library☆17May 4, 2026Updated 5 months ago
- Charter proposal for an “RDF Dataset Canonicalization and Hash Working Group”☆11Dec 16, 2024Updated last year
- Download GitHub repositories☆13May 10, 2025Updated last year
- An evil web server.☆13May 9, 2015Updated 11 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- demos using the OpenRNDR framework☆13Mar 27, 2020Updated 6 years ago
- Original GOKb repo - Moving to https://github.com/openlibraryenvironment/gokb☆11Jan 23, 2018Updated 8 years ago
- visualizations/charts for media collections, based on mediainfo☆14Sep 15, 2022Updated 4 years ago
- ☆60Dec 11, 2025Updated 9 months ago
- MftReader is a Command-Line interface (CLI) program which reads the Master File Table (MFT) from NTFS volume. (C# Implementation with PIn…☆14Sep 13, 2018Updated 8 years ago
- A tool for collection archival slivers of the web and web archives☆20Jun 1, 2026Updated 4 months ago
- A Maven plugin for inspecting, decompiling, and instrumenting Java jar dependencies☆22Mar 27, 2026Updated 6 months ago
- Implementation of "Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions"☆25Aug 27, 2024Updated 2 years ago
- Hadoop-based tool for extraction of large scale synchronous grammars for paraphrasing and machine translation☆15Dec 2, 2016Updated 9 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Tools for preservation of floppy disks☆15Mar 25, 2026Updated 6 months ago
- Go OCFL Implementation☆19Updated this week
- Seeder - Czech webarchive curating tool and public site☆17Aug 27, 2026Updated last month
- A utility for staging files, calculating and validating file checksums, and comparing checksum values between storage locations.☆14Jul 10, 2023Updated 3 years ago
- Docker container for ocropus3 OCR system☆13Aug 19, 2018Updated 8 years ago
- The BES framework, which forms the basis for the Hyrax server☆17Oct 2, 2026Updated last week
- Useful tools to extract malayalam text from the Common Crawl Datasets☆28Jul 10, 2026Updated 3 months ago
- Apache Pony Mail☆15Updated this week
- A dataset downloaded from the deep and scientific web across three major Polar data centers for use in research.☆13Sep 8, 2017Updated 9 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Declarative syntax for defining sets of URLs. No need for error-prone regexs.☆21Apr 24, 2019Updated 7 years ago
- Specification for a query language to request Verifiable Presentations from wallets etc.☆10Apr 23, 2026Updated 5 months ago
- golang readers for ARC and WARC webarchive formats☆20Apr 3, 2023Updated 3 years ago
- code and data used to build a training dataset for dragnet models☆10Nov 29, 2020Updated 5 years ago
- Create CovJSON files from common scientific data formats☆14Apr 24, 2018Updated 8 years ago
- Open-Source Information Retrieval Reproducibility Challenge☆51Jan 11, 2016Updated 10 years ago
- A Java library for Stochastic Gradient Descent (SGD)☆23Nov 1, 2021Updated 4 years ago
- MirrorDataGenerator is a python tool that generates synthetic data based on user-specified causal relations among features in the data. I…☆25Jun 22, 2022Updated 4 years ago
- Citadel: Enterprise Search☆15May 2, 2023Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Mirror of Apache Edgent (Incubating) Samples☆15Feb 14, 2018Updated 8 years ago
- A platform for collecting, analyzing, and visualizing social media data.☆13Dec 27, 2020Updated 5 years ago
- An IETF specification for cryptographic hyperlinking☆15May 2, 2021Updated 5 years ago
- A nascent tutorial/intro to search engine ideas in Nim☆11May 24, 2023Updated 3 years ago
- Easily add authentication to your postgrest API☆25Feb 14, 2019Updated 7 years ago
- Sort-friendly URI Reordering Transform (SURT) python module☆47Sep 11, 2025Updated last year
- ✅ ♿️ A suite of end-to-end tests to check if your application is compliant with the WAI-ARIA authoring practices.☆16Jan 4, 2023Updated 3 years ago