API definition, resources and reference implementation of URL Frontiers
☆65Aug 24, 2026Updated last week
Alternatives and similar repositories for url-frontier
Users that are interested in url-frontier are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Resources for running StormCrawler with Docker services☆10Nov 10, 2024Updated last year
- A set of reusable Java components that implement functionality common to any web crawler☆258Updated this week
- A scalable, mature and versatile web crawler based on Apache Storm☆994Updated this week
- Code and Data of the paper: "Redefining Retrieval Evaluation in the Era of LLMs"☆16Oct 27, 2025Updated 10 months ago
- Website sources for the Apache OpenNLP website☆10Aug 11, 2026Updated 2 weeks ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- SCAlable Preservation Environments☆40Jul 7, 2022Updated 4 years ago
- Add editing UI and other power-user features to Datasette.☆14Mar 4, 2023Updated 3 years ago
- A Text Classification API in Java originally developed by DigitalPebble Ltd. The API is independent from the ML implementations used and …☆48Sep 24, 2021Updated 4 years ago
- Apache OpenNLP Models☆17Updated this week
- CommonCrawl Test version of Nutch☆16Jul 10, 2014Updated 12 years ago
- An alternative approach for probabilistic topic modeling based on agglomerative clustering of topics (not documents)☆12Apr 14, 2021Updated 5 years ago
- Original GOKb repo - Moving to https://github.com/openlibraryenvironment/gokb☆11Jan 23, 2018Updated 8 years ago
- visualizations/charts for media collections, based on mediainfo☆14Sep 15, 2022Updated 3 years ago
- Scientific articles using or citing Common Crawl data☆30Jul 8, 2026Updated last month
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- A tool for collection archival slivers of the web and web archives☆19Jun 1, 2026Updated 2 months ago
- Hadoop-based tool for extraction of large scale synchronous grammars for paraphrasing and machine translation☆15Dec 2, 2016Updated 9 years ago
- Go OCFL Implementation☆18Aug 18, 2026Updated last week
- Repo for Live Coding at Droidcon Berlin☆10Sep 5, 2017Updated 8 years ago
- The geometry of multilingual language model representations (EMNLP 2022).☆22Oct 21, 2022Updated 3 years ago
- Natural language detection, Java bindings for CLD2☆18Aug 2, 2026Updated 3 weeks ago
- Common web archive utility code.☆65Aug 24, 2026Updated last week
- Hadoop integration code for working with with Apache cTAKES☆10Feb 11, 2014Updated 12 years ago
- ifcParserLib is a set of reusable Java components that implement functionality for IFC file parsing.☆10Oct 14, 2020Updated 5 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A utility for staging files, calculating and validating file checksums, and comparing checksum values between storage locations.☆14Jul 10, 2023Updated 3 years ago
- Docker container for ocropus3 OCR system☆13Aug 19, 2018Updated 8 years ago
- The BES framework, which forms the basis for the Hyrax server☆17Updated this week
- Solr and LucidWorks search user interface☆16Apr 29, 2013Updated 13 years ago
- Kylo integration with PDND (previously DAF).☆19Nov 16, 2022Updated 3 years ago
- Apache Pony Mail☆15Jul 29, 2026Updated last month
- A dataset downloaded from the deep and scientific web across three major Polar data centers for use in research.☆13Sep 8, 2017Updated 8 years ago
- The CMR Metadata Review tool is used to curate NASA EOSDIS collection and granule level metadata in CMR for correctness, completeness and…☆26Sep 4, 2025Updated 11 months ago
- ☆14Aug 23, 2026Updated last week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Repository for revision of PREMIS OWL ontology group☆13May 12, 2022Updated 4 years ago
- golang readers for ARC and WARC webarchive formats☆20Apr 3, 2023Updated 3 years ago
- code and data used to build a training dataset for dragnet models☆10Nov 29, 2020Updated 5 years ago
- Create CovJSON files from common scientific data formats☆14Apr 24, 2018Updated 8 years ago
- Open-Source Information Retrieval Reproducibility Challenge☆51Jan 11, 2016Updated 10 years ago
- A Java library for Stochastic Gradient Descent (SGD)☆22Nov 1, 2021Updated 4 years ago
- MirrorDataGenerator is a python tool that generates synthetic data based on user-specified causal relations among features in the data. I…☆25Jun 22, 2022Updated 4 years ago