A small tool which uses the CommonCrawl URL Index to download documents with certain file types or mime-types. This is used for mass-testing of frameworks like Apache POI and Apache Tika
☆74Oct 3, 2026Updated 3 weeks ago
Alternatives and similar repositories for CommonCrawlDocumentDownload
Users that are interested in CommonCrawlDocumentDownload are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Simplified version of a common crawl fetcher☆17Oct 12, 2026Updated 3 weeks ago
- A DropWizard wrapper around Apache Tika.☆10Dec 22, 2016Updated 9 years ago
- Single server/laptop grade file-observatory☆10Mar 30, 2023Updated 3 years ago
- Repository for revision of PREMIS OWL ontology group☆13May 12, 2022Updated 4 years ago
- Remove your comment history on Reddit as deleting an account does not do so.☆22Aug 20, 2019Updated 7 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Library Carpentry Workshop Overview☆11Jan 7, 2025Updated last year
- Collection of my Python Scripts☆41Aug 14, 2020Updated 6 years ago
- A repository containing the materials for a Python workshop a the Library of Congress in May 2019☆12May 23, 2019Updated 7 years ago
- Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends☆57Jan 28, 2024Updated 2 years ago
- Web Based GIS Application☆44Apr 7, 2016Updated 10 years ago
- An introduction to Web Programming for the Cautious and Weary☆13May 10, 2026Updated 5 months ago
- Use somatic mutations to choose a personalized cancer vaccine (tumor-specific immunogenic peptides)☆16Sep 23, 2016Updated 10 years ago
- Compressed Rich Text Format (RTF) compression and decompression in Python☆25Jun 29, 2025Updated last year
- An index of PDF-centric corpora☆188Oct 19, 2026Updated 2 weeks ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Scholarly Communications Workshops☆13Jun 7, 2018Updated 8 years ago
- convert NDNP data to IIIF☆12Jun 7, 2016Updated 10 years ago
- Index Common Crawl archives in tabular format☆132Oct 22, 2026Updated last week
- Utilities useful in cultural heritage imaging and mass digitization projects☆17Sep 10, 2020Updated 6 years ago
- ShEx schemas for common vocabularies and use cases.☆13Oct 7, 2019Updated 7 years ago
- Python for Humanities☆13Updated this week
- Advanced desktop search/corpus exploration prototype☆21Jun 23, 2021Updated 5 years ago
- Repository of documentation about the open datasets published by the UK Web Archive.☆15Jun 21, 2019Updated 7 years ago
- WARC (Web Archive) Input and Output Formats for Hadoop☆38Dec 7, 2014Updated 11 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Android application used to track the NYU bus system.☆45May 21, 2015Updated 11 years ago
- A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/☆202Oct 7, 2018Updated 8 years ago
- A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine☆220Oct 16, 2026Updated 2 weeks ago
- ☆12Oct 8, 2026Updated 3 weeks ago
- SiriProxy plugin that lets you order pizza.☆13Jul 16, 2022Updated 4 years ago
- Stanford CoreNLP NER addon for Apache Tika's NamerEntityParser☆13Feb 26, 2022Updated 4 years ago
- It's like DocBleach, but in your browser☆18Oct 24, 2019Updated 7 years ago
- Repository for course materials and related resources for the ARL Digital Scholarship Institute.☆23Aug 3, 2021Updated 5 years ago
- ☆35Nov 2, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- The OpenSextant Gazetteer is a collection of world-wide place name data☆12Feb 11, 2018Updated 8 years ago
- Common web archive utility code.☆66Oct 1, 2026Updated last month
- Library for Object Linking and Embedding (OLE) data types☆13Oct 9, 2026Updated 3 weeks ago
- Library and tools to access the (split) RAW image format☆12Oct 12, 2026Updated 3 weeks ago
- Web-based page layout editor created for EMOP (Early Modern OCR Project).☆11May 21, 2021Updated 5 years ago
- DocBleachShell is the integration of the great DocBleach, https://github.com/docbleach/DocBleach Content Disarm and Reconstruction tool i…☆21Jan 15, 2022Updated 4 years ago
- Codemeta paper.☆10Jul 10, 2017Updated 9 years ago