Support for writing WARC files with Scrapy
☆24Dec 21, 2019Updated 6 years ago
Alternatives and similar repositories for scrapy-warcio
Users that are interested in scrapy-warcio are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆13Dec 28, 2022Updated 3 years ago
- Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)☆171Aug 18, 2025Updated 7 months ago
- sign elf binaries with GPG☆17Oct 10, 2016Updated 9 years ago
- A command line utility for listing and searching snapshots in web archives☆17Dec 21, 2023Updated 2 years ago
- A multi App to download file from LibGen.io☆12Aug 5, 2019Updated 6 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Streaming WARC/ARC library for fast web archive IO☆452Updated this week
- a simple httpd daemon in python, demonstrating socket activation via systemd☆16Jul 7, 2016Updated 9 years ago
- Basis for constructing a new project on top of mu.semte.ch☆16Mar 1, 2026Updated 3 weeks ago
- Scrape and structure raw data from the Norwegian parliament's API.☆12Oct 24, 2025Updated 5 months ago
- Frontend for ipfs-search.com☆25Oct 7, 2023Updated 2 years ago
- Comparing warc files☆17Feb 21, 2019Updated 7 years ago
- Gemini web proxy☆13Feb 20, 2025Updated last year
- Scorpion Protocol/File-Format☆20Updated this week
- Command line tool for digging into WARC files☆51Updated this week
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- broad template licenses for software that allow or prohibit use for specific purposes☆23Jan 21, 2022Updated 4 years ago
- ✨ The Open-Source WorldServer☆12Mar 4, 2026Updated 3 weeks ago
- Single file C header for UTF-x-to-y conversions + helpers☆13Jun 11, 2023Updated 2 years ago
- Hacker news on Console with auto classifer and recommender in reactjs style code☆15Aug 29, 2019Updated 6 years ago
- silly game over telnet☆12May 31, 2017Updated 8 years ago
- A trend viewer written in Python/JavaScript☆21Nov 15, 2024Updated last year
- Ember addon wrapping an RDFa editor with a public API☆17Updated this week
- PlayStation GPU (WIP)☆18Oct 3, 2023Updated 2 years ago
- Converts HTTrack crawls to WARC files☆34Aug 6, 2024Updated last year
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A fast python implementation of the SimHash algorithm.☆27Oct 27, 2021Updated 4 years ago
- CDXJ Indexing of WARC/ARCs☆33Dec 10, 2024Updated last year
- Sublime Text API Version Documenter☆11Jan 3, 2023Updated 3 years ago
- Official Python package for ArchiveBox, the self-hosted internet archiving solution.☆12Oct 5, 2024Updated last year
- https://git.gir.st/minesVIiper.git - Seldomly Updated Mirror: A minesweeper clone with vi keybindings; https://gir.st/mines.htm☆13Oct 27, 2023Updated 2 years ago
- Building static binaries of some tools using an Alpine chroot with musl☆37Nov 8, 2025Updated 4 months ago
- a static html and gemini mail archive for the 21st century, written in Rust☆28Aug 15, 2024Updated last year
- Gopher+ protocol specification☆16Feb 23, 2021Updated 5 years ago
- Nondestructive warc-in-tar to warc conversion☆27Apr 21, 2013Updated 12 years ago
- DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- A collection of tools for archiving and analysing the internet.☆78Jul 6, 2022Updated 3 years ago
- 4D Miner C++ Modding Headers / 4D-Modding API Headers☆12Mar 8, 2026Updated 3 weeks ago
- 🎭 An introduction to the Internet Archiving ecosystem, tooling, and some of the ethical dilemmas that the community faces.☆57Aug 15, 2024Updated last year
- This repository is a ASM->CPP translation of NFSIISE☆11Mar 15, 2024Updated 2 years ago
- StumpWM Debugger☆11Apr 19, 2025Updated 11 months ago
- A GUI based gopher (protocol) client☆23Dec 9, 2019Updated 6 years ago
- ☆51Aug 18, 2024Updated last year