Florents-Tselai / WarcDB
WarcDB: Web crawl data as SQLite databases.
☆398Updated 8 months ago
Alternatives and similar repositories for WarcDB:
Users that are interested in WarcDB are comparing it to the libraries listed below
- A SQLite extension which loads a Google Sheet as a virtual table.☆516Updated 2 years ago
- A SQLite extension for reading large files line-by-line (NDJSON, logs, txt, etc.)☆398Updated last year
- A SQLite extension for querying, manipulating, and creating HTML elements.☆383Updated last year
- Query SQLite files in S3 using s3fs☆500Updated 2 years ago
- 🗄️ A simple CLI for converting WARC to Parquet.☆109Updated last month
- ☆114Updated 3 years ago
- Rsync-based time machine for Linux, written in Python, for local and remote backups.☆169Updated last year
- Module Oriented Large Archive Specialized Slow Exhaustive Searcher☆113Updated 9 years ago
- The various scripts I use to back up my home computers using ssh and rsync☆199Updated 3 years ago
- Distributed Embeddable Database☆256Updated 2 years ago
- A self-hosted live video streaming platform with Discord authentication, auto-recording and more!☆349Updated 8 months ago
- Create a SQLite database containing metadata from Google Drive☆156Updated 2 weeks ago
- Geocode rows in a SQLite database table☆235Updated 2 years ago
- Minimalist log collector☆114Updated 2 months ago
- ☆228Updated 2 years ago
- Unix-style filesystem-based RSS/Atom/JSON Feed fetcher/reader☆175Updated 3 months ago
- Python module to parse ingredient names. Splitting them into the ingredient, unit and quantity. It is trained on a publicly available dat…☆152Updated last year
- Scrapy rotation proxy package with advanced functions☆95Updated 2 years ago
- α-Indirect Control in Onion-like Networks☆149Updated last year
- Shell scripting for serverless☆141Updated 2 years ago
- Template repository for setting up shot-scraper☆253Updated 3 weeks ago
- DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with …☆814Updated 3 years ago
- Reverse Geocode for OpenStreetmap☆122Updated 6 months ago
- A self hosted recommendation feed generated from your browsing habits☆312Updated 2 years ago
- A simple pleasant build system in Python.☆361Updated last month
- Versioning filesystem for Sqlite☆237Updated 9 months ago
- Functional UUIDs for Python.☆148Updated 3 years ago
- The best RSS Search experience you can find☆626Updated 2 years ago
- Dockerized local and offline backing up of PostgresQL with rotation and compression.☆210Updated last year
- shell-based query tool☆164Updated last year