Using Apache Airflow to schedule web scrapers
☆43Oct 3, 2018Updated 7 years ago
Alternatives and similar repositories for airflow-scraping
Users that are interested in airflow-scraping are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A plugin to Apache Airflow to allow you to run Zip and UnZip commands as an Operator☆12Jul 26, 2023Updated 2 years ago
- ☆18Jun 7, 2026Updated 2 weeks ago
- Internet radio as a service with liquidsoap and icecast wrapped with docker.☆20Nov 27, 2017Updated 8 years ago
- #A/B testing: A step-by-step guide in Python This is a walkthrough of how to design and analyse an A/B test using Python.☆16Aug 17, 2021Updated 4 years ago
- Blog post on ETL pipelines with Airflow☆23Aug 31, 2025Updated 9 months ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Official Keras example projects on Floyd☆11Mar 13, 2017Updated 9 years ago
- An Airflow Plugin that provides a new page to the standard Airflow Web Server to help you perform various operations☆12Nov 28, 2016Updated 9 years ago
- ☆11May 26, 2022Updated 4 years ago
- Swift wrapper for Google's Differential Privacy Project☆12May 8, 2020Updated 6 years ago
- Sharable Grakn knowledge graphs☆13Dec 28, 2022Updated 3 years ago
- Streaming Oracle Database 11g changes into NiFi with Debezium Connector☆12Jul 15, 2021Updated 4 years ago
- Digitized VHS Cassette Editing with Python☆12Mar 20, 2025Updated last year
- Screaming Frog SEO Spider Install Script by Fili (SEO Expert & ex-Google engineer)☆14Apr 12, 2021Updated 5 years ago
- Serving Uncertainty with Bayesian inference, using PyMC3 with Bodywork☆14Jun 21, 2022Updated 4 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Collection of DAGS for DCP database development and generation☆16Aug 29, 2017Updated 8 years ago
- Dead simple cron service for making HTTP calls on a regular schedule.☆14Jul 11, 2020Updated 5 years ago
- Airflow 2.0 configuration with Celery Executor based on Docker containers with Postgres and Redis broker plus Flower and Webserver UI☆15May 13, 2021Updated 5 years ago
- a Hadoop Map Reduce application that retrieves data/articles related to sports from sources like NY Times, Commoncrawl, and Twitter and c…☆13Oct 3, 2019Updated 6 years ago
- Create tables in Google BigQuery, auto-generate their schemas, and retrieve said schemas.☆10Jun 12, 2026Updated last week
- Helm Charts for the Astronomer Platform, Apache Airflow as a Service on Kubernetes☆490Updated this week
- A project to develop the R Shiny applications.☆12May 13, 2019Updated 7 years ago
- Numpyro examples in Python notebooks☆11Sep 7, 2020Updated 5 years ago
- PD patch for a granular sampling instrument that runs on a Bela in a custom box (or on a computer)☆11Nov 30, 2020Updated 5 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Step by Step Tutorial to run a PHP/MySQL website inside docker☆11Oct 2, 2020Updated 5 years ago
- Versatile Metrics Collection for Python☆20Jun 10, 2026Updated last week
- Script generates index.html files for s3 bucket which enables browser experience.☆13Feb 6, 2025Updated last year
- C inference engine for running GLiClass (Generalist and Lightweight Classification) models☆17May 21, 2025Updated last year
- Detects support for Cross-Origin Resource Sharing☆20Oct 8, 2018Updated 7 years ago
- ☆16Jun 24, 2023Updated 2 years ago
- Run Hadoop Cluster within Docker Containers.☆16Mar 6, 2025Updated last year
- A toy poker simulator with a pluggable Player interface to implement Agents that play using both rules based strategies and llms.☆11Jan 30, 2026Updated 4 months ago
- An opinionated implementation of exclusively using airflow DockerOperators for all Operators☆18Mar 14, 2022Updated 4 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Sandbox for Backstage Functions☆15Feb 8, 2024Updated 2 years ago
- Android app to bypass SSL certificate validation (Certificate Pinning).☆17Feb 7, 2016Updated 10 years ago
- This project transforms your terminal into an immersive Fallout-inspired experience, with a customized prompt, the display of Vault Boy A…☆15Apr 21, 2025Updated last year
- R package for Byte Pair Encoding based on YouTokenToMe☆16Jun 13, 2026Updated last week
- A tool for automating the editing and uploading process of YouTube music videos☆15May 23, 2020Updated 6 years ago
- Creating Data Pipelines with Apache Airflow to manage ETL from Amazon S3 into Amazon Redshift☆14Jun 12, 2019Updated 7 years ago
- Create your own powershell, malware desktop app or even clickjacking web with a single command for unix and windows systems.☆17May 26, 2019Updated 7 years ago