ZenRows / scaling-to-distributed-crawling
Repository for the Mastering Web Scraping in Python: Scaling to Distributed Crawling blogpost with the final code.
☆40Updated 2 years ago
Related projects: ⓘ
- Web scraping Page Objects core library☆93Updated 2 months ago
- 🏗️ Create APIs from CSV files within seconds, using fastapi☆78Updated 3 years ago
- Building a Concurrent Web Scraper with Python and Selenium☆35Updated 2 years ago
- Python bindings for Upwork API (OAuth2)☆35Updated last year
- ⚠️ Development moved to Sourcehut☆51Updated last year
- Scrapfly Python SDK for headless browsers and proxy rotation☆30Updated this week
- Learn how to scrape websites with Python, Selenium, Requests HTML, Celery, FastAPI, & NoSQL with Cassandra via AstraDB.☆89Updated 2 years ago
- FastAPI-PostgreSQL-Celery-RabbitMQ-Redis bakcend with Docker containerization☆68Updated last year
- Code examples on how to integrate various types of scrapers with Scraper API.☆23Updated 3 years ago
- Page Object pattern for Scrapy☆119Updated 2 months ago
- Product scraping from Walmart Canada website, with further cleaning and integration of data from a different store.☆15Updated last year
- A simple OCR tool made using FastAPI and Tesseract☆42Updated 3 years ago
- Async bulk data ingestion and querying in various document, graph and vector databases via their Python clients☆32Updated 10 months ago
- Zyte Automatic Extraction integration for Scrapy☆55Updated 2 years ago
- ☆18Updated 3 years ago
- Python clients for Zyte AutoExtract API☆39Updated 2 years ago
- FastAPI with Docker and Traefik☆105Updated last year
- deploying an ML model to Heroku with FastAPI☆48Updated 5 months ago
- Library that helps use puppeteer in scrapy.☆51Updated this week
- List of automatically rated Python packages for FastAPI.☆14Updated last week
- A Python library to interact with ScrapingBee's API for headless browsers and proxy rotation☆19Updated 2 weeks ago
- Python client for Zyte API☆19Updated 3 months ago
- Scrapy extension that gives you all the scraping monitoring, alerting, scheduling, and data validation you will need straight out of the…☆36Updated last month
- JavaScript support and proxy rotation for Scrapy with ScrapingBee.☆34Updated 4 months ago
- A Minimalist End-to-End Scrapy Tutorial☆69Updated 2 years ago
- Spider templates for automatic crawlers.☆19Updated this week
- Fast API SAAS Base App☆70Updated 3 years ago
- Backend, modern REST API for obtaining match and odds data crawled from multiple sites. Using FastAPI, MongoDB as database, Motor as asyn…☆61Updated last year
- Working example for serving a ML model using FastAPI and Celery☆71Updated 2 years ago
- FastAPI + GraphQL☆47Updated last year