alexksikes / mass-scrapingLinks
Quickly download and scrape websites on a massive scale.
☆65Updated 12 years ago
Alternatives and similar repositories for mass-scraping
Users that are interested in mass-scraping are comparing it to the libraries listed below
Sorting:
- Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends☆57Updated last year
- Site Hound (previously THH) is a Domain Discovery Tool☆23Updated 4 years ago
- Scrapes sites. Gets news. Eventually events.☆87Updated 9 years ago
- Scrape google search results☆94Updated 6 years ago
- Extract social media links and account names from websites.☆38Updated 5 years ago
- Python library with common functionality for writing web scrapers☆102Updated 9 years ago
- Automates the process of repeatedly searching for a website via scraped proxy IP and search keywords☆45Updated last year
- Get data about companies from advanced search without the use of API☆63Updated 5 years ago
- Social media monitoring tools such as sentiment analysis, keyword tracking and more☆48Updated 11 years ago
- ScraperWiki Python library for scraping and saving data☆159Updated 2 years ago
- Broad crawler for domain discovery☆19Updated 7 years ago
- Data analytics tool that tracks trending Etsy listings and analyzes tag frequencies to provide SEO insights. Helps shop owners optimize t…☆34Updated 5 months ago
- A library to parse Wayback Machine of archive.org to get a historical views of web pages. It is a useful tool to research on the evolutio…☆20Updated 6 years ago
- [UNMAINTAINED] Firefox addon for Scrapely☆5Updated 9 years ago
- framework for scraping legislative/government data☆86Updated 9 months ago
- Google SEO scraper for "allintitle:keyword" queries.☆22Updated 10 years ago
- A scraper for videos that are trending on YouTube (https://www.youtube.com/feed/trending)☆26Updated 3 years ago
- Scrapy middleware which allows to crawl only new content☆79Updated 2 years ago
- A library to interface with the Linkscape API.☆40Updated 6 years ago
- API - extract a list of keywords from a text.☆18Updated 7 years ago
- Deviant Spy is a native advertising (RevContent) spy tool☆31Updated 6 years ago
- Aiohttp web server API, which scrapes Google and returns scrape results as response. Supports proxies, multiple geos and number of result…☆56Updated last year
- A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.☆116Updated last year
- Paginating the web☆37Updated 11 years ago
- ☆36Updated last year
- Crawl and scrape Yelp's restaurant data for every zip code in the United States (or a specified zipcode). Yelp Crawler.☆56Updated 8 years ago
- A project to attempt to automatically login to a website given a single seed☆124Updated 2 years ago
- A component that tries to avoid downloading duplicate content☆27Updated 7 years ago
- Web Page Inspection Tool UI. Google SERP Preview, Sentiment Analysis, Keyword Extraction, Named Entity Recognition & Spell Check☆24Updated 2 years ago
- Source real estate prices from the Common Crawl.☆27Updated 6 years ago