Alternatives and similar repositories for Crawling-Infrastructure:

Users that are interested in Crawling-Infrastructure are comparing it to the libraries listed below

• NikolaiT / se-scraper
  Javascript scraping module based on puppeteer for many different search engines...
  ☆559Updated 2 years ago
• NikolaiT / scrapeulous
  Cloud crawler functions for scrapeulous
  ☆45Updated 4 years ago
• NikolaiT / stealthy-scraping-tools
  Minimal set of tools to conduct stealthy scraping.
  ☆156Updated 2 years ago
• paulirish / headless-cat-n-mouse
  Is headless chrome currently detectable? Let's pit the detections and detection evasions against eachother.
  ☆656Updated 3 years ago
• apify / actor-scraper
  House of Apify Scrapers. Generic scraping actors with a simple UI to handle complex web crawling and scraping use cases.
  ☆121Updated 2 years ago
• chris124567 / puppeteer-bypassing-bot-detection
  Bypassing bot detection checks with Puppeteer.
  ☆93Updated 4 years ago
• unblocked-web / double-agent
  A test suite of common scraper detection techniques. See how detectable your scraper stack is.
  ☆137Updated 2 years ago
• apify / actor-page-analyzer
  Apify actor that opens a web page in headless Chrome and analyzes the HTML and JavaScript objects, looks for schema.org microdata and JSO…
  ☆150Updated 2 years ago
• NikolaiT / struktur
  Module that extracts structured information from a rendered html site and outputs JSON. HTML to JSON.
  ☆70Updated 3 years ago
• linkedtales / scrapedin
  LinkedIn Scraper (currently working 2020)
  ☆604Updated 2 years ago
• antoinevastel / fpscanner
  ☆567Updated 2 months ago
• prescience-data / foundation
  🧱 A uniform template to use as a foundation for Puppeteer bot construction.
  ☆66Updated 3 years ago
• linkedtales / scrapedin-linkedin-crawler
  Crawler for LinkedIn full profiles 2019
  ☆215Updated 4 years ago
• commoncrawl / cc-index-table
  Index Common Crawl archives in tabular format
  ☆118Updated last month
• freethenation / DFPM
  DFPM is a browser extension for detecting browser fingerprinting.
  ☆117Updated 2 years ago
• bitmakerla / estela
  estela, an elastic web scraping cluster 🕸
  ☆180Updated last month
• TeamHG-Memex / scrapy-rotating-proxies
  use multiple proxies with Scrapy
  ☆758Updated 2 years ago
• apify / browser-pool
  A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppetee…
  ☆94Updated 2 years ago
• apify / header-generator
  NodeJs package for generating browser-like headers.
  ☆70Updated 2 years ago
• scrapoxy / scrapoxy
  Scrapoxy is a super proxies manager that orchestrates all your proxies into one place, rather than spreading management across multiple s…
  ☆2,243Updated this week
• TheWebScrapingClub / TheScrapingClubFree
  The Web Scraping Club Free Repository
  ☆139Updated last week
• apify / got-scraping
  HTTP client made for scraping based on got.
  ☆668Updated last month
• zlurad / serp-parser
  Nodejs lib to parse Google SERP html pages
  ☆47Updated last year
• gajus / puppeteer-proxy
  Proxies Puppeteer Page requests.
  ☆208Updated 8 months ago
• intoli / user-agents
  A JavaScript library for generating random user agents with data that's updated daily.
  ☆1,047Updated this week
• danielgatis / puppeteer-recaptcha-solver
  Recaptcha solver for puppeteer.
  ☆603Updated last month
• apify / proxy-chain
  Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining.
  ☆909Updated last week
• antoinevastel / fp-collect
  Fingerprinting script of Fingerprint-Scanner
  ☆245Updated 2 months ago
• austinoboyle / scrape-linkedin-selenium
  `scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into struct…
  ☆485Updated 2 years ago
• prescience-data / puppeteer-botcheck
  🕵‍♂ Bot detection tests for Puppeteer. Hide and seek!
  ☆94Updated 2 years ago