NikolaiT / struktur
Module that extracts structured information from a rendered html site and outputs JSON. HTML to JSON.
☆69Updated 3 years ago
Related projects ⓘ
Alternatives and complementary repositories for struktur
- 🧱 A uniform template to use as a foundation for Puppeteer bot construction.☆62Updated 3 years ago
- Cloud crawler functions for scrapeulous☆44Updated 3 years ago
- Minimal set of tools to conduct stealthy scraping.☆150Updated last year
- Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues.☆415Updated last year
- A test suite of common scraper detection techniques. See how detectable your scraper stack is.☆135Updated 2 years ago
- DFPM is a browser extension for detecting browser fingerprinting.☆115Updated last year
- Automatically extracts structured information from webpages☆108Updated 2 years ago
- Apify actor that opens a web page in headless Chrome and analyzes the HTML and JavaScript objects, looks for schema.org microdata and JSO…☆150Updated last year
- Bypassing bot detection checks with Puppeteer.☆94Updated 4 years ago
- NodeJs package for generating browser-like headers.☆64Updated 2 years ago
- ☆107Updated 7 months ago
- Modern tests to detect automated browser behavior. Cover most important leaks from Puppeteer and Playwright.☆27Updated 2 weeks ago
- Fingerprinting script of Fingerprint-Scanner☆232Updated 7 months ago
- 🕵♂ Bot detection tests for Puppeteer. Hide and seek!☆83Updated last year
- A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppetee…☆89Updated last year
- Extract text from HTML☆130Updated 4 years ago
- Add-ons for Playwright: adblocker, stealth mode☆46Updated 3 years ago
- Changes a EC2 machine's IP address every ten minutes using Elastic IP☆81Updated 3 years ago
- 🛡🎭 A conceptual patch which modifies some vanilla puppeteer files to decrease detection rates.☆46Updated 3 years ago
- Index Common Crawl archives in tabular format☆106Updated last week
- 🔐 Tooling to access Puppeteer's internal Isolated World.☆18Updated 3 years ago
- Proxies Puppeteer Page requests.☆201Updated 2 months ago
- Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Support…☆107Updated last year
- A fork of Dragnet that also extract author, headline, date, keywords from context, as well as built in metadata extraction all in one pac…☆234Updated 10 months ago
- Advanced Node proxy checker (node proxy verifier, node proxy tester) with socks and https support☆108Updated 2 years ago
- Helps to extract shortest optimal css-selector and multi-selector.☆26Updated 7 years ago
- A suite of tools for protecting the web's open knowledge.☆130Updated last month
- Home of fingerprint injector.☆63Updated 2 years ago