A crawler can use different strategy to different url pattern
☆24Nov 24, 2016Updated 9 years ago
Alternatives and similar repositories for CrawlerX
Users that are interested in CrawlerX are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- a tool for "giab" (github issue as blog) guys☆11May 13, 2016Updated 10 years ago
- An Express middleware module that will redirect user HTTP requests that contain uppercase letters, to the same URL in lowercase form.☆14Feb 14, 2017Updated 9 years ago
- ☆23Dec 18, 2014Updated 11 years ago
- Get a random property from an object☆15Jul 2, 2024Updated 2 years ago
- Get a random key from an object☆18Jul 2, 2024Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Trampolines, to emulate tail-recursion.☆14Oct 31, 2018Updated 7 years ago
- Synchronously write a file and create its ancestor directories if needed☆14May 26, 2019Updated 7 years ago
- eBay SDK with Promise / Stream interface☆10Sep 30, 2016Updated 9 years ago
- node.js博客系统☆10May 5, 2015Updated 11 years ago
- A through2 wrapper to create an Array.prototype.filter analog for streams☆34Feb 28, 2026Updated 4 months ago
- JavaScript implementation of the Damerau-Levenshtein edit distance algorithm☆28May 9, 2012Updated 14 years ago
- 🚰 Lisk faucet☆13Jan 14, 2022Updated 4 years ago
- Compliant Promises/A+ implementation which doesn't actually work☆12Jan 1, 2026Updated 6 months ago
- Simple blockchain implementation in NodeJS/JavaScript☆13May 14, 2017Updated 9 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- 一个根据配置合并js文件的gulp插件，合并时会自动提取模块间的依赖☆27Nov 6, 2017Updated 8 years ago
- A through2 wrapper to for simple stream.PassThrough spies.☆21Jun 17, 2015Updated 11 years ago
- static file server middleware for connect. loads files once at startup and saves gzipped versions in memory☆18Apr 28, 2020Updated 6 years ago
- html to jade☆21Nov 6, 2015Updated 10 years ago
- i̛͇͚̠͊ͥͧ̊n̲̱̙͔̄̈́ͩ͠ͅv̤͛͐͊ͥͪo̘͆͛̑̾ͅk̘͔̪̹̪̜̖͇͋̈ͩ̕e̙̲͉̘͍̭̬ͪ̉ and banish zalgo☆10Nov 3, 2020Updated 5 years ago
- Ensure that when you `npm publish`, the "latest" tag is only set for the truly latest version.☆21Jan 13, 2026Updated 6 months ago
- A simple HTTP provider that follows the web3 specification.☆11Dec 17, 2016Updated 9 years ago
- A super simple module for making low level queries to the Ethereum RPC layer.☆13Dec 6, 2022Updated 3 years ago
- Your own Yandex Market Language (YML) generator, with blackjack and validation☆14Apr 26, 2025Updated last year
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- MVCTodoApp built with functors. Models redux & react-redux architecture.☆13Oct 12, 2016Updated 9 years ago
- Modern Buffer API polyfill without footguns☆27Apr 22, 2021Updated 5 years ago
- Unpipe a stream from all destinations☆22Dec 15, 2020Updated 5 years ago
- ☆16Oct 16, 2015Updated 10 years ago
- Client for the ETHLottery☆17Nov 4, 2022Updated 3 years ago
- 前端框架中的实用技术☆14Sep 26, 2014Updated 11 years ago
- AWS signing. Originally pulled from LearnBoost/knox, maintained as vendor in request, now a standalone module.☆30Jun 2, 2020Updated 6 years ago
- A priority job queue backed by redis, built for eggjs.☆10Jun 13, 2018Updated 8 years ago
- Serialize (encode) HTML character references☆21Apr 3, 2024Updated 2 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- A drop-in, low-overhead monitoring web dashboard for a linux machine.☆17Mar 25, 2024Updated 2 years ago
- 🏆 Blog with manager (vue&ssr)☆14Jan 7, 2023Updated 3 years ago
- gofmt for coffee☆10Apr 8, 2017Updated 9 years ago
- nodejs core exec() function thunk wrapper for "co"☆45Apr 22, 2014Updated 12 years ago
- The Lingotek Filesystem Connector (ltk) links your files and folders to the Translation Network™☆13May 7, 2026Updated 2 months ago
- 这里为"阿航的技术小站"的Flutter实战中所有的DEMO代码仓库☆11Jan 5, 2021Updated 5 years ago
- Node.js + express + mongodb 博客实例☆10Jan 5, 2017Updated 9 years ago