Alternatives and similar repositories for portia

Users that are interested in portia are comparing it to the libraries listed below

• binux / pyspider

  View on GitHub
  A Powerful Spider(Web Crawler) System in Python.
  ☆17,044Apr 30, 2024Updated last year
• scrapy / scrapely

  View on GitHub
  A pure-python HTML screen-scraping library
  ☆1,888Apr 4, 2022Updated 3 years ago
• scrapinghub / splash

  View on GitHub
  Lightweight, scriptable browser as a service with an HTTP API
  ☆4,199Aug 2, 2024Updated last year
• scrapy / scrapy

  View on GitHub
  Scrapy, a fast high-level web crawling & scraping framework for Python.
  ☆59,720Updated this week
• scrapy / scrapyd

  View on GitHub
  A service daemon to run Scrapy spiders
  ☆3,085Jan 16, 2026Updated last month
• scrapinghub / frontera

  View on GitHub
  A scalable frontier for web crawlers
  ☆1,325Jun 6, 2025Updated 8 months ago
• rmax / scrapy-redis

  View on GitHub
  Redis-based components for Scrapy.
  ☆5,648Jul 6, 2024Updated last year
• DormyMo / SpiderKeeper

  View on GitHub
  admin ui for scrapy/open source scrapinghub
  ☆2,778May 4, 2023Updated 2 years ago
• scrapy-plugins / scrapy-splash

  View on GitHub
  Scrapy+Splash for JavaScript integration
  ☆3,242Feb 11, 2025Updated last year
• Gerapy / Gerapy

  View on GitHub
  Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
  ☆3,500Oct 29, 2024Updated last year
• istresearch / scrapy-cluster

  View on GitHub
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,230Nov 7, 2023Updated 2 years ago
• my8100 / scrapydweb

  View on GitHub
  Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI.…
  ☆3,407Feb 19, 2025Updated 11 months ago
• qinxuye / cola

  View on GitHub
  A high-level distributed crawling framework.
  ☆1,506Jul 31, 2022Updated 3 years ago
• geekan / scrapy-examples

  View on GitHub
  Multifarious Scrapy examples. Spiders for alexa / amazon / douban / douyu / github / linkedin etc.
  ☆3,265Nov 3, 2023Updated 2 years ago
• codelucas / newspaper

  View on GitHub
  newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:
  ☆14,977Dec 6, 2025Updated 2 months ago
• lorien / grab

  View on GitHub
  Web Scraping Framework
  ☆2,452Sep 19, 2025Updated 4 months ago
• gnemoug / distribute_crawler

  View on GitHub
  使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现
  ☆3,253Apr 18, 2017Updated 8 years ago
• scrapinghub / scrapyrt

  View on GitHub
  HTTP API for Scrapy spiders
  ☆879Updated this week
• crawlab-team / crawlab

  View on GitHub
  Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
  ☆12,151Dec 5, 2025Updated 2 months ago
• aivarsk / scrapy-proxies

  View on GitHub
  Random proxy middleware for Scrapy
  ☆1,670Oct 1, 2019Updated 6 years ago
• BruceDone / awesome-crawler

  View on GitHub
  A collection of awesome web crawler,spider in different languages
  ☆7,104Jun 16, 2024Updated last year
• grangier / python-goose

  View on GitHub
  Html Content / Article Extractor, web scrapping lib in Python
  ☆4,061Dec 26, 2021Updated 4 years ago
• holgerd77 / django-dynamic-scraper

  View on GitHub
  Creating Scrapy scrapers via the Django admin interface
  ☆1,162Feb 19, 2022Updated 3 years ago
• getredash / redash

  View on GitHub
  Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.
  ☆28,221Updated this week
• apache / superset

  View on GitHub
  Apache Superset is a Data Visualization and Data Exploration Platform
  ☆70,505Updated this week
• clips / pattern

  View on GitHub
  Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.
  ☆8,854Jun 10, 2024Updated last year
• MechanicalSoup / MechanicalSoup

  View on GitHub
  A Python library for automating interaction with websites.
  ☆4,840Jan 9, 2026Updated last month
• lorien / awesome-web-scraping

  View on GitHub
  List of libraries, tools and APIs for web scraping and data processing.
  ☆7,764Jan 13, 2026Updated last month
• scrapy / scrapyd-client

  View on GitHub
  Command line client for Scrapyd server
  ☆778Dec 15, 2025Updated 2 months ago
• psf / requests-html

  View on GitHub
  Pythonic HTML Parsing for Humans™
  ☆13,877Apr 16, 2024Updated last year
• apache / airflow

  View on GitHub
  Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
  ☆44,274Updated this week
• sanic-org / sanic

  View on GitHub
  Accelerate your web app development | Build fast. Run fast.
  ☆18,637Jan 7, 2026Updated last month
• huginn / huginn

  View on GitHub
  Create agents that monitor and act on your behalf. Your agents are standing by!
  ☆48,688Updated this week
• explosion / spaCy

  View on GitHub
  💫 Industrial-strength Natural Language Processing (NLP) in Python
  ☆33,201Nov 27, 2025Updated 2 months ago
• metabase / metabase

  View on GitHub
  The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data
  ☆45,911Updated this week
• soimort / you-get

  View on GitHub
  Dumb downloader that scrapes the web
  ☆56,738Apr 27, 2025Updated 9 months ago
• scrapinghub / dateparser

  View on GitHub
  python parser for human readable dates
  ☆2,780Updated this week
• celery / celery

  View on GitHub
  Distributed Task Queue (development branch)
  ☆28,013Updated this week
• puppeteer / puppeteer

  View on GitHub
  JavaScript API for Chrome and Firefox
  ☆93,537Updated this week